features that depend upon features (composite features)

[johnnyreilly]

Heya!

I'm a keen user of dapr. I generally do this in the context of a devcontainer. I'm interested in how to migrate to doing the same using features. If I read things right, I need to use the docker-in-docker feature. On top of that I think I'd need to have some kind of dapr feature which would contain something like this:

RUN apt-get update \
    && apt-get install python3-pip -y \
    # Install Dapr
    && wget -q https://raw.githubusercontent.com/dapr/cli/master/install/install.sh -O - | /bin/bash \
    # Clean up
    && apt-get autoremove -y && apt-get clean -y && rm -rf /var/lib/apt/lists/* 

# Add daprd to the path for the VS Code Dapr extension.
ENV PATH="${PATH}:/home/${USERNAME}/.dapr/bin"

I'm guessing that there's some way to have "dependencies" for features? So the dapr feature would depend upon docker-in-docker. Am I headed in the right direction here? Or am I ... not?

It's possible that this is related: devcontainers/cli#163

[joshspicer]

Hey, John Reilly (@johnnyreilly)! With the current Features spec as it stands today, the default behavior is that the implementing tool decides the installation order. Our current goal with Features is that installation of one is atomic and idempotent (although we recognize that might not always be possible).

Currently we have two mechanisms to override the default installation order - they can be found in the "Installation Order" section of the spec.

I'm not too familiar with "dapr", but if "dapr" strictly requires the "docker-in-docker" Feature ahead of time, it sounds like the "installsAfter" property might be a good fit.

(We'd love your feedback either way on how to improve the Features spec)

[johnnyreilly]

Interesting. So for my own use case, I'd be looking to use docker-in-docker and building upon it with dapr. So I'd use "installsAfter" quite comfortably. But it's possible people might not be using docker-in-docker, in which case would this create a problem?

My guess is not , assuming installsAfter doesn't create a hard dependency on the mentioned features being installed. That assumption may not be valid though?

Where I am right now, I have quite a complex devcontainer setup, and I think features could massively simplify it

If you look at the devcontainer in this repo: https://github.com/johnnyreilly/dapr-devcontainer-debug-and-deploy

You'll see it's somewhat involved, with extra script files and whatnot. I think with features, that in the end that becomes much simpler. I just need:

• dotnet
• node
• docker in docker
• dapr

Would it help to flesh this out if I attempted to contribute a dapr feature to the repo? I could provide it with an installsAfter of docker-in-docker and see how it goes?

I do wonder if installsAfter should be opt-outable for people who are satisfying that dependency another way.

[Chuxel]

Yeah installsAfter affects ordering but does not create a direct dependency that forces the download. That allows you to alter behavior based on whether or not something has already happened. e.g., if you need python, you can say that your feature happens after the python feature, and if there's no python by the time your feature executes, then you can install the system one to meet the need. That avoids issues with multiple versions of something being installed and inflating an image size unexpectedly.

You can also ignore that ordering by specifying overrideFeatureInstallOrder in devcontainer.json as an opt-out to your point there.

However, there is the idea of a "composite" feature we could explore. In this case, your feature would have a hard dependency on another feature (likely a specific version of it as well). The thing we'd need to iron out is what happens when someone also specifies that dependency directly.

One possibility is it could behave like #44, another would be that it literally fires twice, but then the dependent feature would need to be designed to handle that.

Let's move this discussion to the spec repo on the general topic of a composite feature.

[AlexanderLanin]

I was missing hard-dependencies since I started experimenting with features, but actually that would be a bad idea. The whole point of features is to avoid god like Dockerfiles. We want small reusable pieces. Dependencies between features make it impossible to use a different feature to install python. Or maybe it's already in my base image and I don't even need a feature at all.

My solution for the moment is to detect whether python is installed and in the correct version. If not, my feature exits with an error message telling the user to add python to the Dockerfile or to use *any* python feature.

Assuming features are named reasonably, what I would love is to have installsAfter: "*python*"

[joshspicer]

Assuming features are named reasonably, what I would love is to have installsAfter: "python"

Using a regex is an interesting idea. It would be assuming a lot, but I bet it would be right the majority of the time (and I suppose any hint is better than random?)

cc/ Christof Marti (@chrmarti) If not regexes, could be interesting to explore how else to drop general hints about install order.

[AlexanderLanin]

How about fallback to dependencies? Maybe that's some middle ground between hard dependencies and soft dependency order?

What I currently do:

if ! type python3 >/dev/null 2>&1; then
  echo "ERROR: Use a base image with python3 installed, add it to your Dockerfile or use a python feature like https://github.com/devcontainers/features/tree/main/src/python"
  exit 1
fi

How about a silent fallback, somehow supported by cli?

if ! type python3 >/dev/null 2>&1; then
  echo "INFO: python3 not installed, adding default python installation..."
  import ghcr.io/devcontainers/features/python:1
fi

[gund]

Hey, I just want to add one more use-case to the "features composition" proposal.

Basically I would like to be able to create a feature aggregate that will simply install other features, you can think of this as having a "preset" feature with maybe options to set versions and/or options of the other (sub)features it's going to install.
This is basically what I would want to do to create my own custom dev environment in a single feature that I can take as a base and optionally add more features on top of it in different projects.

So maybe nesting infinitely many features inside features is not a very nice idea, so instead we could come up with the "preset features" that have a special ability to simply configure which features does it consists of and what it's default versions/options are, and then on the consuming side a user would be able to configure each sub-feature version/options as needed when using this "preset feature".

What do you think about this idea?