Skip to content

Nixidy Architecture Guide for Contributors#

This guide provides a comprehensive overview of the nixidy codebase to help new contributors understand the project structure, key concepts, and development workflow.

Table of Contents#

Project Structure#

nixidy/
├── actions/                # GitHub Actions (build, switch)
├── cli/                    # Python-based CLI tool
├── docs/                   # Documentation (MkDocs)
│   ├── developer_guide/    # Contributor-facing documentation
│   └── user_guide/         # User-facing documentation
├── lib/                    # Nix function library
│   ├── helpers/            # Helper scripts (e.g. update-chart.sh)
│   ├── default.nix         # Library entry point
│   ├── helm.nix            # Helm-related functions
│   ├── kube.nix            # Kubernetes utility functions
│   ├── kustomize.nix       # Kustomize-related functions
│   └── tests.nix           # Library unit tests
├── modules/                # NixOS-style modules
│   ├── applications/       # Application submodule
│   │   ├── default.nix     # Application options and config
│   │   ├── argocd.nix      # ArgoCD options (syncPolicy, destination, ...)
│   │   ├── helm.nix        # Helm release processing
│   │   ├── kustomize.nix   # Kustomize processing
│   │   ├── lib.nix         # Application helper functions
│   │   ├── objects.nix     # Resource type registry and final object list
│   │   └── yamls.nix       # Raw YAML processing
│   ├── generated/          # Auto-generated resource options
│   │   ├── argocd.nix      # ArgoCD CRD options
│   │   └── k8s/            # Kubernetes resource options by version
│   ├── nixidy/             # Core nixidy configuration
│   │   ├── default.nix     # Core options (env, target, charts, ...)
│   │   ├── app-of-apps.nix # App-of-apps pattern generation
│   │   ├── defaults.nix    # Default settings (`nixidy.defaults`)
│   │   ├── extra-files.nix # Extra files configuration
│   │   └── transforms.nix  # objectTransforms rule engine
│   ├── testing/            # Testing framework modules
│   │   ├── default.nix     # Test suite configuration
│   │   └── eval.nix        # Test evaluation logic
│   ├── build/              # Build output packages
│   │   ├── default.nix     # Option surface, layout wiring, assertions
│   │   ├── layout.nix      # Pure per-application [FileSpec] core
│   │   ├── render.nix      # FileSpec -> shell fragment
│   │   ├── emit-environment.nix  # environment/activation/bootstrap/extras
│   │   └── apply.nix       # Apply path + activation post-process
│   ├── applications.nix    # Main applications option
│   ├── default.nix         # Module entry point
│   ├── modules.nix         # Module list
│   └── templates.nix       # Template system
├── pkgs/                   # Nix packages and generators
│   └── generators/         # CRD and K8s schema generators
│       ├── compile/            # Schema → options compilation
│       │   ├── walk.nix            # Shared schema traversal (backend-parameterized)
│       │   ├── backend-text.nix    # Text backend: emits Nix source
│       │   ├── backend-value.nix   # Value backend: live module values
│       │   ├── generator.nix       # File assembler (text backend → .nix file)
│       │   ├── module.nix          # Module assembler (value backend → module value)
│       │   └── runtime.nix         # Runtime helpers as values (for module.nix)
│       ├── sources/            # Schema acquisition
│       │   ├── versions.nix        # Kubernetes versions config
│       │   ├── k8s.nix             # Kubernetes OpenAPI spec acquisition
│       │   ├── crd.nix             # CRD file acquisition
│       │   └── chart.nix           # Helm chart CRD acquisition
│       ├── crd2jsonschema.py       # CRD to JSON schema converter
│       ├── default.nix             # Generator entry point (accessors)
│       ├── equiv-test.nix          # Accessor equivalence checks
│       └── test_crd2jsonschema.py  # crd2jsonschema unit tests
├── tests/                  # Module unit tests
│   ├── helm/               # Helm-specific tests
│   ├── kustomize/          # Kustomize-specific tests
│   └── *.nix               # Individual test files
├── flake.nix               # Nix flake definition
└── default.nix             # Non-flake entry point

Core Concepts#

1. NixOS Module System#

Nixidy is built on top of the NixOS module system. If you're unfamiliar with it, read the NixOS Module System documentation first.

Key concepts:

  • Options: Define the configuration interface with types, defaults, and descriptions
  • Config: Contains the actual configuration values after module evaluation
  • Imports: Modules can import other modules to extend functionality
  • Special Args: Additional arguments passed to all modules (lib, pkgs, config, etc.)

2. Application Lifecycle#

User Config → Module Evaluation → Resource Processing → Manifest Generation → Output
  1. User Configuration: Nix files defining applications and resources
  2. Module Evaluation: NixOS module system merges all configurations
  3. Resource Processing: Helm/Kustomize/YAML converted to typed resources
  4. Manifest Generation: Resources serialized to YAML files
  5. Output: Packages created for different deployment strategies

3. Resource Type System#

Resources are organized by Group/Version/Kind (GVK):

resources.<group>.<version>.<kind>.<name> = { ... };

# Examples:
resources.core.v1.ConfigMap.my-config = { ... };
resources.apps.v1.Deployment.nginx = { ... };
resources."networking.k8s.io".v1.Ingress.main = { ... };

Aliases provide convenient access: 

resources.configMaps.my-config = { ... };      # → core.v1.ConfigMap
resources.deployments.nginx = { ... };          # → apps.v1.Deployment
resources.ingresses.main = { ... };             # → networking.k8s.io.v1.Ingress

4. Processing Pipeline#

All input sources are normalized to typed nix resources:

┌─────────────────┐
│  Helm Charts    │──┐
├─────────────────┤  │
│  Kustomize      │──┤──→ [GVK Classification] ──→ [Typed Nix Resources] ──→ [YAML Output]
├─────────────────┤  │
│  Raw YAML       │──┘
└─────────────────┘

Module System Deep Dive#

Main Modules#

modules/applications.nix#

Defines the top-level applications option:

{
  options.applications = mkOption {
    type = attrsOf (submoduleWith {
      modules = [ ./applications ] ++ config.nixidy.applicationImports;
      specialArgs.nixidyDefaults = config.nixidy.defaults;
    });
  };
}

Key responsibilities:

  • Creates application submodules
  • Manages Kubernetes version selection (nixidy.k8sVersion)
  • Imports generated resource options

modules/nixidy/#

Core nixidy configuration, split across several files. default.nix defines the main options and imports the rest:

{
  options.nixidy = {
    env = mkOption { ... };                    # Environment name
    target.repository = mkOption { ... };      # Git repository URL
    target.branch = mkOption { ... };          # Target branch
    target.rootPath = mkOption { ... };        # Root path for manifests
    build.revision = mkOption { ... };         # Revision written to `.revision`
    objectTransforms = mkOption { ... };       # Env-wide transform rules
    charts = mkOption { ... };                 # Helm chart sources
    chartsDir = mkOption { ... };              # Directory to build charts from
  };
}

The directory also contains:

  • defaults.nix: default settings (nixidy.defaults, e.g. default transformers)
  • app-of-apps.nix: bootstrap app-of-apps pattern generation
  • extra-files.nix: extra files (nixidy.extraFiles) rendered into the output
  • transforms.nix: the objectTransforms rule type, matcher, and assertions

Also handled here:

  • Chart attribute set building (from nixidy.chartsDir)
  • Public apps list management
  • Global assertions and warnings

modules/applications/default.nix#

Defines individual application options:

{
  options = {
    name = mkOption { ... };
    namespace = mkOption { ... };
    createNamespace = mkOption { ... };
    project = mkOption { ... };
    annotations = mkOption { ... };
    labels = mkOption { ... };
    objectTransforms = mkOption { ... };  # App-scoped transform rules
    assertions = mkOption { ... };        # Build-time assertions
  };
}

Related options live in sibling modules:

  • modules/applications/argocd.nix: ArgoCD-specific options (syncPolicy, destination, compareOptions, finalizer)
  • modules/applications/objects.nix: the resource type registry (types), typed resources, and the internal final objects list

Key responsibilities:

  • Application metadata and settings
  • Resource type registration
  • Final object list generation

modules/applications/helm.nix#

Helm chart integration:

{
  options.helm.releases = mkOption {
    type = attrsOf (submodule {
      options = {
        chart = mkOption { ... };
        values = mkOption { ... };
        transformer = mkOption { ... };
      };
    });
  };
}

Processing flow:

  1. helm.buildHelmChart templates the chart
  2. builtins.readFile reads output
  3. kube.fromYAML parses to attribute sets
  4. transformer function applied
  5. partitionObjects (from lib.nix) groups objects by GVK
  6. Added to resources or objects

modules/applications/kustomize.nix#

Kustomize integration (similar pattern to Helm):

{
  options.kustomize.applications = mkOption {
    type = attrsOf (submodule {
      options = {
        kustomization.src = mkOption { ... };
        kustomization.path = mkOption { ... };
        transformer = mkOption { ... };
      };
    });
  };
}

modules/applications/yamls.nix#

Raw YAML manifest support:

{
  options.yamls = mkOption {
    type = listOf str;
    description = "List of YAML manifest strings";
  };

  # Also: extraRawYamls that are YAML files copied verbatim into
  # the output directory (not parsed into Nix, so they can't be patched)
}

modules/build/#

Creates output packages:

  • environmentPackage: All application manifests combined
  • extrasPackage: Extra files from nixidy.extraFiles
  • activationPackage: For nixidy switch operations
  • declarativePackage: For kubectl apply --prune. Emits an apply script that consumes environmentPackage and runs objectTransforms postProcess rules at apply time; it no longer renders standalone manifest files.
  • bootstrapPackage: App-of-apps manifest

Everything is derived from one seam: layout.nix turns each application into a list of FileSpec (one per output file), exposed as the internal config.build.layout, keyed by the config.applications attribute name. The emitters consume that list and never re-derive file paths:

File Responsibility
layout.nix Pure core: eval-time rewrite transforms, object grouping, file classification, per-app [FileSpec]
render.nix One FileSpec to the shell fragment that writes it
emit-environment.nix environmentPackage, activationPackage, bootstrapPackage, extrasPackage
apply.nix declarativePackage's apply script and the shared activation postProcess fragments
default.nix Option surface, layout wiring, environment-scope assertions

modules/templates.nix#

Template system for reusable patterns:

{
  options.templates = mkOption {
    type = attrsOf (submodule {
      options = {
        options = mkOption { ... };  # Template parameters
        output = mkOption { ... };   # Resource generator function
      };
    });
  };
}

Templates become application imports, allowing: 

applications.myapp.templates.webApp.frontend = {
  image = "nginx:latest";
  replicas = 3;
};

Helper Functions (modules/applications/lib.nix)#

{
  # Extract Group/Version/Kind from Kubernetes object
  getGVK = object: {
    group = ...;   # "core" for v1, otherwise first part of apiVersion
    version = ...; # Version string
    kind = ...;    # Kind string
  };

  # Flatten *List objects (e.g., ConfigMapList → [ConfigMap, ...])
  flattenListObjects = ...;

  # Split a flat object list into typed `resources` (registered GVKs) and
  # untyped `objects`. This is the single intake path shared by the helm,
  # kustomize, and yamls modules
  partitionObjects = ...;

  # Manifest filename stem for an object: `<Kind>-<dashed-name>`.
  # Used by both `build.nix` and `yamls.nix` so they always agree
  objectBaseName = ...;
}

Library Functions#

Entry Point (lib/default.nix)#

Extends nixpkgs.lib with nixidy functions:

lib.extend (self: old: {
  kustomize = import ./kustomize.nix { ... };
  helm = import ./helm.nix { ... };
  kube = import ./kube.nix { ... };
})

Helm Functions (lib/helm.nix)#

Function Description
downloadHelmChart Downloads chart from Helm registry
buildHelmChart Templates chart with values
getChartValues Parses chart's default values.yaml
mkChartAttrs Creates chart attrset from directory structure
mkChartsUpdateScript Builds a script that runs every chart's updateScript

Kube Functions (lib/kube.nix)#

Function Description
fromYAML Parses YAML string to attribute sets
fromOctal Converts octal string to integer
removeLabels Removes specified labels from manifests

Kustomize Functions (lib/kustomize.nix)#

Function Description
buildKustomization Builds kustomize application

Code Generators#

Overview (pkgs/generators/)#

Nixidy generates typed Nix options from:

  1. Kubernetes OpenAPI schemas
  2. Custom Resource Definitions (CRDs)

Kubernetes Schema Generation#

pkgs/generators/default.nix wires everything together and acquisition lives in pkgs/generators/sources/:

  1. sources/versions.nix lists the Kubernetes versions and their source hashes
  2. sources/k8s.nix fetches the Kubernetes source for each version and extracts the OpenAPI swagger spec
  3. Generates namespaced resource info
  4. Produces Nix options via compile/generator.nix

Output: modules/generated/k8s/v1.XX.nix

Shared schema walk and backends#

The schema-to-options logic lives in one place, compile/walk.nix, parameterized over a backend so the same traversal can produce either Nix source text or live module values:

  • compile/walk.nix - the single traversal of the (JSON) schema. All type/coercion branch logic (int-or-string, additionalProperties→attrsOf, coerce-by-name lists, patch-merge-key, skipCoerceToList, specialMapKeys, nested submodules, etc.) lives here, expressed against an abstract backend b.
  • compile/backend-text.nix - emits Nix source. Used by the file generators; it also inlines the runtime helpers (see below) as source so committed standalone files stay self-contained.
  • compile/backend-value.nix - produces live module values using runtime.nix.
  • compile/generator.nix - thin assembler over the text backend (renders + nixfmt + writes a .nix file).
  • compile/module.nix - thin assembler over the value backend (returns a module function).
  • compile/runtime.nix - the per-generated-module runtime helpers (coercedTo, mergeValuesByKey, submoduleForDefinition, …) as values, imported by the value backend. The text backend inlines source-form equivalents of these same helpers. Only the walk is single-sourced; the helpers must exist in both forms because live values can't be serialized back to source and committed files can't reference nixidy internals.

CRD Generation#

CRD accessors form a matrix over output shape × source, all built on the shared walk:

output source files (src) Helm chart
generated file fromCRD fromChartCRD
module value fromCRDModule fromChartCRDModule
raw objects crdObjects crdObjectsFromChart 
fromCRD {
  name = "cilium";
  src = pkgs.fetchFromGitHub { ... };
  crdFiles = [ "path/to/crd.yaml" ];  # list of YAML files under `src`
  kindFilter = [ ];          # Optional: only these CRD kinds (default: all)
  namePrefix = "";           # Optional prefix for attribute names
  attrNameOverrides = { };   # Manual name overrides
  skipCoerceToList = { };    # Skip list coercion for specific fields
}

fromChartCRD {
  name = "cert-manager";
  chartAttrs = { repo = "..."; chart = "..."; version = "..."; };
  kindFilter = [ "Certificate" ];  # Optional: only these CRD kinds (default: all)
  kubeVersion = "v1.31.0";         # Optional: helm template --kube-version
}

The *Module variants take the same arguments and return a module value (no file); crdObjects/crdObjectsFromChart return the raw CustomResourceDefinition manifests as values.

Renamed arguments

The overloaded crds argument was split: source-based accessors now take crdFiles (the list of YAML files) and chart-based accessors take kindFilter (the kind filter). crds remains as a deprecated alias on every accessor (resolved by the shared renamedArg helper, which warns and points at the new name).

CRD Processing (crd2jsonschema.py)#

Python script that:

  1. Reads CRD YAML files
  2. Extracts OpenAPI v3 schemas
  3. Flattens $ref references
  4. Outputs JSON schema for the shared walk (compile/walk.nix)

Testing Framework#

Module Tests (tests/)#

Located in tests/, using nixidy's testing framework.

Test Structure#

# tests/my-feature.nix
{
  lib,
  config,
  ...
}:
let
  apps = config.applications;
in
{
  # Define test configuration
  applications.test1 = {
    namespace = "test";
    resources.configMaps.cm.data.FOO = "bar";
  };

  # Define test assertions
  test = {
    name = "my feature test";
    description = "Description of what's being tested";
    assertions = [
      {
        description = "ConfigMap should have FOO key";
        expression = (elemAt apps.test1.objects 0).data;
        expected = { FOO = "bar"; };
      }
      {
        description = "Custom assertion function";
        expression = apps.test1.objects;
        assertion = objs: length objs == 1;
      }
    ];
  };
}

Running Tests#

# Run module tests
nix run .#moduleTests

# Run library tests
nix run .#libTests

Test Registration (tests/default.nix)#

{
  testing = {
    name = "nixidy modules";
    tests = [
      ./configmap.nix
      ./create-namespace.nix
      ./helm/with-values.nix
      # ... more tests
    ];
  };
}

Library Tests (lib/tests.nix)#

Uses lib.runTests pattern:

{
  kube = {
    fromYAML = {
      testSingleObject = {
        expr = lib.kube.fromYAML "...";
        expected = [ { ... } ];
      };
    };
    removeLabels = {
      testLabelPresent = {
        expr = lib.kube.removeLabels ["helm.sh/chart"] { ... };
        expected = { ... };
      };
    };
  };
}

Development Workflow#

Prerequisites#

  • Nix with flakes enabled
  • Basic understanding of NixOS module system

Common Commands#

# Format code
nix fmt

# Run static linter
nix run .#staticCheck

# Run library tests
nix run .#libTests

# Run module tests
nix run .#moduleTests

# Run crd2jsonschema unit tests
nix run .#crd2jsonschemaTest

# Assert CRD file/module/objects accessors agree with each other
nix run .#crdAccessorTest

# Generate Kubernetes modules
nix run .#generate

# Serve documentation locally
nix run .#docsServe

Development Cycle#

  1. Make changes to modules or library
  2. Write tests for new functionality
  3. Run tests to verify changes
  4. Format code with nix fmt
  5. Run linter with nix run .#staticCheck
  6. Test manually with a sample configuration

Manual Testing#

Create a test configuration:

# test-config.nix
{
  nixidy.target = {
    repository = "https://github.com/test/repo.git";
    branch = "main";
  };

  applications.test = {
    namespace = "test";
    createNamespace = true;
    resources.deployments.nginx.spec = {
      selector.matchLabels.app = "nginx";
      template = {
        metadata.labels.app = "nginx";
        spec.containers.nginx.image = "nginx:latest";
      };
    };
  };
}

Build and inspect:

nix run .#cli -- build .#test
tree result/
cat result/test/Deployment-nginx.yaml

Adding New Features#

Adding a New Application Option#

  1. Define the option in modules/applications/default.nix: 

    {
  options = {
    myNewOption = mkOption {
      type = types.bool;
      default = false;
      description = "Description of the option.";
    };
  };
}

  2. Use the option in config: 

    {
  config = lib.mkIf config.myNewOption {
    # Configuration when option is enabled
  };
}

  3. Write tests in tests/: 

    # tests/my-new-option.nix
{
  applications.test1 = {
    myNewOption = true;
    # ...
  };

  test = {
    name = "my new option";
    description = "Test the new option";
    assertions = [ ... ];
  };
}

  4. Register test in tests/default.nix

  5. Document in docs/user_guide/

Adding a New Library Function#

  1. Add function to appropriate file in lib/: 

    # lib/kube.nix
{
  myNewFunction =
    # Parameter description
    param:
    # Implementation
    ...;
}

  2. Add documentation as comments: 

    /*
  Description of function.

  Type:
    myNewFunction :: ParamType -> ReturnType

  Example:
    myNewFunction "input"
    => "output"
*/
myNewFunction = ...;

  3. Write tests in lib/tests.nix: 

    {
  kube = {
    myNewFunction = {
      testBasicCase = {
        expr = lib.kube.myNewFunction "input";
        expected = "output";
      };
    };
  };
}

Adding a New Resource Processor#

Similar to Helm/Kustomize, create a new module:

  1. Create module modules/applications/myprocessor.nix: 

    {
  nixidyDefaults,
  lib,
  config,
  ...
}:
let
  helpers = import ./lib.nix lib;
in
{
  options.myProcessor = mkOption {
    type = with types; attrsOf (submodule { ... });
  };

  config = {
    # Process inputs and add to resources/objects
    resources = mkMerge [ ... ];
    objects = [ ... ];
  };
}

  2. Import in modules/applications/default.nix: 

    {
  imports = [
    ./helm.nix
    ./kustomize.nix
    ./yamls.nix
    ./argocd.nix
    ./objects.nix
    ./myprocessor.nix  # Add here
  ];
}

Common Tasks#

Updating Kubernetes Versions#

  1. Edit pkgs/generators/sources/versions.nix: 

    {
  "1.37.0" = {
    hash = "sha256-...";
    spec = "api/openapi-spec/swagger.json";
    discovery = {
      core = "api/discovery/api__v1.json";
      aggregated = "api/discovery/aggregated_v2.json";
    };
  };
}

  2. Regenerate: 

    nix run .#generate

  3. Update default version in modules/applications.nix if needed (the nixidy.k8sVersion enum is derived automatically from the generated files in modules/generated/k8s/)

Adding a New Sync Option#

  1. Add option in modules/applications/argocd.nix under syncPolicy.syncOptions: 

    {
  syncPolicy.syncOptions.myOption = mkOption {
    type = types.bool;
    default = false;
    apply = val: if val then "MyOption=true" else null;
    description = "Description";
  };
}

  2. The apply function converts to ArgoCD sync option format

  3. convertSyncOptionsList automatically collects non-null options

Debugging Module Evaluation#

Use builtins.trace for debugging:

{
  config = lib.mkMerge [
    (builtins.trace "Processing ${config.name}" {
      # ...
    })
  ];
}

Or use lib.debug.traceValSeqN:

{
  objects = lib.debug.traceValSeqN 2 config.resources;
}

Code Style#

Nix#

  • Format: Use nix fmt (nixfmt)
  • Sorting: Keep attribute sets alphabetically sorted
  • Inherit: Use inherit where possible to reduce verbosity
  • Imports: Group imports logically
  • Types: Use specific types over types.anything when possible
# Good
{
  lib,
  config,
  pkgs,
  ...
}:
let
  inherit (config) namespace;
  inherit (lib) mkOption types;
in
{
  options.myOption = mkOption {
    type = types.str;
    default = "";
  };
}

# Avoid
{lib, config, pkgs, ...}: let
  namespace = config.namespace;
in {
  options.myOption = lib.mkOption {
    type = lib.types.str;
    default = "";
  };
}

Python (CLI)#

  • Follow PEP 8 guidelines
  • Use type hints for all function signatures
  • Document public functions with docstrings

Documentation#

  • Use MkDocs syntax
  • Include code examples with syntax highlighting
  • Cross-reference related documentation
  • Keep language clear and concise

Getting Help#

Summary#

Key files for different tasks:

Task Files
Add application option modules/applications/default.nix
Add nixidy option modules/nixidy/
Add library function lib/*.nix
Add resource processor modules/applications/
Add template feature modules/templates.nix
Modify build output modules/build/
Add K8s version pkgs/generators/sources/versions.nix
Write module test tests/*.nix, tests/default.nix
Write library test lib/tests.nix

Welcome to the nixidy project! We look forward to your contributions.