Project package
Each DipDup project consists of a YAML config and a Python package of a specific structure. It could be placed anywhere, but needs to be importable. The package name is defined in the config file.
To generate all necessary directories and files according to config run the init command. You should run it every time you significantly change the config file.
The structure of the resulting package is the following:
| Path | Description |
|---|---|
📁 abi | Contract ABIs used to generate typeclasses |
📁 configs | Environment-specific configs to merge with the root one |
📁 deploy | Dockerfiles, compose files, and default env variables for each environment |
📁 graphql | Custom GraphQL queries to expose with Hasura engine |
📁 handlers | User-defined callbacks to process contract data |
📁 hasura | Arbitrary Hasura metadata to apply during configuration |
📁 hooks | User-defined callbacks to run manually or by schedule |
📁 models | DipDup ORM models to store data in the database |
📁 sql | SQL scripts and queries to run manually or on specific events |
📁 types | Automatically generated Pydantic dataclasses for contract types |
dipdup.yaml | Root DipDup config; can be expanded with env-specific files |
dipdup_indexer -> . | A little helper, symlink to let you import the package from the root directory. |
pyproject.toml | Python package metadata (introduced in PEP 518; see details) |
Makefile | Makefile with shortcuts for common commands (run make for help) |
Also, there are .dockerignore, .gitignore files and py.typed marker (PEP 561). Usually, you won't need to modify them.
ABIs and typeclasses
DipDup uses contract type information to generate Pydantic models to work with strictly typed data. We call these models typeclasses. Modules in the types directory are generated automatically from contract ABIs and JSONSchemas during init. You can modify them manually, but usually won't need to. Under the hood, the process is roughly the following:
- Contract ABIs are fetched from public sources or provided by the user.
- DipDup converts these ABIs to intermediate JSONSchemas.
- JSONSchemas converted to Pydantic models with datamodel-code-generator.
This approach allows working with complex contract types with nested structures and polymorphic variants.
Nested packages
Callbacks can be grouped into packages to organize the project structure. Add one or multiple dots to the callback name to define nested packages:
package: dipdup_indexer
hooks:
backup.restore:
callback: backup.on_restore
After running the init command, you'll get the following directory tree (shortened for brevity)
dipdup_indexer
├── hooks
│ ├── backup
│ │ └── on_restore.py
└── sql
└── backup
└── on_restore
Handler callbacks can be grouped the same way. Note, that the callback name still needs to be a valid Python module path: only lowercase letters, underscores, and dots.