Implement the following plan:
Plan: Module Support (#INCLUDE, Preprocessor, SConscript)
Context
Occam programs use #INCLUDE "course.module" to import library modules. The KRoC project (cloned at ./kroc/) contains the course module source at ./kroc/modules/course/libsrc/. The module's composition is defined by SConscript (Python) build files. To support this, we need: (1) a textual preprocessor for #IF/#DEFINE/#ENDIF/#INCLUDE, (2) a tool to generate .module files from SConscript, and (3) wiring into the CLI.
The preprocessor is a pre-lexer text expansion pass (like C's cpp), producing a single expanded string that feeds into the existing lexer.New(string).
Phase 1: Preprocessor — Conditional Directives
New file: preproc/preproc.go
type Preprocessor struct {
defines map[string]string // symbol → value (empty string for flag-only defines)
includePaths []string
errors []string
}
- Line-by-line processing with a condition stack for
#IF/#ELSE/#ENDIF nesting
- When a line's first non-whitespace is
#, parse as directive
- Excluded lines → emit blank lines (preserves line numbers for error messages)
Directives to handle:
| Directive | Action |
|---|---|
| #DEFINE SYMBOL | Add to defines map |
| #IF expr | Push condition stack, evaluate expr |
| #ELSE | Flip active state |
| #ENDIF | Pop condition stack |
| #COMMENT "..." | No-op (emit blank line) |
| #PRAGMA ... | No-op (emit blank line) |
| #USE "..." | No-op (emit blank line) — KRoC compiled library linking, not needed for source transpilation |
| #INCLUDE "..." | Phase 2 |
Expression evaluator (minimal recursive descent) supporting patterns found in real code:
- TRUE / FALSE
- DEFINED (SYMBOL.NAME)
- NOT (expr)
- (SYMBOL = value) — for (TARGET.BITS.PER.WORD = 32)
Predefined symbols: TARGET.BITS.PER.WORD = "64" (Go is always 64-bit int).
New file: preproc/preproc_test.go — unit tests for conditional directives, nesting, #ELSE, line count preservation, edge cases.
Phase 2: Preprocessor — #INCLUDE File Inclusion
Additions to preproc/preproc.go:
ProcessFile(filename string) (string, error) — reads file, processes it#INCLUDE "file" resolution: first relative to current file's directory, then each includePaths entry- Recursive processing: included file content is processed through the same
Preprocessor instance (shared defines map, so include guards work)
- Circular include detection via a "currently processing" file set (by absolute path)
Include guard pattern (standard in occam modules):
#IF NOT (DEFINED (COURSE.MODULE))
#DEFINE COURSE.MODULE
...
#ENDIF
Additional tests: file inclusion, search path resolution, include guard double-inclusion prevention, nested includes, circular include error, file-not-found error. Tests use os.MkdirTemp for temp file trees.
Phase 3: Module File Generator from SConscript
New file: modgen/modgen.go
Minimal SConscript parser using regex, extracting:
- varname = Split('''...''') → list of source files
- OccamLibrary('name.lib', source_var, OCCBUILDFLAGS='--include file.inc') → library name, sources, includes
- --need libname flags → dependency modules
Output: .module file text with include guards + #INCLUDE directives.
Example generated course.module:
#IF NOT (DEFINED (COURSE.MODULE))
#DEFINE COURSE.MODULE
#INCLUDE "consts.inc"
#INCLUDE "utils.occ"
#INCLUDE "string.occ"
#INCLUDE "demo_cycles.occ"
#INCLUDE "demo_nets.occ"
#INCLUDE "file_in.occ"
#INCLUDE "float_io.occ"
#INCLUDE "random.occ"
#ENDIF
New file: modgen/modgen_test.go — tests against real course SConscript content.
Phase 4: Wire into CLI + E2E Tests
Modify: main.go
- Add
-I path flag (repeatable) for include search paths
- Add
-D SYMBOL flag (repeatable) for predefined symbols
- Add
gen-module subcommand: occam2go gen-module [-o output] <SConscript>
- Insert preprocessor before lexer in the main pipeline:
pp := preproc.New(WithIncludePaths(...), WithDefines(...))
expanded, err := pp.ProcessFile(inputFile)
l := lexer.New(expanded)
Modify: codegen/e2e_test.go — add file-based e2e test helper and tests for:
1. Simple #INCLUDE of a constants file → use constant in program
2. #IF FALSE excluding code
3. Include guard preventing double inclusion
Files Summary
| Action |
File |
| Create |
preproc/preproc.go |
| Create |
preproc/preproc_test.go |
| Create |
modgen/modgen.go |
| Create |
modgen/modgen_test.go |
| Modify |
main.go |
| Modify |
codegen/e2e_test.go |
Verification
After each phase, run go test ./... to confirm no regressions. After Phase 4:
1. go test ./preproc — preprocessor unit tests pass
2. go test ./modgen — module generator tests pass
3. go test ./codegen — existing + new e2e tests pass
4. Manual test: go run . gen-module kroc/modules/course/libsrc/SConscript generates correct course.module
5. Manual test: create a hello_world.occ with #INCLUDE "course.module" and transpile with -I kroc/modules/course/libsrc
If you need specific details from before exiting plan mode (like exact code snippets, error messages, or content you generated), read the full transcript at: /home/david/.claude/projects/-home-david-projects-codeassociates-occam2go/ffdf224e-89eb-40a8-9de1-c4688775e3cb.jsonl