title: Support INT16, INT32, INT64 integer types state: OPEN author: associate-1 labels: comments: 0 assignees: projects: milestone: number: 46 -- ## Description The parser does not recognize the extended integer width types `INT16`, `INT32`, and `INT64`. These are standard occam types (not occam-pi) for explicit-width integers. ### Syntax ```occam INT16 i16: INT32 i32: INT64 i64: i16 := INT16 b i64 := INT64 microsecs ``` They appear in variable declarations, type conversions, and VAL abbreviations. ### Affected files - `kroc/modules/course/examples/cast.occ` (lines 38-40 and throughout) - `kroc/modules/course/examples/casting.occ` (lines 38-40 and throughout) - `kroc/modules/course/examples/commstime.occ` (line 36: `VAL INT64 nanosecs IS ...`) ### Current error ``` cast.occ:38: unexpected token: : ``` The parser doesn't recognize `INT16`/`INT32`/`INT64` as type keywords, so it can't parse the variable declaration. ### Proposed Go mapping | Occam | Go | |-------|-----| | `INT16 x:` | `var x int16` | | `INT32 x:` | `var x int32` | | `INT64 x:` | `var x int64` | | `INT16 expr` (conversion) | `int16(expr)` | | `INT32 expr` (conversion) | `int32(expr)` | | `INT64 expr` (conversion) | `int64(expr)` | | `MOSTNEG INT16` | `math.MinInt16` | | `MOSTPOS INT16` | `math.MaxInt16` | | etc. | etc. | ### Implementation Following the existing pattern for type support: 1. **Lexer** (`token.go`): Add `INT16`, `INT32`, `INT64` token types and keyword entries 2. **Parser** (`parser.go`): Add these to type-parsing logic (declarations, parameters, conversions) 3. **AST** (`ast.go`): No changes needed — existing type string fields can hold `"INT16"` etc. 4. **Codegen** (`codegen.go`): Map to Go types in `goType()`, handle in type conversions, add MOSTNEG/MOSTPOS constants ### Related - #43 - Course examples tracking issue
This is a well-specified issue. Let me plan and implement support for INT16, INT32, INT64 types. I'll follow the existing patterns for INT/BYTE/REAL32/REAL64.
Let me explore the codebase to understand the current implementation patterns for types.
Entered plan mode. You should now focus on exploring the codebase and designing an implementation approach. In plan mode, you should: 1. Thoroughly explore the codebase to understand existing patterns 2. Identify similar features and architectural approaches 3. Consider multiple approaches and their trade-offs 4. Use AskUserQuestion if you need to clarify the approach 5. Design a concrete implementation strategy 6. When ready, use ExitPlanMode to present your plan for approval Remember: DO NOT write or edit any files yet. This is a read-only exploration and planning phase.
Let me explore the codebase to understand how existing types (INT, BYTE, REAL32, REAL64) are handled across all the relevant files.
{
"subagent_type": "Explore",
"prompt": "I need to understand how the occam2go transpiler handles integer and real types across the codebase. The goal is to add INT16, INT32, INT64 types following the same patterns.\n\nPlease explore and provide detailed findings for:\n\n1. **Lexer** (`lexer/token.go`, `lexer/lexer.go`): How are INT_TYPE, BYTE_TYPE, REAL32_TYPE, REAL64_TYPE token types defined? What keyword entries exist? Show the relevant code sections.\n\n2. **Parser** (`parser/parser.go`): How are these types used in:\n - Variable declarations (e.g., `INT x:`)\n - Type conversions in expressions (e.g., `INT expr`)\n - Proc/function parameters\n - The `parseType()` or equivalent function\n - MOSTNEG/MOSTPOS handling\n - Any other type-related parsing\n\n3. **Codegen** (`codegen/codegen.go`): How does `goType()` map occam types to Go types? How are type conversions generated? How are MOSTNEG/MOSTPOS mapped?\n\n4. **AST** (`ast/ast.go`): How are types represented in AST nodes (VarDecl, TypeConversion, etc.)?\n\nBe thorough - I need to see all the places where INT_TYPE, BYTE_TYPE, REAL32_TYPE, REAL64_TYPE appear so I can add INT16/INT32/INT64 in parallel."
}[
{
"type": "text",
"text": "Perfect! Now I have comprehensive information. Let me compile my findings:\n\n## Comprehensive Findings: Type Handling in occam2go\n\nI've thoroughly explored the codebase to understand how integer and real types are currently handled. Here are the detailed findings:\n\n---\n\n### 1. LEXER (`lexer/token.go`, `lexer/lexer.go`)\n\n**Token Type Definitions** (lines 77-82 in token.go):\n```go\nINT_TYPE\nBYTE_TYPE\nBOOL_TYPE\nREAL_TYPE\nREAL32_TYPE\nREAL64_TYPE\n```\n\n**Keyword Mapping** (lines 215-220 in token.go):\n```go\n\"INT\": INT_TYPE,\n\"BYTE\": BYTE_TYPE,\n\"BOOL\": BOOL_TYPE,\n\"REAL\": REAL_TYPE,\n\"REAL32\": REAL32_TYPE,\n\"REAL64\": REAL64_TYPE,\n```\n\n**Token Name Mapping** (lines 167-172 in token.go):\n```go\nINT_TYPE: \"INT\",\nBYTE_TYPE: \"BYTE\",\nBOOL_TYPE: \"BOOL\",\nREAL_TYPE: \"REAL\",\nREAL32_TYPE: \"REAL32\",\nREAL64_TYPE: \"REAL64\",\n```\n\n**Lexer behavior**: The `LookupIdent()` function (lines 245-250) maps keyword strings to token types. The lexer itself doesn't need modification for type handling—it's purely keyword recognition.\n\n---\n\n### 2. PARSER (`parser/parser.go`)\n\n**Type Token Checking** (lines 1978-1983):\n```go\nfunc isTypeToken(t lexer.TokenType) bool {\n\treturn t == lexer.INT_TYPE || t == lexer.BYTE_TYPE ||\n\t\tt == lexer.BOOL_TYPE || t == lexer.REAL_TYPE ||\n\t\tt == lexer.REAL32_TYPE || t == lexer.REAL64_TYPE\n}\n```\nThis helper appears at **17 locations** in the parser (checked at lines 168, 433, 435, 563, 565, 600, 602, 811, 813, 1065, 1067, 1653, 2276, 2278, 2795, 2797, 2807).\n\n**Variable Declaration Parsing** (lines 234-259, 263-315):\n- `parseVarDecl()` extracts the type as a string from `p.curToken.Literal`\n- `parseVarDeclOrAbbreviation()` handles both `INT x:` declarations and `INT x IS expr:` abbreviations\n- Type names are stored as strings in AST nodes (e.g., `decl.Type = p.curToken.Literal`)\n\n**Function Declaration Parsing** (lines 2200-2240):\n- Multi-result functions parse return types: `INT, INT FUNCTION foo(...)`\n- Return types collected in `fn.ReturnTypes` slice as strings (line 2203: `ReturnTypes: []string{p.curToken.Literal}`)\n\n**Type Conversions in Expressions** (lines 2807-2814):\n```go\ncase lexer.INT_TYPE, lexer.BYTE_TYPE, lexer.BOOL_TYPE, lexer.REAL_TYPE, lexer.REAL32_TYPE, lexer.REAL64_TYPE:\n\ttoken := p.curToken\n\tp.nextToken()\n\tleft = &ast.TypeConversion{\n\t\tToken: token,\n\t\tTargetType: token.Literal,\n\t\tExpr: p.parseExpression(PREFIX),\n\t}\n```\n\n**MOSTNEG/MOSTPOS Parsing** (lines 2791-2806):\n```go\ncase lexer.MOSTNEG_KW, lexer.MOSTPOS_KW:\n\ttoken := p.curToken\n\tisNeg := token.Type == lexer.MOSTNEG_KW\n\tif !p.peekTokenIs(lexer.INT_TYPE) && !p.peekTokenIs(lexer.BYTE_TYPE) &&\n\t\t!p.peekTokenIs(lexer.BOOL_TYPE) && !p.peekTokenIs(lexer.REAL_TYPE) &&\n\t\t!p.peekTokenIs(lexer.REAL32_TYPE) && !p.peekTokenIs(lexer.REAL64_TYPE) {\n\t\tp.addError(fmt.Sprintf(\"expected type after %s, got %s\", token.Literal, p.peekToken.Type))\n\t\treturn nil\n\t}\n\tp.nextToken()\n\tleft = &ast.MostExpr{\n\t\tToken: token,\n\t\tExprType: p.curToken.Literal,\n\t\tIsNeg: isNeg,\n\t}\n```\n\n**Procedure Parameter Parsing** (lines 1985-2215):\n- Parameters are parsed with type information in `ProcParam.Type` (string field, line 146 in ast.go)\n- Type validation uses `isTypeToken()` at multiple points\n\n---\n\n### 3. AST (`ast/ast.go`)\n\n**Type Representation**:\n\n1. **VarDecl** (lines 36-44):\n - `Type` field: stores type name as string (`\"INT\"`, `\"BYTE\"`, `\"REAL32\"`, etc.)\n - `Names` field: slice of variable names\n\n2. **TypeConversion** (lines 299-308):\n ```go\n type TypeConversion struct {\n Token lexer.Token // the type token (INT, BYTE, etc.)\n TargetType string // \"INT\", \"BYTE\", \"BOOL\", \"REAL\"\n Expr Expression // the expression to convert\n }\n ```\n\n3. **MostExpr** (lines 318-326):\n ```go\n type MostExpr struct {\n Token lexer.Token // the MOSTNEG or MOSTPOS token\n ExprType string // \"INT\", \"BYTE\", \"REAL32\", \"REAL64\", etc.\n IsNeg bool // true for MOSTNEG, false for MOSTPOS\n }\n ```\n\n4. **ProcParam** (lines 143-154):\n - `Type` field: stores parameter type as string\n - `ChanElemType` field: for channel element types\n\n5. **FuncDecl** (lines 167-177):\n - `ReturnTypes` field: `[]string` slice of return type names\n\n6. **RecordField** (lines 468-471):\n - `Type` field: field type as string\n\n7. **Abbreviation** (lines 487-496):\n - `Type` field: type name as string\n\n---\n\n### 4. CODEGEN (`codegen/codegen.go`)\n\n**Type Mapping Functions**:\n\n**occamTypeToGoBase** (lines 1400-1419):\n```go\nfunc (g *Generator) occamTypeToGoBase(occamType string) string {\n\tswitch occamType {\n\tcase \"INT\":\n\t\treturn \"int\"\n\tcase \"BYTE\":\n\t\treturn \"byte\"\n\tcase \"BOOL\":\n\t\treturn \"bool\"\n\tcase \"REAL\":\n\t\treturn \"float64\"\n\tcase \"REAL32\":\n\t\treturn \"float32\"\n\tcase \"REAL64\":\n\t\treturn \"float64\"\n\tdefault:\n\t\treturn occamType\n\t}\n}\n```\n\n**occamTypeToGo** (lines 1421-1446):\n```go\nfunc (g *Generator) occamTypeToGo(occamType string) string {\n\tswitch occamType {\n\tcase \"INT\":\n\t\treturn \"int\"\n\tcase \"BYTE\":\n\t\treturn \"byte\"\n\tcase \"BOOL\":\n\t\treturn \"bool\"\n\tcase \"REAL\":\n\t\treturn \"float64\"\n\tcase \"REAL32\":\n\t\treturn \"float32\"\n\tcase \"REAL64\":\n\t\treturn \"float64\"\n\tdefault:\n\t\t// Check if it's a protocol name\n\t\tif _, ok := g.protocolDefs[occamType]; ok {\n\t\t\treturn \"_proto_\" + occamType\n\t\t}\n\t\t// Check if it's a record type name\n\t\tif _, ok := g.recordDefs[occamType]; ok {\n\t\t\treturn occamType\n\t\t}\n\t\treturn occamType // pass through unknown types\n\t}\n}\n```\n\n**Type Conversion Code Generation** (lines 2378-2382):\n```go\ncase *ast.TypeConversion:\n\tg.write(g.occamTypeToGo(e.TargetType))\n\tg.write(\"(\")\n\tg.generateExpression(e.Expr)\n\tg.write(\")\")\n```\n\n**MOSTNEG/MOSTPOS Code Generation** (lines 926-953):\n```go\nfunc (g *Generator) generateMostExpr(e *ast.MostExpr) {\n\tswitch e.ExprType {\n\tcase \"INT\":\n\t\tif e.IsNeg {\n\t\t\tg.write(\"math.MinInt\")\n\t\t} else {\n\t\t\tg.write(\"math.MaxInt\")\n\t\t}\n\tcase \"BYTE\":\n\t\tif e.IsNeg {\n\t\t\tg.write(\"0\")\n\t\t} else {\n\t\t\tg.write(\"255\")\n\t\t}\n\tcase \"REAL32\":\n\t\tif e.IsNeg {\n\t\t\tg.write(\"-math.MaxFloat32\")\n\t\t} else {\n\t\t\tg.write(\"math.MaxFloat32\")\n\t\t}\n\tcase \"REAL64\":\n\t\tif e.IsNeg {\n\t\t\tg.write(\"-math.MaxFloat64\")\n\t\t} else {\n\t\t\tg.write(\"math.MaxFloat64\")\n\t\t}\n\t}\n}\n```\n\n**Math Import Detection** (lines 773-924):\n- `containsMostExpr()` function recursively checks statements for MOSTNEG/MOSTPOS expressions\n- `exprNeedsMath()` function checks expressions:\n - At line 895: `return e.ExprType != \"BYTE\"` (BYTE doesn't need math import, uses literal 0/255)\n - At line 102-123: If math-requiring expressions found, \"math\" import is added\n\n**Type Usage Locations in Codegen**:\n- Variable declarations: `generateVarDecl()` (line 1026) uses `occamTypeToGo(decl.Type)`\n- Channel declarations: `generateChanDecl()` (line 1050) uses `occamTypeToGo(decl.ElemType)`\n- Array declarations: `generateArrayDecl()` (line 1079) uses `occamTypeToGo(decl.Type)`\n- Protocol generation: `occamTypeToGoBase()` (lines 1182, 1189, 1213)\n- Record field generation: `occamTypeToGoBase()` (line 1392)\n- Procedure parameters: `generateProcParams()` (lines 1889-1901) uses `occamTypeToGo()`\n- Function return types: `generateFuncDecl()` (lines 1975-1982) uses `occamTypeToGo()`\n- Abbreviations: `generateAbbreviation()` (line 262) uses `occamTypeToGo(abbr.Type)`\n\n---\n\n### 5. TESTS\n\n**Type Usage Examples** (from `e2e_types_test.go`):\n- Variable declarations: `INT x:`, `BYTE b:`, `REAL32 a:`, `REAL64 b:`\n- Type conversions: `INT b`, `BYTE n`, `REAL32 5`, `REAL64 10`\n- MOSTNEG/MOSTPOS: `MOSTNEG INT`, `MOSTPOS BYTE`, `MOSTNEG REAL32`, `MOSTPOS REAL64`\n\n---\n\n### SUMMARY TABLE: Where Types Appear\n\n| Component | Location | Pattern | Notes |\n|-----------|----------|---------|-------|\n| **Lexer** | token.go:77-82, 215-220 | Token enum + keyword map | Add new token types here |\n| **Lexer** | token.go:167-172 | tokenNames map | Add string names here |\n| **Parser** | parser.go:1978-1983 | `isTypeToken()` | Add new type checks in 17 places |\n| **Parser** | parser.go:2807-2814 | Type conversion parsing | Automatically handles new types |\n| **Parser** | parser.go:2791-2806 | MOSTNEG/MOSTPOS parsing | Automatically handles new types |\n| **AST** | ast.go:37-44 | VarDecl.Type (string) | No changes needed |\n| **AST** | ast.go:299-308 | TypeConversion.TargetType (string) | No changes needed |\n| **AST** | ast.go:318-326 | MostExpr.ExprType (string) | No changes needed |\n| **Codegen** | codegen.go:1402-1419 | `occamTypeToGoBase()` | Add new type mappings here |\n| **Codegen** | codegen.go:1421-1446 | `occamTypeToGo()` | Add new type mappings here |\n| **Codegen** | codegen.go:926-953 | `generateMostExpr()` | Add MOSTNEG/MOSTPOS cases here |\n| **Codegen** | codegen.go:893-895 | `exprNeedsMath()` | Update math import detection if needed |\n\n---\n\n### KEY INSIGHTS FOR INT16/INT32/INT64 ADDITION\n\n1. **No changes to AST** — types are stored as strings, so new types automatically work\n2. **Parser changes** — `isTypeToken()` function needs updating at **~17 locations**\n3. **Lexer changes** — 3 places: token enum, keywords map, tokenNames map\n4. **Codegen changes** — 2 mapping functions + MOSTNEG/MOSTPOS switch cases\n5. **Math imports** — If INT16/INT32/INT64 use `math.MinInt`, update `exprNeedsMath()` check\n6. **Go type mapping**:\n - INT16 → `int16`\n - INT32 → `int32`\n - INT64 → `int64`\n7. **MOSTNEG/MOSTPOS** values:\n - INT16: `math.MinInt16` / `math.MaxInt16`\n - INT32: `math.MinInt32` / `math.MaxInt32`\n - INT64: `math.MinInt64` / `math.MaxInt64`"
},
{
"type": "text",
"text": "agentId: ae18ae30441539b2b (for resuming to continue this agent's work if needed)\n<usage>total_tokens: 64077\ntool_uses: 24\nduration_ms: 62806</usage>"
}
]Excellent, very thorough exploration. Let me now check the isTypeToken() function usage pattern more carefully - the agent said it appears at 17 locations, but most of those are likely just in isTypeToken() itself and the places that call it. Since isTypeToken() is a function, updating it in one place should propagate to all callers.
Let me also look at the MOSTNEG/MOSTPOS parsing to understand whether INT16/32/64 needs special handling for math imports.
Let me now write the plan.
Great, the exploration is very thorough. The key insight is that isTypeToken() is a single function that's called in 17 places — updating the function itself propagates everywhere. Let me write the plan.
# Plan: Support INT16, INT32, INT64 integer types (Issue #46) ## Context The parser doesn't recognize `INT16`, `INT32`, `INT64` as type keywords, causing parse failures on real-world occam programs (e.g., `cast.occ`, `commstime.occ`). These are standard occam explicit-width integer types that need the same treatment as `INT`, `BYTE`, `REAL32`, `REAL64`. ## Changes ### 1. Lexer — `lexer/token.go` Add 3 new token types and keyword entries: - **Token enum** (~line 82): Add `INT16_TYPE`, `INT32_TYPE`, `INT64_TYPE` after `REAL64_TYPE` - **tokenNames map** (~line 172): Add `"INT16"`, `"INT32"`, `"INT64"` entries - **keywords map** (~line 220): Add `"INT16": INT16_TYPE`, `"INT32": INT32_TYPE`, `"INT64": INT64_TYPE` ### 2. Parser — `parser/parser.go` - **`isTypeToken()`** (~line 1978): Add `|| t == lexer.INT16_TYPE || t == lexer.INT32_TYPE || t == lexer.INT64_TYPE` — this single change propagates to all ~17 call sites - **MOSTNEG/MOSTPOS parsing** (~line 2795): Add `!p.peekTokenIs(lexer.INT16_TYPE) && !p.peekTokenIs(lexer.INT32_TYPE) && !p.peekTokenIs(lexer.INT64_TYPE)` to the guard check - **Type conversion prefix** (~line 2807): Add `lexer.INT16_TYPE, lexer.INT32_TYPE, lexer.INT64_TYPE` to the case list ### 3. AST — `ast/ast.go` No changes needed — types stored as strings. ### 4. Codegen — `codegen/codegen.go` - **`occamTypeToGoBase()`** (~line 1400): Add cases `"INT16" → "int16"`, `"INT32" → "int32"`, `"INT64" → "int64"` - **`occamTypeToGo()`** (~line 1421): Add same cases - **`generateMostExpr()`** (~line 926): Add cases for INT16/INT32/INT64 mapping to `math.MinInt16`/`math.MaxInt16`, etc. - **`exprNeedsMath()`** (~line 893): Current check is `return e.ExprType != "BYTE"` — INT16/32/64 all need math import, so no change needed (they won't match "BYTE") ### 5. Tests - **Parser unit tests** (`parser/parser_test.go`): Add test for INT16/INT32/INT64 variable declarations and type conversions - **Codegen unit tests** (`codegen/codegen_test.go`): Add test for INT16/INT32/INT64 code generation - **E2E tests** (`codegen/e2e_test.go` or `codegen/e2e_types_test.go`): Add test that declares INT16/32/64 variables, uses type conversions and MOSTNEG/MOSTPOS, compiles and runs ### 6. Documentation - Update `TODO.md` to mark INT16/INT32/INT64 as supported ## Verification ```bash go test ./... # all tests pass ./occam2go -I kroc/modules/course/libsrc -D TARGET.BITS.PER.WORD=32 -o /tmp/course_out.go kroc/modules/course/libsrc/course.module go vet /tmp/course_out.go # course module still compiles ```
{
"allowedPrompts": [
{
"tool": "Bash",
"prompt": "run tests"
},
{
"tool": "Bash",
"prompt": "build project"
},
{
"tool": "Bash",
"prompt": "run course module transpilation and vet"
}
],
"plan": "# Plan: Support INT16, INT32, INT64 integer types (Issue #46)\n\n## Context\n\nThe parser doesn't recognize `INT16`, `INT32`, `INT64` as type keywords, causing parse failures on real-world occam programs (e.g., `cast.occ`, `commstime.occ`). These are standard occam explicit-width integer types that need the same treatment as `INT`, `BYTE`, `REAL32`, `REAL64`.\n\n## Changes\n\n### 1. Lexer — `lexer/token.go`\n\nAdd 3 new token types and keyword entries:\n\n- **Token enum** (~line 82): Add `INT16_TYPE`, `INT32_TYPE`, `INT64_TYPE` after `REAL64_TYPE`\n- **tokenNames map** (~line 172): Add `\"INT16\"`, `\"INT32\"`, `\"INT64\"` entries\n- **keywords map** (~line 220): Add `\"INT16\": INT16_TYPE`, `\"INT32\": INT32_TYPE`, `\"INT64\": INT64_TYPE`\n\n### 2. Parser — `parser/parser.go`\n\n- **`isTypeToken()`** (~line 1978): Add `|| t == lexer.INT16_TYPE || t == lexer.INT32_TYPE || t == lexer.INT64_TYPE` — this single change propagates to all ~17 call sites\n- **MOSTNEG/MOSTPOS parsing** (~line 2795): Add `!p.peekTokenIs(lexer.INT16_TYPE) && !p.peekTokenIs(lexer.INT32_TYPE) && !p.peekTokenIs(lexer.INT64_TYPE)` to the guard check\n- **Type conversion prefix** (~line 2807): Add `lexer.INT16_TYPE, lexer.INT32_TYPE, lexer.INT64_TYPE` to the case list\n\n### 3. AST — `ast/ast.go`\n\nNo changes needed — types stored as strings.\n\n### 4. Codegen — `codegen/codegen.go`\n\n- **`occamTypeToGoBase()`** (~line 1400): Add cases `\"INT16\" → \"int16\"`, `\"INT32\" → \"int32\"`, `\"INT64\" → \"int64\"`\n- **`occamTypeToGo()`** (~line 1421): Add same cases\n- **`generateMostExpr()`** (~line 926): Add cases for INT16/INT32/INT64 mapping to `math.MinInt16`/`math.MaxInt16`, etc.\n- **`exprNeedsMath()`** (~line 893): Current check is `return e.ExprType != \"BYTE\"` — INT16/32/64 all need math import, so no change needed (they won't match \"BYTE\")\n\n### 5. Tests\n\n- **Parser unit tests** (`parser/parser_test.go`): Add test for INT16/INT32/INT64 variable declarations and type conversions\n- **Codegen unit tests** (`codegen/codegen_test.go`): Add test for INT16/INT32/INT64 code generation\n- **E2E tests** (`codegen/e2e_test.go` or `codegen/e2e_types_test.go`): Add test that declares INT16/32/64 variables, uses type conversions and MOSTNEG/MOSTPOS, compiles and runs\n\n### 6. Documentation\n\n- Update `TODO.md` to mark INT16/INT32/INT64 as supported\n\n## Verification\n\n```bash\ngo test ./... # all tests pass\n./occam2go -I kroc/modules/course/libsrc -D TARGET.BITS.PER.WORD=32 -o /tmp/course_out.go kroc/modules/course/libsrc/course.module\ngo vet /tmp/course_out.go # course module still compiles\n```\n"
}The user doesn't want to proceed with this tool use. The tool use was rejected (eg. if it was a file edit, the new_string was NOT written to the file). STOP what you are doing and wait for the user to tell you how to proceed.