- The constraint
- Small teams with no AI engineers and three kinds of problem that needed different deployment targets: a customer-facing assistant on cloud models, policy questions that could never leave the building, and back-office work nobody had time to script.
- Who I talked to
- founder
Context
Since late 2024 I have worked part-time with early-stage companies that wanted AI in production but had no one to build it. The work combines architectural guidance with hands-on delivery, and it follows the problem rather than a favourite stack: some of it belongs in the cloud, some of it must never leave the building.
The constraint
Each problem wanted a different deployment target. A shopping assistant for customers could use cloud models and needed to be live quickly. Questions over internal company policies involved documents that must never be sent to an external API, so that system had to run entirely inside the firewall. Back-office routines had to be operable by the people who already did the work, not by an engineer on call.
Architecture
graph LR B[Business<br/>and customers] --> SA[Shopping assistant<br/>Gemini on Vertex AI] B --> DA[Document analyzer<br/>on-prem, offline] B --> OPS[Back-office automation<br/>CLI agentic harness] SA --> G[Tool calling<br/>+ guardrails] G --> CAT[(Store catalogue)] DA --> R[(Local vector index<br/>+ local model)] OPS --> T[Typed tools<br/>+ approval steps]
The shopping assistant runs on Gemini through the Vertex AI API. It answers product questions and makes recommendations by calling tools against the store’s catalogue, inside a conversation-flow layer with guardrails around what it may say and do. The document analyzer is retrieval-augmented generation over company policies with a local model and a local index and no internet path at all. The back-office work runs through a command-line agentic harness with typed tools and approval steps where they matter.
What I did
I designed and built all three kinds of system, from architecture through production. I worked directly with the founders, which is how the deployment target for each system was decided: by what the data was, not by what was fashionable.
Outcome
These companies moved from AI experiments to production systems without hiring an AI team. The document analyzer shows that secure knowledge retrieval is viable where data cannot leave the building, and processes that used to need dedicated engineering effort now run as repeatable operations the business owns. The same pattern, one harness deployed wherever the data is allowed to live, is what I apply at government scale.
Stack
PythonGoogle Vertex AIGeminitool callingOllamaRAGPostgreSQLCLI agentic harnessguardrailsevaluation