# Rodrigo Vaz **Software Engineer – AI Systems, LLM Evaluation & Reliability** Backend Engineer | AI Engineer | LLM Systems Remote (Brazil) · [email protected] · [github.com/ShriekingNinja](https://github.com/ShriekingNinja) · [berkano.io](https://berkano.io) · [wk.al](https://wk.al) ----- ## Summary Software engineer with 10+ years of experience building reliable systems across industrial and software environments, with recent focus on AI systems and LLM evaluation. Designed and implemented a modular validation framework for language model outputs, including hallucination detection, contradiction analysis, adversarial testing, and structured output verification. Strong background in fault analysis, root-cause investigation, and reliability engineering applied to AI pipeline architecture. ----- ## Core Skills **AI & Machine Learning:** LLM evaluation · Prompt engineering · Adversarial testing / red-teaming · Hallucination & contradiction detection · Output validation pipelines · HuggingFace · LangChain **Programming & Backend:** Python (primary) · C/C++ · Ruby · JavaScript · Node.js · Express.js · Ruby on Rails · REST APIs · API development · System design · Git · Linux **Databases & Infrastructure:** PostgreSQL · MongoDB · Structured logging · Reproducible experiment pipelines **Reliability Engineering:** Fault analysis · Root-cause investigation · Acceptance testing · TTD/TTR metrics · No-drift enforcement · Rollback & repair architecture ----- ## Projects ### LLM Validation & Reliability Framework — Berkano Protocol *Independent · Open Source (GPL-3.0) · 2023 – Present* [berkano.io](https://berkano.io) · [github.com/ShriekingNinja](https://github.com/ShriekingNinja) - **Live Demo:** <https://huggingface.co/spaces/berkano-protocol/demo> — LLM output validation, contradiction detection, and structured response gating. - Built modular validation middleware for LLM pipelines: factual verification, contradiction detection, structured reasoning checks. - Implemented input/output validation layers for structured generation workflows, including parsing, gating, and verification mechanisms. - Integrated validation components into LLM pipelines and backend services for real-time output validation. - Structured system for real-time validation of LLM outputs with deterministic gating and repair mechanisms. - Built adversarial test suites simulating prompt injection, urgency manipulation, and instruction hijacking. - Implemented rollback and repair mechanisms for output correction, with append-only structured logging. - Defined evaluation metrics: time-to-detect (TTD), time-to-repair (TTR), severity classification (C0–C4). - Produced 1.6M+ words of structured experiment logs across 1,186 entries — open-source, publicly verifiable. ### Roboinvest — Stock Analysis Platform *Full-Stack Project* - Built full-stack investment analysis platform with web scraping, real-time data processing, and decision support logic. - Designed and implemented backend APIs and data pipelines for financial data collection and processing. - Created dashboards for portfolio tracking and visualization. - Stack: Ruby on Rails, PostgreSQL, JavaScript. ### AluGames — Board Game Rental Marketplace *Full-Stack Project* - Built full-stack marketplace with backend APIs, user accounts, inventory management, and booking logic. - Stack: Ruby on Rails, Bootstrap. ----- ## Professional Experience ### Industrial Commissioning Engineer / Systems Engineer *Various Projects · 10+ years* - Led commissioning, validation, and fault diagnosis of large-scale industrial systems. - Designed and executed system acceptance tests, SOPs, and operational procedures. - Specialized in root-cause analysis and system reliability under real-world production constraints. - Managed and coordinated commissioning teams and system integration workflows. - Contributed to development and deployment of PCMsys — project management and commissioning system. - Applied no-drift, rollback-as-default, and structural debugging mindset now carried directly into AI reliability and pipeline work. ----- ## Open Source & Public Work **Protocol & Docs:** [berkano.io](https://berkano.io) **Research Logs:** [wk.al](https://wk.al) **GitHub:** [github.com/ShriekingNinja](https://github.com/ShriekingNinja) ----- ## Education **BSc Electrical Engineering** *(non-traditional path)* **Additional training:** PLC Programming · Industrial Automation & Instrumentation · BS7671 Electrical Systems ----- ## Systems Focus Focus on reliability, validation, and fault-tolerant system design across both industrial and AI environments. Experience applying: - Rollback-first architecture - Structured validation layers - Failure detection and repair loops - Deterministic output gating for LLM systems ----- ## Additional - Languages: Portuguese (native), English (fluent) - Strong independent contributor — high output capacity, comfortable working autonomously and iterating rapidly - Able to collaborate with research teams and contribute to technical writing and documentation