This page gives a brief overview of some projects I have completed. The GitHub repos’ README.md files go into more detail if you are curious.

Learning to Play Blackjack through Deep Reinforcement Learning

https://github.com/Jamie-Rodriguez/RL-Blackjack

Using OpenAI Gym for the environment, I created an agent that can learn to play blackjack with the most optimal strategy through deep reinforcement learning (RL).

Before moving to full function approximation, I first implemented a tabular method of Monte Carlo sampling. This was to test and ensure that the RL algorithm was functioning correctly, before adding a neural network on top.

Once I was satisfied with the RL algorithm, it was time to replace the state-action table with a function approximator - in this case a neural network.

The deep reinforcement learning agent was able to converge to the optimal solution, however it took much longer than simpler tabular method. This is an expected cost of using function approximation over tabular methods however.

See my GitHub link for more in-depth explanation and analysis.

HTMLify — Convert Source-Code to Semantically-Labelled HTML

https://github.com/Jamie-Rodriguez/htmlify

This tool was made to help myself with this very site! It converts source-code into semantically-labelled HTML where the CSS class attribute specifies the type of lexical token. From there you just need a CSS stylesheet specifying styles for the lexical token classes to have syntax highlighting!

In order to be confident that my lexer/parser can handle any source-code thrown at it, I implemented quite rigorous testing for this project using a lot of property-based tests.

I choose Rust for this project as

  1. It’s a language I had my eye on as it promises some big benefits and I wanted to verify these claims, and explore what the ecosystem is like in a medium-sized project.
  2. It was a nice level of abstraction for this project — high enough that the standard library includes useful data structures such as maps and sets, and yet low enough to have good performance.

See the GitHub repository for more specific details.

Tic-Tac-Clojure; Experiments with AI

https://github.com/Jamie-Rodriguez/tic-tac-clojure

Implemented a tic-tac-toe game engine in Clojure, in a purely functional style, using a modern Clojure stack (CLI + Deps).

I created this for two reasons:

  1. To learn the modern, in-built Clojure development ecosystem, specifically using CLI + Deps with unit testing.
  2. To use this engine/environment to experiment with reinforcement learning agents.

I then created an AI agent that can play the game. It uses an algorithm called Monte Carlo tree search which, as the name describes, uses the Monte Carlo sampling technique to approximate and then search the game tree.

Will probably experiment with more classical reinforcement learning algorithms in the future.

4-Clojure; An event-sourced, CQRS banking system written in Clojure

https://github.com/Jamie-Rodriguez/4-clojure

This is the result of my experimental implementation of an event-sourced (ES), CQRS system. I deliberately wanted to experiment with writing my own ES+CQRS system without a library, in order to gain a deeper understanding of how such systems are designed.

I had a small amount of experience with CQRS systems at work that used Elixir and the library called Commanded for CQRS; but I wanted to understand the CQRS architecture and concept at a deeper level. I found that the best way to do this was to write my own system from scratch.

I was finding that the CQRS system at work was quite complex and difficult to extend. I wanted to see if I could create a simpler system that was easier to extend and maintain. For this, Clojure was perfect as it is an extremely expressive language and it made prototyping a breeze.

In the end, I found that my implementation was very simple and easy to extend compared to the system that I had to use at work. It changed my view that CQRS is not necessarily complex or hard to maintain as I had previously experienced and read online.

In the future I might extend this project to have more domain logic/features and also to handle more complex scenarios such as events that may arrive out of order. I may also experiment with different testing strategies for this system.

Lost in Monad’s Land — Monads in C++

https://github.com/Jamie-Rodriguez/cpp-monads

What better way to understand how something works than to make one yourself?

This repo is the result of me trying to ascertain how the IO monad works. For fun I decided to write it in a language that is the polar-opposite of a purely functional programming language - interestingly it did not turn out as convoluted as I thought.

I actually started first by implementing other monads that I already had a decent understanding of their behaviour - Maybe and List, which helped gain a solid general understanding of the monad pattern before applying it to the IO monad.

Adventures in Modern C Development: HTTP Server

https://github.com/Jamie-Rodriguez/c-http-server

Minimal example of a HTTP server using the POSIX sockets library, written in C.

I created this project simply to see how difficult it is to write a HTTP server using C. Would I recommend it? Unless it is a simple server, I would say… “no”.

However I think that this repo is useful as a template for a modern C project that includes unit testing for test-driven development.

I organised this project with a more modern, yet minimal project structure; featuring unit testing using the lightweight library Unity.

To remain minimalistic, this project only uses Make for it’s buildsystem. The makefile makes use of Make’s “pattern rules” so that you don’t need to individually include a file in both the runtime and unit-testing builds every time you add a new file to the project.

Infrastructure as Code - AWS Static Site

https://github.com/Jamie-Rodriguez/aws-site

AWS Infrastructure as Code to deploy a static site fully automatically (and cheaply). It’s how this site is deployed!

See GitHub link for details of the resources and architecture created.