How I Automated My Trading Strategy Using AWS Cloud for Free (Part 1)

This year I launched a strategy subscription service for a long-only ETF strategy developed in house. I learned a lot through this process but I made several mistakes that pushed me to learn new skills and improve the product offering. In this series I will discuss my initial mistakes, and how correcting them led me to automate the system using AWS cloud and how you can too. 

Mistake #1

First mistake was not considering automation in the beginning. I had the script, I ran it daily, used the signals and put in the orders. It seemed basic and simple. And to an extent it was. But as time went on, the top feedback I received was that it was too time consuming to maintain manually. To be honest I was also fatiguing at using different scripts along with manual intervention to maintain the system. In the age of AI and automation it was unacceptable. 

Mistake #2

Second mistake was not considering hosting the script in the cloud. I had already started moving to automate some tasks. I had task schedulers and cron jobs running regular updates. However, the task schedulers in Microsoft were buggy. I created one and it disappeared. I created another for a different time and then they both started working. I was having difficulty tracking down the duplicate. I said forget it for now, I’ll run Linux cron jobs instead. At first it was ok but then I quickly discovered that cron jobs don’t have system permissions to wake up the computer (from sleep/hibernation) in order to run the job. So if the computer was sleeping the job wouldn’t run and the updates didn’t get sent. I tried to resolve this by using a shell script but that also failed to consistently wake up the computer long enough to run the task. 

At this point it became painfully obvious that a cloud solution was needed to resolve these and other issues. In earnest I began researching various cloud providers before deciding to use AWS. I chose AWS for several reasons but what I liked most were the generous free tiers that would not be exceeded using these scripts. This would also allow me to test the system with no upfront costs. At that point I began learning about the different services and how they worked together. 

The Turning Point

Eventually, after the initial frustration, the desire to build a better product and learn something new kicked in and this is how I began learning about system design. Eventually I settled on the below system architecture. Over the next few blogs I’ll be discussing what I learned through the process and shilling my skills for potential clients who come across this. Also if there’s enough demand from people who want to learn how to do it themselves I will create a course which will cover this process, in detail, from start to finish from strategy development, to system design, to implementation.

The Architecture Preview

View fullsize

retirement algorithm Trading system architecture

I’ll be using the following architecture diagram as the outline of the upcoming series of posts. I’ll talk a little bit about the design decisions and tradeoffs including areas where that I want to improve the system.

What I really like about the diagram is that it shows how simply API’s can be swapped, notification systems changed or DB’s altered.

Conclusion

If you would like to work with me on a similar project fill out the [form at this link] or contact me via [bcr@blackarbs.com]. For comments questions and feedback, leave a comment, join the discord or send an email. I’m also looking for ideas about which strategy I should demonstrate in the upcoming tutorial.