Tag: Education

New YouTube Series Launched: Building Your AWS Trading Data Pipeline!

I just published Part 1 of my new YouTube series, and I'm excited to share it with you all! After my recent post about automating trading strategies with AWS Cloud, many of you asked for a deeper dive into the technical implementation. Well, here it is! What's in Part 1? In this first video, we're tackling the foundation: configuring your AWS data pipeline and writing the Python code to make it all work. I walk through the exact steps I use in my own automated trading system, showing you how

READ MORE →

How I Automated My Trading Strategy Using AWS Cloud for Free (Part 1)

This year I launched a strategy subscription service for a long-only ETF strategy developed in house. I learned a lot through this process but I made several mistakes that pushed me to learn new skills and improve the product offering. In this series I will discuss my initial mistakes, and how correcting them led me to automate the system using AWS cloud and how you can too. Mistake #1 First mistake was not considering automation in the beginning. I had the script, I ran it daily, used the s

READ MORE →

How to Get (Almost) Free Tick Data

Access to high quality, cost effective market data is a continuing problem for retail traders. I was recently told about the ongoing efforts of the startup brokerage “Alpaca”. The gentleman I spoke with said the API gave access to the tick data of thousands of stocks everyday and without cost. I thought it was too good to be true but recently I took a little bit of time to investigate. In this article I will describe the basic process to accessing the tick data, and some basic code I was exper

READ MORE →

Mean Reversion Strategies in Python (Course Review)

This post contains affiliate links. An affiliate link means Blackarbs may receive compensation if you make a purchase through the link, without any extra cost to you. Blackarbs strives to promote only products and services which provide value to my business and those which I believe could help you, the reader. In this post I will be reviewing the course “Mean Reversion Strategies by Dr. E.P. Chan” (<—affiliate link, use discount code LONGSHORT for an additional 5% off at checkout). First off, f

READ MORE →

Introducing the Take Profits Strategy Subscription

Before I address the topic of this post, I want to provide a little background context for the new initiatives of Blackarbs. If you want to skip to the offer just click this link. The Beginning I started my career in finance, fresh out of college in 2008/2009, at a proprietary trading shop. At that time, I had very little experience trading, but due to the referral of a good friend, in combination with my background in low-stakes no-limit hold-em poker, I was able to land the job. It was an e

READ MORE →

Labeling and Meta-Labeling Returns for ML Prediction

Post Outline * Introduction * Links * Embedded Notebook INTRODUCTION This post focuses on Chapter 3 in the new book Advances in Financial Machine Learning by Marcos Lopez De Prado. In this chapter De Prado demonstrates a workflow for improved return labeling for the purposes of supervised classification models. He introduces multiple concepts but focuses on the Triple-Barrier Labeling method, which incorporates profit-taking, stop-loss, and holding period information, and also meta-labeli

READ MORE →

How to Build a Sequential Option Scraper with Python and Requests

Post Outline * Recap * The Problem * The Solution * Barchart Scraper Class * Barchart Parser Class * Utility Functions * Putting it all together * The Simple Trick * Next Steps Recap In the previous post I revealed a web scraping trick that allows us to defeat AJAX/JavaScript based web pages and extract the tables we need. We also covered how to use that trick to scrape a large volume of options prices quickly and asynchronously using the combination of aiohttp and asyncio. The Pr

READ MORE →

How to Scrape and Parse 600 ETF Options in 10 mins with Python and Asyncio

Post Outline * Intro * Disclaimers * The Secret to Scraping AJAX Sites * The async_option_scraper script * first_async_scraper class * expirys class * xp_async_scraper class * last_price_scraper class * The option_parser Module * The Implementation Script * References Intro This is Part 1 of a new series I'm doing in semi real-time to build a functional options data dashboard using Python. There are many underlying motivations to attempt this, and several challenges to imp

READ MORE →

Can We Use Mixture Models to Predict Market Bottoms?

Post Outline * Recap * Hypothesis * Strategy * Conclusion * Caveats and Areas of Exploration * References Recap In Part 1 we learned about Hidden Markov Models and their application using a toy example involving a lazy pet dog. In Part 2 we learned about the expectation-maximization algorithm, K-Means, and how Mixture Models improve on K-Means weaknesses. If you still have some questions or fuzzy understanding about these topics, I would recommend reviewing the prior posts. In those po

READ MORE →

Intro to Expectation-Maximization, K-Means, Gaussian Mixture Models with Python, Sklearn

Post Outline * Part 1 Recap * Part 2 Goals * Jupyter (IPython) Notebook * References part 1 recap In part 1 of this series we got a feel for Markov Models, Hidden Markov Models, and their applications. We went through the process of using a hidden Markov model to solve a toy problem involving a pet dog. We concluded the article by going through a high level quant finance application of Gaussian mixture models to detect historical regimes. part 2 goals In this post, my goal is to impar

READ MORE →

Introduction to Hidden Markov Models with Python Networkx and Sklearn

Post Outline * Who is Andrey Markov? * What is the Markov Property? * What is a Markov Model? * What makes a Markov Model Hidden? * A Hidden Markov Model for Regime Detection * Conclusion * References Who is Andrey Markov? Markov was a Russian mathematician best known for his work on stochastic processes. The focus of his early work was number theory but after 1900 he focused on probability theory, so much so that he taught courses after his official retirement in 1905 until his death

READ MORE →

Understanding Hidden Variables with Python - Research Roadmap

Post Outline * Motivating the Journey * Where Do Edges Come From? * The Problem with Traditional Research * The Hidden Side * A Brief Description: * Part 1 - A Visual Introduction to Hidden Markov Models with Python * Part 2 - Exploring Mixture Models with Scikit-Learn and Python * Part 3 - Predicting Market Bottoms with Scikit-Learn and Python Motivating the Journey Where do Edges Come From? Edges come from superior ability to identify and execute profitable strategies

READ MORE →

Advanced Time Series Plots in Python

POST OUTLINE 1. Motivation 2. Get Data 3. Default Plot with Recession Shading 4. Add Chart Titles, Axis Labels, Fancy Legend, Horizontal Line 5. Format X and Y Axis Tick Labels 6. Change Font and Add Data Markers 7. Add Annotations 8. Add Logo/Watermarks MOTIVATION Since I started this blog a few years ago, one of my obsessions is creating good looking, informative plots/charts. I've spent an inordinate amount of time learning how to do this and it is still a work in a progress. Howe

READ MORE →

Time Series Analysis (TSA) in Python - Linear Models to GARCH

If you’re interested in automating your strategy or process using AWS Cloud click here. Post Outline * Motivation * The Basics * Stationarity * Serial Correlation (Autocorrelation) * Why do we care about Serial Correlation? * White Noise and Random Walks * Linear Models * Log-Linear Models * Autoregressive Models - AR(p) * Moving Average Models - MA(q) * Autoregressive Moving Average Models - ARMA(p, q) * Autoregressive Integrated Moving Average Models -

READ MORE →

Aggregating Free Options Data with Python

Post Outline * Motivation * Code Requirements * Creating our Scraper Class * Aggregating the Data * Github Gist Code * Disclaimers Motivation This year I implemented a simulated trading strategy based on the research paper titled "What Does Individual Option Volatility Smirk Tell Us About Future Equity Returns" by Yuhang Xing, Xiaoyan Zhang and Rui Zhao. The authors show that their SKEW factor has predictive power for equity returns for up to 6 months. Because historical options data

READ MORE →

Estimated Longevity of a Retirement Portfolio in Years (Python)

Post Outline: * What is Shortfall Risk (Shortfall Probability, Ruin Risk, Risk of Ruin etc.)? * Issues with Traditional Shortfall Risk Estimates * Key Insights (Solutions) * Python Code (Implementation) What is Shortfall Risk (Shortfall Probability, Risk of Ruin, etc.)? portfolio value will fall below some minimum acceptable levelover some time horizonAnalystNotes.com Issues with Traditional Shortfall Risk Estimates In the article "It's Time to Retire Ruin (Probabilities)" the author,

READ MORE →

How to get Free Intraday Stock Data with Python and BarCharts OnDemand API

To this day the most popular article I have ever written on this blog was "How to get Free Intraday Stock Data with Netfonds". Unfortunately the Netfonds API has really declined in terms of usability, with too many popular stocks missing, and irregular trade and price quotes. Simply put, as the API went down, so did the code. However, all hope is not lost. The wonderful people at BarChart.com have created a well documented, easily accessible API for intraday stock data and even near real-time q

READ MORE →

Guest Post Previously Featured on RectitudeMarket.com (09/02/2015)

**Note: This post already appeared as a guest post on rectitudemarket.com. The reason I'm posting this article when it is 'outdated', is twofold. 1) I think it's beneficial to review previous works especially when one has the benefit of hindsight. This helps us determine the accuracy and bias of the research presented. 2) I further introduce the concept of conditional cumulative returns, which adds insight to what happens to our securities' returns given some other event occurring. In this case,

READ MORE →

Get Free Financial Data w/ Python (Fundamental Ratios-From Finviz.com)

A simple script to scrape fundamental ratios from Finviz.com. This basic code can be tailored to suit your application. The function should produce the following: Very simple adaptable code, allowing you to spend more time analyzing the data and less time aggregating it.

READ MORE →

How do Bank Stocks Perform During Periods of Rising Rates? (Python Code Version)

This is the Python version of a guest article that originally appeared on RectitudeMarket.com. In this version I include the Python code used to generate the anaylsis. This subject has garnered a healthy debate among market participants in recent weeks. Conventional wisdom says that banks and the financial sector overall should benefit from a rising rate environment. The story goes that bank profitability is inextricably linked to `Net Interest Margin (NIM)`. If rates are rising, it is assumed

READ MORE →

How to get Free Intraday Stock Data from Netfonds

Daily stock data is everywhere for free. Yahoo, Google, and Quandl all provide useful daily stock prices for basic number crunching. However computational analysis for intraday stock data is much harder to find. In fact, Intraday stock data can be very expensive. So what is a cost conscious quant supposed to do? The Norwegian website Netfonds.no provides free intraday data on stocks and ETF's on the NYSE, Nasdaq, and Amex exchanges. They provide up to 5 days of trade/bid/offer data. I wrote som

READ MORE →

What I'm Watching ( Week of 12.29.14 )

This week it's simple. I'm watching/waiting for volume to reenter the market and confirm or reject last week's holiday action. Last week ( SPY, DIA, QQQ ) all traded to new 52 week highs with my short target ( EWC ) bouncing significantly in sympathy. ( OIL ) price action appears to have moderated a little in terms of downside volatility; buyers and sellers that remained haven't shown conviction one way or the other yet. There is good reason to be skeptical of last week's broad market moves.

READ MORE →

What I'm Watching ( Week of 12.15.14 )

Canada iShares ( EWC ) Canadian assets are facing major headwinds in the current environment. 1. The fall in global oil prices hurt producers' bottom line. Alberta is bracing for a decline in GDP into next year as a result. Click for links to articles. Quarter of new Canadian oil projects vulnerable if oil falls below US$80: IEA With oil trading below US$60, provinces brace for impact of global oil price shock The above chart shows that there are some producers' breakeven Oil price as low

READ MORE →

What I Saw ( Week of 12.8.14 )

Tuesday of this week I expressed a mildly bearish sentiment regarding the broad market. Based on stock market internals including weakening market participation, the duration and velocity of the near vertical bounce off of the October lows it seemed increasingly likely that we would see some profit taking and some sort of correction. Let's see if this estimation was validated by the market. I would say it looks like market participants did decide it was a good time for profit taking/risk reduct

READ MORE →

What I'm Watching ( Week of 12-8-14 )

How long does equity optimism last without correction? Over the weekend I examined the market internals and came away mildly bearish at these levels. Extreme optimism and lack of participation among the broad indexes make this a week ripe for profit taking and I've been aggressive with my trailing stops etc during the last 2 trading sessions. When will OIL finally bounce? Have investors finally capitulated? I'm short oil via the Inverse ETF SCO ( wish I had more size ) however this has been a

READ MORE →

What I'm Watching ( Week of 12.1.2014 )

1. Do the airline stocks and oil continue to trend strongly in opposite directions? 2. Do we see a continuation of last week's sell off in small caps and does that translate to a mini correction? 3. When risk (high beta small caps/high yield) is sold do we see a continued bid in Utilities and Consumer Staples? 4. Keeping an eye out on high yield bonds ( HYG ) and Investment grade corporate bonds ( LQD ) for clues... What are you watching this week?

READ MORE →