Project Update_iVC Reporting Engine
/Still working industriously behind the scenes I thought to take some time and give a progress report. Good news is the iVC Reporting Engine is almost fully operational. I've been able to automate the following processes:
Data Collection: I have two methods to obtain public company filings from the SEC via Python scripts.
- The primary method I use leverages the excellent services of the free (for now) data provider, Quandl.com. They aggregate and distribute data from several primary sources including the SEC and normalize as best as possible given whatever business constraints they have as a young riser. In congruence with their mission statement to improve data accessibility for all, they have also provided a Python module and general API for the motivated entities out there to automate data queries.
- Additionally I have a backup database of every public company filing since 2012, which leverages Arelle's open-source efforts to standardize and improve the XBRL standard for finance-IT adoption. My db is configured to use the PostgreSQL standard with Arelle's schema. This still requires more examination of the XBRL format, and/or command line interfaces to return the data I require. Marked: in Development for production purposes.
- Data Processing: There is overlap with the above, however this includes organizing/filing of the data returned by the collection process. Not a lot to say here other than the files are self-organizing and allow for flexible structures during continued operational development.
Calculation Engine: This is pretty much the final major component before large scale testing can begin. It is broken down as follows.
- I've been able to transition ~99% of my previous Excel and Python calculations into this component. It accesses the processed data, and strips it down to run the calculations as efficiently as possible currently averaging ~15.5 secs per cycle.
Under Construction: All that's left really is to incorporate the plotting/charting functions and final production output format. Once that is complete live production testing can begin.
You better tell somebody, Blackarbs LLC is coming.
