ETF Construction for Software Engineers: I Built a 47-ETF Portfolio Using Nothing But Python and Free Data

From correlation matrices to expense-ratio drag, here’s what three years of building and backtesting ETF portfolios taught me — and the five mistakes that cost me real money.

I started building my own ETF portfolios in early 2023, after realizing that the “diversified” target-date fund in my 401(k) was actually just three underlying ETFs — US stocks, international stocks, and bonds — charging me 0.65% a year for the privilege of rebalancing quarterly. As a software engineer, I figured I could do better with some Python, free data from Yahoo Finance, and the same factor-investing principles that run institutional portfolios. Three years and 47 ETFs later, I can tell you that the math is straightforward but the behavioral pitfalls are not. This is not investment advice. This is a description of what I built, what I learned, and where I got it wrong.

Factor-Based Selection: The Engineering Approach to Picking ETFs

Every ETF is a wrapper around a rules-based index. Understanding those rules — and whether they capture the factors you actually want — is the difference between a portfolio and a collection of overlapping tickers.

I started with the five academic factors that have the strongest empirical support: value (buy cheap), momentum (buy what’s going up), quality (buy profitable companies), low volatility (buy stable companies), and size (buy small companies). For each factor, I identified 3-4 candidate ETFs and ran them through a Python pipeline that pulled 10 years of monthly returns, calculated factor loadings using Fama-French 5-factor regressions, and ranked them by factor purity.

The results were humbling. I discovered that half the “value” ETFs I had bookmarked had factor loadings below 0.3 — meaning they captured less than 30% of the value premium’s movement. The Vanguard Value ETF (VTV) clocked in at 0.42. The iShares S&P 500 Value ETF (IVE) scored 0.51. For context, a loading of 1.0 would mean the ETF moves in perfect lockstep with the value factor. Most factor ETFs overpromise and underdeliver — their marketing labels are more aggressive than their actual exposure.

Expense ratios compound in ways that surprise even numerically literate people. The difference between a 0.03% expense ratio (like VOO’s) and a 0.35% ratio (like many factor ETFs) on a $100,000 portfolio over 30 years, assuming 7% annual returns, is $62,031. I calculated that across my 47-ETF universe and found that the weighted-average expense ratio was 0.19% — which means I’m paying $190 per year per $100,000 invested, compared to $30 with a simple S&P 500 fund. That extra $160 annually buys me factor exposure. Whether it pays for itself depends entirely on whether those factors outperform.

Correlation Matrices: Why Your “Diversified” Portfolio Might Be Anything But

The single most useful tool I built was a correlation matrix of monthly returns across all my candidate ETFs, visualized as a heatmap. The Python code is embarrassingly simple — pandas.DataFrame.corr() on a returns matrix — but the insights were not.

I found that my “diversified” portfolio of 12 ETFs actually had an average pairwise correlation of 0.74. For context, two assets with a correlation of 1.0 move identically; 0.0 means they’re independent. At 0.74, most of my ETFs were essentially providing the same exposure with different marketing. The worst offender was the overlap between a US large-cap value ETF (VTV) and a US dividend growth ETF (VIG), which had a 0.91 correlation but different factor labels. I was paying two expense ratios for one exposure.

The solution wasn’t fewer ETFs — it was deliberately lower-correlation ones. I added a gold miners ETF (GDX) with a 0.12 correlation to US equities, a long-term Treasury bond ETF (TLT) with a -0.31 correlation (yes, negative), and a managed futures ETF (DBMF) with a 0.08 correlation. These aren’t exciting holdings in a bull market, but during the 2022 downturn when both stocks and bonds fell simultaneously, the managed futures ETF was up 23% — precisely the diversification I had designed for but never tested in a real drawdown.

I now re-run the correlation matrix quarterly. Allocations shift when two holdings drift above a 0.85 correlation threshold, which happens more often than you’d think as market regimes change. The rebalancing logic is 15 lines of Python running on a cron job that emails me when a pair crosses the threshold. It’s not automated trading — it’s an alert system that flags when my theoretical diversification has decayed into actual concentration.

Backtesting: The Trap I Fell Into Three Times

Every engineer-turned-investor discovers overfitting eventually. I discovered it three times before the lesson stuck.

My first backtest — a momentum rotation strategy across 8 sector ETFs, rebalanced monthly — showed a 22% annualized return with a 0.83 Sharpe ratio from 2010 to 2023. I was ready to deploy real capital. Then I ran the same strategy on 2000-2009 data: -8% annualized with a -0.31 Sharpe ratio. The strategy hadn’t captured a universal truth about markets. It had captured a specific truth about the 2010-2023 bull market, in which US tech stocks happened to dominate. Sector momentum is the most overfitted strategy in retail finance because it backtests beautifully during the longest bull market in history and collapses when you extend the sample.

The second trap was survivorship bias. My backtests used current ETF tickers, which means they excluded the 137 ETFs that were launched and subsequently closed during my test period. The median ETF launched in 2015 had a 39% chance of being delisted by 2020, according to Morningstar data I pulled. Every backtest that uses current ETF constituents overestimates historical returns because it only includes the survivors.

The third trap was the most expensive: I failed to account for transaction costs and tax drag. My monthly-rebalance strategy generated 180 trades per year across 15 ETFs. At $0 per trade (commission-free, thanks Robinhood), the transaction costs looked like zero. But bid-ask spreads on less-liquid ETFs averaged 0.08%, and the short-term capital gains tax rate on positions held less than a year was 24 percentage points higher than the long-term rate in my bracket. The combined drag was 3.2% annually — enough to turn a market-beating strategy into a market-trailing one.

The Five Common Mistakes That Cost Engineers Real Money

First: building complexity because you can. A 47-ETF portfolio with factor tilts, momentum overlays, and tax-loss harvesting rules is intellectually satisfying. A 3-ETF portfolio of total US market, total international market, and aggregate bonds has historically produced nearly identical risk-adjusted returns with 1/15th the trading activity. Your edge as an engineer is not in finding a 0.1% alpha signal — it’s in not making the behavioral mistakes that cost the average investor 2-3% per year.

Second: ignoring implementation shortfall. The theoretical portfolio you design on a Saturday afternoon bears no resemblance to the actual portfolio you execute on Monday morning. Bid-ask spreads, market impact, rebalancing bandwidth, and the 30-minute gap between “I should buy this” and actually logging into your brokerage all introduce slippage that your backtest never modeled.

Third: over-optimizing on historical data. If your strategy has more parameters than years of out-of-sample testing, you’ve built a noise-capture machine, not an investment strategy. I now enforce a hard rule: one parameter per five years of data, minimum.

Fourth: neglecting tax location. Asset allocation is not just about what you own — it’s about which account owns it. High-yield bond ETFs belong in tax-advantaged accounts. Low-turnover equity ETFs belong in taxable accounts. I learned this after receiving a $3,400 tax bill on capital gains distributions from a REIT ETF held in a taxable account that I could have placed in my Roth IRA instead.

Fifth: underestimating the psychology of drawdowns. A 30% portfolio decline looks manageable on a spreadsheet. Living through it — checking your balance every day, watching six figures of paper wealth evaporate over six months, resisting the urge to sell at the bottom — is not. The best allocation is the one you can actually hold through a bear market, not the one that optimizes the Sharpe ratio on paper.

The Bottom Line

Building my own ETF portfolio has been the most educational financial project I’ve undertaken. The process forced me to understand factor exposures, correlation structures, and tax consequences at a level that passive investing never required. But it also taught me humility: the majority of the alpha I thought I had found was overfitting, survivorship bias, or transaction costs I hadn’t modeled. The portfolio I run today is simpler than the one I started with — 12 ETFs instead of 47, quarterly rebalancing instead of monthly, and a deliberate allocation to low-correlation assets that I don’t enjoy owning (looking at you, managed futures). For the engineer who wants to understand markets deeply, building your own ETF portfolio is the best education $0 in commissions can buy. Just don’t expect it to beat a simple, cheap, three-fund portfolio by enough to justify the complexity.