mattpearson blog

S&P500 December Seasonality

Sun, 11 Dec 2016 00:00:00 GMT

As we approach the last weeks of the year, something to keep in mind for December’s seasonality. X axis is day of month, and Y axis is cumulative return of S&P500 over December for years 1953 to 2016.

Average returns are nice, but knowing how volatilite it is would also be good to know. Utilizing a pivot table to calculate, the annualized standard deviation for each day is plotted below:

Python Code

import datetime
import pandas_datareader as web
import pandas as pd
import numpy as np

gspc = web.DataReader( '^GSPC', 'yahoo', datetime.datetime(1953,1,1), datetime.datetime(2016, 12, 10))

df = pd.DataFrame( [ gspc['Adj Close'] ] ).transpose()
df.columns=['SPY']
r = df['SPY'].pct_change().shift(-1)
df['SpyRet'] = r
df['DOM'] = df.index.day
df['Month'] = df.index.month

df_dec = df[ df['Month'] == 12 ]
t = pd.pivot_table( df_dec, values='SpyRet', columns='DOM', aggfunc=np.mean)
(t.cumsum()*100.0).plot( grid=True)

t = pd.pivot_table( df_dec, values='SpyRet', columns='DOM', aggfunc=np.std)
(t*np.sqrt(252)*100).plot( grid=True, title='December Return Annualized Volatility By Day')

Decoding CME's MDP 3.0

Fri, 07 Oct 2016 00:00:00 GMT

MDP stands for Market Data Protocol, and CME is on their third revision of it. In this latest revision, they decomissioned FAST encoding and replaced it with SBE, which stands for Simple Binary Encoding. Real Logic, a firm based in the UK, has developed an example SBE implementation that generates code in both Java and C++ from a SBE template file. CME’s confluence wiki page has a good primer on SBE and is a must-read for anybody serious in connecting with CME’s market data feed.

Once you are familiar with SBE you can jump right in to see the different types of messages by perusing their template file.

Note, Real Logic’s tool requires that you have Java JDK installed, so if you don’t, then run:

su -c "yum install java-1.8.0-openjdk-devel"

Next, use git to download and install Real Logic’s simple binary encoding example implementation. The following linux commands will walk you through getting the latet snapshot from their github repository, building the software, downloading the CME SBE template file, and generating C++ code that implements SBE to the CME’s spec.

$ git clone https://github.com/real-logic/simple-binary-encoding.git
$ ./gradlew
$ cd ./sbe-tool/build/libs
$ wget ftp://ftp.cmegroup.com/SBEFix/NRCert/Templates/templates_FixBinary.xml
$ java -jar -Dsbe.target.language=cpp -Dsbe.keyword.append.token=_ sbe-tool-1.5.2-SNAPSHOT-all.jar templates_FixBinary.xml

Once these commands are completed, you will have a new folder called 'mktdata' with classes (using the namespace 'mktdata') created for you by the Real Logic SBE tool using the CME-provided template file.

A huge plus is that it supports and utilizes C++11 features.

Hedging Factors

Tue, 27 Sep 2016 00:00:00 GMT

Review

The CAPM suggests that a firm’s expected excess return (Ri) above the risk-free rate (Rf) is simply a scaled version of the market portfolio (Rm) expected excess return. Risk-free rate can be the rate of return on US treasury bonds. In other words: ( Ri - Rf ) = scaling factor * ( Rm - Rf ). Put differently,

( Ri - Rf ) = alpha + beta * ( Rm - Rf ).

Gene Fama and Ken French argued convincingly that there are additional risk factors that can better describe the expected excess return, namely a size premium (SMB) and a value premium (HML). Recent research has shown that the expected return can further be improved with additional factors beyond size and value. Cliff Asness eloquently describes them as follows:

Premium Factors
- RM-RF - The return spread between the capitalization weighted stock market and cash.
- SMB - The return spread of small minus large stocks (i.e., the size effect).
- HML - The return spread of cheap minus expensive stocks (i.e., the value effect).
- RMW - The return spread of the most profitable firms minus the least profitable.
- CMA - The return spread of firms that invest conservatively minus aggressively.

With this knowledge, how can you evaluate the exposure a portfolio has to these risk factors?

Regression

By re-expressing the CAPM to use the additional risk factors, the regression can be expressed as:

( Ri - Rf ) = alpha + beta1 * ( Rm - Rf ) + beta2 * HML + beta3 * SMB + beta4 * RWM + beta5 * CMA

Ken French maintains a library of factors for doing exactly these kinds of regressions. You can access it here.

One of the most famous portfolio managers of the 20th century is Peter Lynch, who was the PM for Fidelity’s Magellan Fund. What were his factor exposures during the time he was managing the fund, from 1977 to 1990?

The regression results follow:

Obviously, his fund had a statistically strong positive relationship with the market (MKT-RF), and clearly a statistically strong negative relationship with HML and RMW factors. The negative relationship with HML suggests he was exposed to more growth stocks than value stocks, and firms that were least profitable rather than most profitable. Given the small t-statistics to SMB and CMA we can not make statistically significant conclusions as to his portfolio’s exposures to these risk factors.

Remember when you are running such regressions, subtract out the risk free rate from the returns of the market and the security or fund you are analyzing. While the current ZIRP regime we’re currently in won’t make much of a difference, when you go back to the 80s and 90s it definitely will.

How to hedge?

The analysis above, the risk factors are not tradable. Within the past few years, however, there has been a large increase in the number of exchange traded funds that provide factor exposure. If you, as an investor in the Magellan fund, wanted to hedge the exposure to growth stocks (HML factor), you can simply buy $0.30 worth of a fund that tracks HML for each $1.00 that is invested in Magellan:

After running the same regression, you can now see that the coefficient and t-statistic for the HML factor are effectively zero. In practice, use an ETF that is both easily shortable with low short-rate fee to achieve the best results.

Python Code

This code is adapted from the solid work by nakulnayyar. It has been changed to use the new five factor model, and show an example of hedging.

import pandas as pd
import numpy as np
from StringIO import StringIO
from zipfile import ZipFile
from urllib import urlopen
import pandas.io.data as web
import statsmodels.api as sm
url = urlopen("http://mba.tuck.dartmouth.edu/pages/faculty/ken.french/ftp/F-F_Research_Data_5_Factors_2x3_CSV.zip")
zipfile = ZipFile(StringIO(url.read()))
FFdata = pd.read_csv(zipfile.open('F-F_Research_Data_5_Factors_2x3.CSV'),
                     header = 0, names = ['Date','MKT-RF','SMB','HML','RMW','CMA','RF'],
                     skiprows=3)

FFdata = FFdata[:638]

//#Convert YYYYMM into Date
FFdata['Date'] = pd.to_datetime(FFdata['Date'], format = "%Y%m")
FFdata.index = FFdata['Date']
FFdata.drop(FFdata.columns[0], axis=1,inplace=True)

#Drop Days in YYYY-MM-DD
FFdata.index = FFdata.index.map(lambda x: x.strftime('%Y-%m'))

#Convert into float
FFdata = FFdata.astype('float')
FFdata.tail()

#Get Data from Yahoo
start = datetime(1980,1,1)
end = datetime(1990,10,1)
f = web.get_data_yahoo("FMAGX", start, end, interval='m')

#Delete Columns
f.drop(f.columns[[0,1,2,3,4]], axis=1, inplace=True)
#Fix Date Column
f.index = f.index.map(lambda x: x.strftime('%Y-%m'))

f['Return'] = f['Adj Close'].pct_change().shift(0) * 100.0
f.head()



data2 = pd.concat([f,FFdata], axis = 1)
data2['XRtrn'] = (data2['Return'] - data2['RF'])
df = data2[np.isfinite(data2['XRtrn'])]
df.tail()


y = df['XRtrn']


X = df.ix[:,[2,3,4,5,6]]
X = sm.add_constant(X)
model = sm.OLS(y, X)
results = model.fit()
print(results.summary())

y.plot()

#Hedge HML
y1 = y - results.params[3] * df['HML']

model = sm.OLS(y1, X)
results = model.fit()
print(results.summary())

Linux Security

Mon, 05 Sep 2016 00:00:00 GMT

What should be the first thing you do with a new linux installation? Make it more secure! They include:

Disable root login from sshd
Allow authentication only through pre-configured ssh keys.
Install fail2ban

sshd changes

First, edit /etc/ssh/sshd_config (sudo vi /etc/ssh/sshd_config), and change

#PermitRootLogin yes
#AuthorizedKeysFile      .ssh/.authorized_keys

PermitRootLogin no
AuthorizedKeysFile      .ssh/.authorized_keys

Next, generate a ssh key on your local computer for authentication. There are plenty of how-to guides to do this (such as https://devops.profitbricks.com/tutorials/use-ssh-keys-with-putty-on-windows/ for Windows and https://www.digitalocean.com/community/tutorials/how-to-set-up-ssh-keys—2 for linux) but the essence is as follows for Windows users:

Download PuTTYgen and generate a key.
For "key comment", enter your username
For "key passphrase", you can enter a passphrase for extra security if wanted.
click "Save private key" and note location and filename.
In PuTTY, create a new session, and go to Connection→SSH→Auth→Browse and choose your private key. Save this session with the IP address.
On your linux box, edit ~/.ssh/.authorized_keys and paste in the content stored in PuTTYgen section titled "Public key for pasting into OpenSSH authorized_keys file". Make sure you copy ALL of it, including the beginning "ssh-rsa" text, AND the ending text after the "==" symbols.
Confirm the .authorized_keys file is mode 600 (chmod 600 .authorized_keys).

Wrap up these changes with a sshd restart:

sudo service sshd restart

fail2ban

Fail2ban is a service that modifies your iptables rules based on activity that is seen in your log files. If someone attempts to repeatedly log in with incorrect credentials, you can choose to have their IP banned from connecting again for a period of time. A good how-to guide for installing and configuring fail2ban is here:

https://www.digitalocean.com/community/tutorials/how-to-protect-ssh-with-fail2ban-on-centos-7

CME New Release VPN

Sat, 03 Sep 2016 00:00:00 GMT

This guide is written to help others go through the process of configuring a system to connect to the Chicago Mercantile Exchange’s New Release environment over a self-hosted VPN connection. It’s meant not to be an in-depth explanation of all terms and concepts, but rather a guide/map to go from a new router to having a working connection for those with only a high level understanding of routers.

Step 1: Router

You will need a router with built-in ISAKMP and IPSEC capabilities. One of the low cost options is to go with an 800 series Cisco router such as the 871W which you can pick up on Ebay in the $50 to $60 range, and it includes 802.11 wireless capabilities. This guide is written using an 871W router.

For those readers who are not networking experts or Cisco CCNAs, there are many acronyms that are used. A few of them are briefly explained here to help get you up to speed.
- ISAKMP - This is part of the security mechanism and is a subset of IKE. http://networkengineering.stackexchange.com/questions/1/whats-the-difference-between-ike-and-isakmp
- GRE - Generic Routing Encapsulation. This allows multicast traffic to be sent over the VPN tunnel. http://www.cisco.com/c/en/us/support/docs/ip/ip-multicast/43584-mcast-over-gre.html
- PIM - Protocol Independent Multicast. You’ll need to "enable multicast routing" and "pim sparse-mode" on the router for the multicast traffic to work properly. http://www.cisco.com/c/en/us/td/docs/ios/solutions_docs/ip_multicast/White_papers/mcst_ovr.html
- VLAN - Virtual LAN. Note that on the 871W, you can not assign a specific IP or DHCP group to the physical interfaces. You must assign the interfaces to be part of a VLAN and then the VLAN can be assigned a DHCP pool for IP assignment.
- Advanced Security or Advanced IP - Cisco offers two flavors of software to run on their routers. The "PIM" capabilities are in the Advanced IP package, not the Advanced Security package despite the need for a VPN tunnel! Make sure you are running a package with the K9 designator because it has AES/3DES encryption.
- ACL - Access Lists. These allow you to choose what traffic can and can not go through your router.
- NAT - Network Address Translation. This is a technology that allows your router to create a sub network and, through the use of differing ports, pass the traffic through one (public) IP. "ip nat inside" tells the vlan to NAT ip that are connected to it, and "ip nat outside" tells the interface (you would execute this command on your WAN port) to NAT traffic to the outside world.
- dot1Q - This is to tell your wireless interface to use a specific vlan. For example "encapsulation dot1Q 10 native" uses vlan 10.

Step 2: Hardware

In addition to the 800 series Router, to get started, you’ll need a so-called "Rollover Cable". http://www.ebay.com/sch/i.html?_nkw=cisco+rollover+usb&_sacat=0 This allows you to configure the router over a serial protocol. Use putty to connect, and choose Serial for the method. For my setup, it was COM3 and 9600 baud. To connect to CME you’ll need to have two physical interfaces available on your server. One for public IP traffic, and the other for CME New Release traffic.

Step 3: The Big Picture

A high level overview of how your network topology can look is as follows. You will have two separate networks (VLANs), one for public internet traffic, and the other specifically for CME NR traffic. The Vlan IDs (10 and 20, below) correspond to the sample configuration at the end.

You’ll create two VLANs, one for public IP traffic ("Vlan 10"), and the other for CME New Release traffic ("Vlan 20"). For wireless router models, you’ll add the wireless connection to be part of Vlan 10. Assign two of the fast ethernet ports to vlan 10 (FA0 and FA1), and the other two (FA2 and FA3) to Vlan 20. This will allow you to have two physical connections to public IP and two to New Release. You can always attach a networking switch to a vlan 10 port to expand internet access to other devices. To have the appropriate IPs assigned to each VLAN, you’ll create two DHCP pools for which IPs can come. On vlan 10, you can stay with the traditional 192.168.1.x range and on vlan 20, you must use the range that the CME assigns you.

Note for Wireless users: In order to get wireless working propertly, you’ll need to create a both a sub interface and a "bridge group". At a high level, a bridge group allows you to connect the physical interface with the wireless interface into the same group. This is important: you won’t assign an IP address range to vlan 10, just tell it to be part of the bridge group and with "no ip address". Then, in the bridge group configuration, you will assign the IP range for which the DHCP pool exits. However, for vlan 20 (because it’s not part of a bridge group with a wireless interface), you will assign an ip range that matches what CME tells you. This is evident in the sample configuration.

Step 4: Configure Router / "Cisco 101"

To connect to the router, you do not use the Netgear/Linksys approach of pointing your browser to 192.168.1.1. Instead, you need to connect via the serial "rollover" cable.

Once you connect to the router, there will be a few key commands that you’ll use over and over:
- "write erase" — This will erase the "running-config" and the "startup-config". These are the file names of the two configuration files on the router.
- "copy running-config startup-config" — This will copy the configuration that you have created to be the one run when the router is rebooted.
- "enable" — This will bring the router into priviledged mode so you can issue basic show commands.
- "config t" — This will bring the router into global configuration mode where you can actually change settings.
- "exit" — Go back up one "level" in your configuration. For example if you are configuring a sub-interface and want to return back to the interface level, just type exit.
- "shut" — Brings down an interface.
- "no XXXX" — Removes or takes away a command. For example, if you type "no shut", then it will bring UP an interface. Likewise, "no ip pim sparse-mode" will REMOVE the "ip pim sparse-mode" from the configuration.
- "switchport access" — This tells the router to associate a physical interface to a particular vlan.

A couple notes on wifi connection: You’ll notice there exists an entry "interface Dot11Radio0.10". This means you are creating a sub-interface off the 802.11 wireless interface (that’s where the ".10" comes in). The number after the period is for the vlan number for which the subinterface should be part of. You also use this vlan number with your dot1Q encapulation command. See the sample configuration. "guest mode" means that the SSID is broadcast (you’ll pick it up on your access point list on your laptop/phone).

To get started, connect to the router through putty, and follow these instructions:

Execute "write erase" first (to restore factory configuration).
Next, "enable" to go into priviledged mode.
Then, "show running-config" to see what the initial configuration looks like.
Lastly, "config t" so you can start changing settings (you should see the '#' prompt which means you can now issue commands).

Once you are in this mode, type in the commands from the attached configuration file. Replace the XXX.XXX.XXX.XXX IPs with those provided by CME.

Step 5: Configure Linux

In order to get this to work properly, you will need to: 1. diable RP filtering, 2. configure your firewall properly (or just turn it off), and 3. add the appropriate routing table entries so that traffic to CME goes to the correct interface. On the commands that follow, the interface for CME is enp3s0 (change it to match yours).

Routing table: route add -net XXX.YYY.ZZZ.0 netmask 255.255.255.0 gw AAA.BBB.CCC.1 dev enp3s0 (Change the X,Y,Z, and A,B,C to the IP range assignedby CME).

Also add the route for multicast traffic to the appropriate interface: sudo route add -net 224.0.0.0/8 dev enp3s0

To join a multicast group on ip 224.0.28.1:14310, you can use the command: iperf -s -u -B 224.0.28.1 -i 1 -p 14310

To see if multicast traffic is coming in on the interface: tcpdump -i enp3s0 -s0 -vv net 224.0.28.0/24

Note, on Centos 7, the commands to change your interfaces have changed. Here is a helpful cheat-sheet for relevant commands: https://access.redhat.com/sites/default/files/attachments/rh_ip_command_cheatsheet_1214_jcs_print.pdf

Useful Links:

http://stevehardie.com/2013/05/cisco-877w-configure-wireless-and-wired-on-single-subnet/ https://supportforums.cisco.com/discussion/11801006/does-configuring-two-vlans-cisco-871-router-divides-bandwidth-internet-half https://supportforums.cisco.com/discussion/10567086/change-ip-router-cisco-871

Sample Router Configuration

Current configuration : 4011 bytes
!
version 12.4
no service pad
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
!
hostname Router
!
boot-start-marker
boot-end-marker
!
!
no aaa new-model
!
resource policy
!
ip cef
!
!
no ip dhcp use vrf connected
ip dhcp excluded-address 192.168.1.32 192.168.1.255
!
ip dhcp pool vlan20pool
network XXX.XXX.XXX.0 255.255.255.0
default-router XXX.XXX.XXX.1
dns-server 8.8.8.8
!
ip dhcp pool vlan10pool
network 192.168.1.0 255.255.255.0
default-router 192.168.1.1
dns-server 8.8.8.8
!
!
ip multicast-routing
!
!
!
!
!
!
crypto isakmp policy 2
encr 3des
hash md5
authentication pre-share
crypto isakmp key SECRET_KEY address XXX.XXX.XXX.XXX
!
!
crypto ipsec transform-set cmevpn esp-3des esp-md5-hmac
!
crypto map cmevpn 1 ipsec-isakmp
set peer XXX.XXX.XXX.XXX
set transform-set cmevpn
match address 100
!
bridge irb
!
!
!
interface Tunnel0
ip address XXX.XXX.XXX.XXX 255.255.255.252
ip pim sparse-mode
tunnel source XXX.XXX.XXX.XXX
tunnel destination XXX.XXX.XXX.1
!
interface Loopback0
ip address XXX.XXX.XXX.XXX 255.255.255.255
!
interface FastEthernet0
switchport access vlan 10
!
interface FastEthernet1
switchport access vlan 10
!
interface FastEthernet2
switchport access vlan 20
!
interface FastEthernet3
switchport access vlan 20
!
interface FastEthernet4
ip address dhcp
ip nat outside
ip virtual-reassembly
duplex auto
speed auto
crypto map cmevpn
!
interface Dot11Radio0
no ip address
!
broadcast-key vlan 10 change 45
!
!
encryption vlan 10 mode ciphers tkip
!
ssid YOUR_SSID_NETWORK_NAME
vlan 10
authentication open
authentication key-management wpa
guest-mode
wpa-psk ascii 0 YOUR_SSID_PASSWORD
!
speed basic-1.0 basic-2.0 basic-5.5 6.0 9.0 basic-11.0 12.0 18.0 24.0 36.0 48.0 54.0
station-role root
no cdp enable
!
interface Dot11Radio0.10
encapsulation dot1Q 10 native
no cdp enable
bridge-group 1
bridge-group 1 subscriber-loop-control
bridge-group 1 spanning-disabled
bridge-group 1 block-unknown-source
no bridge-group 1 source-learning
no bridge-group 1 unicast-flooding
!
interface Vlan1
no ip address
ip access-group 102 in
shutdown
!
interface Vlan10
no ip address
ip access-group 102 in
ip nat inside
ip virtual-reassembly
bridge-group 1
bridge-group 1 spanning-disabled
!
interface Vlan20
ip address XXX.XXX.XXX.1 255.255.255.0
ip pim sparse-mode
ip nat inside
ip virtual-reassembly
!
interface BVI1
ip address 192.168.1.1 255.255.255.0
ip nat inside
ip virtual-reassembly
!
ip route XXX.XXX.XXX.1 255.255.255.255 FastEthernet4
ip route XXX.XXX.XXX.0 255.255.255.128 Tunnel0
ip route XXX.XXX.XXX.128 255.255.255.128 FastEthernet4
ip route XXX.XXX.XXX.255 255.255.255.255 Tunnel0
!
!
no ip http server
no ip http secure-server
ip pim rp-address XXX.XXX.XXX.254
ip mroute XXX.XXX.XXX.0 255.255.255.128 Tunnel0
ip nat inside source list 1 interface FastEthernet4 overload
!
access-list 1 permit 192.168.1.0 0.0.0.255
access-list 102 permit ip any any
!
!
!
!
control-plane
!
bridge 1 protocol ieee
bridge 1 route ip
!
line con 0
no modem enable
line aux 0
line vty 0 4
login
!
scheduler max-task-time 5000
end

RP Filtering

Fri, 02 Sep 2016 00:00:00 GMT

Why is my server not receiving multicast traffic?

Out of the box linux installations oftentimes enable so-called "reverse path" filtering. This essentially means that if the source ip of the udp packet is not routable through the interface in which the packet arrived, then it will be dropped.

To turn off (disable) this filtering, and allow multicast traffic through, set the rp_filter to 0:

echo "0" > /proc/sys/net/ipv4/conf/all/rp_filter

Lastly, edit /etc/sysctl.conf and change: ipv4.conf.all.rp_filter = 0

More information available: http://www.slashroot.in/linux-kernel-rpfilter-settings-reverse-path-filtering and http://centoshowtos.org/network-and-security/rp_filter/