Index or Lookup Emerging Threats or AlienVault OTX

Currently I have my Emerging Threats being run through Suricata, and the OTX being run through Zeek on my SecurityOnion which then feeds to my Splunk instance.  I seem to have already broken my Kibana (will have to research that next - side panels run, but its all white space where the actual data should be). 

After reading a few articles: 
Malware Analysis
IDS alert challenge
Splunk Enterprise Security
IDS rules for PulledPork

Now I know that SecurityOnion uses pulledpork currently for its rule management, but I believe in future versions it will be moving to Suricata-update (well at least for Suricata).  But that does not matter currently. 

Emerging Threats updates a file called downloaded.rules, and OTX created a file called otx.dat, these both hold the rules which can be either parsed into an index or TSV/CSV out into a lookup table. 

Currently, I am trying to decide which way would be the most beneficial to most me, and other users of my system. 

I also think there is at least another file that goes with the Snort VRT which goes into further details based on the SID and is called the opensource.gz  For Emerging Threats it is called SID-Descriptions-ETOpen.json.gz which I think provides the same information for that rule set.

I am leaning towards putting the rules into an index, but I could be swayed either way, or to not even try this, but I think this has potential for analysts who try and hunt, to be able to look back at what signatures are hit on, and not have to Google every one of them.   

SecurityOnion with Splunk

Recently I installed SecurityOnion 16.04, and wanted to play with Splunk on top of SecurityOnion (I know it basically has Elastic with Kibana, but I just wanted to use a different tool at the same time. 

Setting up Splunk was pretty straight forward, installed it on a second VM running Ubuntu.  I then added the Splunk forwarder to SecurityOnion.  (7.2.5.1 for both).   Another pretty easy install.

Next, I downloaded the TA's (Technology Add-ons) from SplunkBase for Bro and Suricata.  Bro setup was easy, just setup Splunk to monitor the bro/current logs, and I was receiving data on Splunk.  Suricata was a little different, as I was only getting the Suricata stats. 

SecurityOnion uses separate log files, but I from what I understand (correct me if I am wrong) Splunk gets more details in the event data was something like the eve json that is possible with Suricata.  So with that, I decided to start up the eve json file (called it suricata.json).

I started to get the majority of my Suricata feeds into Splunk, currently only missing the actual alerts being fired.  I will need to go back and look at what is missing in either the monitoring or in the eve json file (suricata.yaml)

It is nice to be able to see the same data in two different SIEM type products, to have the ability to compare search queries and see different visualizations. 

Intel Threats ingesting into SecurityOnion

Earlier today I re-created my SecurityOnion VM, and turned on the port mirroring on my TP-LINK switch.  One thing I still need to figure out is if I need to mirror both ingress/egress on separate ports or is it a 1-for-1. 

I signed up for IBM X-Force Intel feeds, AlienVault OTX Threat Feeds, and Critical Threat feeds.  Currently, I only set up on the AlienVault OTX threats as I am not sure if I can have multiple threats loaded.  I think I can just have not had time to read through everything. 

I used the guide found at SecurityOnion OTX Intel Threat setup with Bro(Zeek). 

Overall the instructions provided by the SecurityOnion team were easy to follow and ran into no errors.  One thing I would have like but understand from a security point of view was the input of the API key.  It a blank field and no cursor movement to see if you have put in a specific number of characters.  I used cut/paste on my second attempt (first one I tried to hand copy from my main machine to the VM) 

Now to sit back and watch Bro and Suricata report back what's going on my network.  Next project will be adding a Splunk forwarder.  I know there is Kibana, but I would like to have the option for both. 

Can anyone think of any other type of files/feeds I should be ingesting in Bro/Suricata?   Or another application to run on SecurityOnion? 

SecurityOnion 16.04 up and running

I am still waiting on purchasing a new to me system for my security onion setup.  In the meantime I have added an additional NIC to my main computer and set up a SecurityOnion VM. 

The plan was to use the additional NIC to mirror the main ethernet cable from my router (EdgeRouter Lite) to the switch (TP LINK SG2008)

In the TP-LINK I setup port 1 Ingress/Egress to be mirrored to port 8 and ran a new ethernet cable from port 8 to the new NIC on the PC.    Next, I installed the SecurityOnion VM, and make all the necessary updates to the system.

Using the two NICs, one setup as management, and the second in promiscuous mode.   After this, I opened up Kibana to see the traffic.  On a side note, I have not used ELK stack often more used to Splunk but was able to get a general first impression.

One thing I noticed was my internet speeds also were drastically slower, but I think this was just timing and not directly related to the port mirroring.   I undid all my changes to my NIC card just in case, and tested speeds from my router (4mbs)  when I should be getting 150mbs per my internet provider. 

Later that evening, my speeds returned with no real changes by me.  So I will go back and re-setup the switch, and continue to ingest data.

Future plans:
Ingest IBM X-Force Intel Feeds
Ingest AlienVault OTX Intel Feeds
Setup Splunk forwarder
Install Splunk (new VM)
MITRE ATTA&K Framework