MalTrail - Day 2

 So back on working on MalTrail for the second day.  I am able to access the web front end, but still no data is coming through.  

First thing I tried was to edit the interfaces in the maltrail.conf from any to ens34 (The current sniffing interface on Security Onion).  It might be better to use Bond0 as in 2.3 they use a bonded interface.  

Restarting the docker (Docker restart maltrail).    No new errors in the /var/log/maltrail/error.log, but also no traffic showing in the application. 

Next step:  docker exec -it maltrail bash to get into the Docker container.  I am going to kill both python programs and try and restart them. 

• pkill server.py
• pkill sensor.py
• python server.py

Well starting server.py crashes the python program as it cannot find ens34.  So now to the next question how do I bond a passive interface to a Docker container?

I decided to look at how the Suricata Docker container is being used.  

docker inspect so-suricata

Looks like it's an environment variable setting it to the bond0.   

Reading up on setting up a Docker container per-interface looks like you have to have an ip address associated with the interface (could be wrong). 

Next, I read up on --net=host, but that might also cause other issues as the ports are dropped possibly between the Docker container and the host? (That's how I read it)

I do know that Security Onion uses a Docker network, but like before trying to keep things on the same server but separate.  

MalTrail - SecurityOnion@Home (Try)

 I was reading up on MalTrail, and wanted to get that up and operational on my SecurityOnion VM (I know it probably can't handle it, but there is no fun in that).  At the moment I am not sure how cpu/ram intensive MalTrail is,  but I did see that it has been dockerized.  

One thing of note, is I am going into this knowing that it will be unsupported by the SecurityOnion team,  which is understandable.  

So with that, I am playing on my home setup of SecurityOnion, which I often test things with.  

Below is a screenshot of the Docker instructions.  

Seems easy enough, I made a slight change to the export location. and skipped the apt parts :), since I am using CentOS, and would like to limit the application installed.   

The for line I also skipped as the the sniffing port is already set to promisc

I could have added the Docker container into SecurityOnion elastic-net, but that could cause other issues down the road.  

The install went pretty easy, no errors that I could see, but I could not access the site.  

http://localhost:8338  

My thoughts were that I needed to open the firewall for this.    Reading up on SecurityOnion looks like I needed to create a Portgroup, and then add it to the INPUT for Standalone? 

*This is a standalone install of SecurityOnion

• so-firewall addportgroup maltrail
• so-firewall addport 8338
• salt system_standalone state.apply firewall

 Tried again to access http://localhost:8338  (No Success)

Next I tried  netstat -tulnp | grep LISTEN

Which looks like this:

This is the file I edited: /opt/so/saltstack/local/salt/firewall/assigned_hostgroups.local.map.yaml.

Reading more into this, I think I needed to add the ports to the Analyst Role, but I could be getting confused, cause I also see that its referencing the file: /opt/so/saltstack/local/pillar/minions/<HOSTNAME>_<ROLE>.sls

------------------

After this whole write-up, I figured it out.  I used the documentation from SecurityOnion and this section.  

Basically I added the maltrail port group to the Analyst role for the standalone machine through the standalone.sls file.   

---Now to figure out why its not actually reporting anything.  

SecurityOnion@Home - Alerts

 It's been a bit of time with my SecurityOnion server monitoring my network (out-of-band) and wanted to do an update on the type of alerts that I am seeing, and at least try to identify what some of them might be.

Below is a screen shot of some of the alerts.  I also noticed that I am having issues Pivoting to PCAP inside SecurityOnion right now, not sure if it is something I possibly did, or just something with the .130 update (I have not done PCAP since the upgrade)   

 1.  GPL WEB_SERVER:  This looks like my Lenovo Smart Hub is trying to talk to something else on my network (Which is currently not on since I use DHCP, might switch them all to static for better tracking).  But it looks like to trying to talk to these other devices about youtube.

2.  I pay for Spotify for the family, so not really going to delve to deep on this one.

3.  The next 2x are discord which I know is in use also

4.  .cloud - Family is mostly Apple, and a quick look they are all the apple devices.

5.  Microsoft Update - Validated

6.  ET-DNS as Non-Compliant DNS traffic -  Well this is my new cell phone, unsure what is doing.

7.  DNS Query TOR Hidden (.onion) - Also my cell phone.  

***Cell phone is Samsung 22 Ultra (Unlocked) on Tmobile.   I will have to do some more research on these two to see what the phone is doing.  

SecurityOnion@Home - Custom Alert Queries

 After the update to 2.3.130, I saw a discussion post about sorting and not staying on Alerts, which lead me down a rabbit hole about alerts.  I noticed that there were a few prebuilt alerts, but how do I add new ones?  

After a few minutes, I found a discussion post that referenced the alerts.queries.json file.  Did some searching, and like all the other files that are customizable for the soc they are under: 

/opt/so/saltstack/default/salt/soc/files/soc/

The three files currently are: alerts.queries.json, dashboard.queries.json, and hunt.queries.json.

An example was I added geodata to the alerts.queries.json

{ "name": "Group By Source IP/Port/Geo, Destination IP/Port/Geo, Name", "query": "* | groupby source.ip source.port source.geo.region_iso_code destination.ip destination.port destination.geo.region_iso_code rule.name" },

So I now have a new alert that can do checks on (This was just proof that it was available because at the time there was no reference in the documentation.  

I took the next step and forked the Securityonion docs, and made a change to add the Custom Queries, which the SecurityOnion team merged with the official 2.3 documentation.   After reviewing the update I saw that they added the dashboard/hunt queries reference  (At the time I did not know those were also available) 

SecurityOnion@Home - Mass Export PCAP?

 Another day, another task to try.  I have a requirement to store PCAP offline, and n 16.04 there were folders for PCAP /nsm/*/dailylogs which I could rsync over to another storage device.  In 2.3 there is a folder path for PCAP /NSM/PCAP but the files are not truly PCAP data files.  

The files are created from Stenographer and can be converted back to PCAP (that's my understanding at least).  That's how the SOC interfaces pull the PCAP based off of Alerts.   But what if you wanted to copy all PCAP data.  

Going through the SecurityOnion documentation and the discussion forums, I found mention of a  script created by the SecurityOnion team called so-pcap-export.   From the documentation:

I tested it with sudo so-pcap-export 'after 30m ago' output (leave off the .pcap as it is added in the script.

This created the file inside of the /nsm/pcapout folder.  So this works for exporting, but it looks to only create one file at the moment.  It would be nice to have the ability to create PCAP on set intervals, from the files created with Stenographer. 

More searching led me to this script on the discussion forums for SecurityOnion.  

https://github.com/Security-Onion-Solutions/securityonion/discussions/4038

Its a python script created by a Rob Hackworth, that I am still trying to get to work at the moment but it looks like it uses the same Stenoquery, just adds the ability to break up the PCAP by intervals.  So far when I have tried it and it does create multiple PCAPs based off intervals.  The only issue is that this version looks to only do by dates, and does not do by saying 24h, so I could copy days worth of PCAP. 

One item of note from the script: 

The time interval is critical as Steno will only let you export ~76G before it fails

Now I am not sure where that information comes from, maybe the users testing.    The next set of testing would be to see how CPU intensive the converting of the files to PCAP is on the system, and then the addition of a possible rsync to this.  

Currently trying to think of better ways, different ways to accomplish the task.    I was thinking maybe using an NFS share and having the script copy the files directly to the shared folder (This might be heavy intensive, especially in the current build-out of our SecurityOnion.  

I wonder if it's possible also someone set a marker on the last set of steno files that you converted back to PCAP so you are not recreating the same PCAP files.  

SecurityOnion@Home - Adding Warning Banner

 I did a little digging on the SecurityOnion discussion forms and found out they have added a way to include a warning banner on the Login Page of Security Onion 2.3.  It looks like it was an undocumented add with 2.3.30 (but was not stable as the unique path would get overwritten on updates. 

In 2.3.50 they have added banner.md and it can be updated the same as the Motd.md (Part of the SOC Customization).  The file itself is a Markdown file, and here is the  Markdown Guide.

   

To customize the Login Banner content, copy banner.md as follows and then edit /opt/so/saltstack/local/salt/soc/files/soc/banner.md 

sudo cp /opt/so/saltstack/default/salt/soc/files/soc/banner.md /opt/so/saltstack/local/salt/soc/files/soc/

SecurityOnion@Home - Suricata Include

 At work we started to use the include statement in our Suricata.yaml file while we were using SecurityOnion 16.04 and with the switch to SecurityOnion 2.3, I wanted to see how easy this would be to implement.  

First things first, we go from having a Suricata.yaml for each interface to having one bonded interface, so that makes updating simple.  With the bonded interface there is no more /etc/nsm/interface/suricata.yaml. I did find a suricata.yaml file under /opt/so/conf/suricata/ but it's created and managed with Salt, which means I probably cannot/should not edit that file directly. 

I am running a standalone version of SecurityOnion at my house so the steps might be a little different in a distributed environment.   

In the salt configuration first place I found a reference to the suricata yaml file was at:

/opt/so/saltstack/default/salt/suricata/default.yaml  which build out the suricata.yaml to /opt/so/conf/suricata.  

I decided to put the test_var.yaml @ /opt/so/conf/suricata/ 

Lets see what happens when I add the include file directly to the default.yaml file.  

1. After the update you run:  salt minionname_standalone state.highstate
2. Run:  so-status 

             3. Run: so-suricata-start

     4. Run: docker logs so-suricata

 It's looking for the file in the /etc/suricata/ folder (Which does not exist on-base OS, but it does exist on the Docker container for Suricata.  

We can validate this with docker inspect so-suricata (Looking for Binds)

As you can see that there is a /etc/suricata (The right side of the bind, which is the Docker container).  Looks like for the most part there are files that are bound between the Docker image and the core OS of SecurityOnion. 

I thought about putting the include file on the docker file @ /etc/suricata, but this does not sound like a good option for two reasons:

1. Upgrades would erase this
2. Probably more important, unsure if the included file inside the docker would work due to connecting the core OS.  

 From the last image it looks like I could potentially put the file in two directory locations:

1.  /nsm/suricata ---> This is where the eve.json files are, so probably not the best location
2.  /opt/so/conf/suricata/rules -->  which maps to /etc/suricata/rules on the docker container

Back to salt to make some updates to the sls files.   I am editing the /opt/so/saltstack/local/pillars/minions/name_standalone.sls

Note: Remember spacing (2) on yaml files

1. Edit the sls at the end:

            suricata:
              config:

                include:/etc/suricata/rules/test_var.yaml

1. Next I copy the test_var.yaml -> /opt/so/conf/suricata/rules
2. Then I run: salt minionname_standalone state.highstate
3. Check Docker containers with so-status (All good)
4. And validate with docker logs so-suricata
   
   
   

Now I have an include file as part of SecurityOnion, and my local teams can update that file which could cause Suricata to fail, but it's better than them trying to update the sls files under minions to add/remove variables in my opinion (which could break more functionality besides Suricata)