How Race Communications Leveraged Kentik to Stop Mirai Botnet Infection and Abuse

The Mirai botnet was first discovered back in 2016 but has continued to persist and abuse common vulnerabilities and exposures (CVEs) on IoT devices, including home routers and many other network-connected devices. In short, the network of bots was built by malicious actors who exploited remote access and control protocol ports over many different device types, producing damaging traffic levels and creating an advanced, powerful tool that can be used for large-scale DDoS attacks and many nefarious purposes.

When Kentik customer, Race Communications, a provider of reliable, high-speed internet and advanced communications to communities throughout California), learned of Mirai’s potential risk to its customers, the team knew it needed to act fast. In this blog post, we outline how Race Communications was able to leverage Kentik’s powerful network analytics to identify malicious traffic associated with Mirai, determine which of Race Communications’ customer IP addresses were being used by the botnet, and ultimately, save its online IP reputation.

The Race to Stop Mirai
Race Communications was alerted to the potential Mirai risk when the team received a letter from another network online noting that IP address(es) that Race Communications owned were acting maliciously over the internet. These addresses were apparently port scanning IP addresses that belonged to the company sending the letter. The letter took on the form of a formal complaint and asked that Race Communications cease this activity.

While the port scans were only coming from a few hosts, Race Communications noticed that the complaint was against an entire /24 IP block. This had the potential to lead to the entire /24 block getting blacklisted for malicious activity. This type of blacklisting could potentially cause other Race Communications customers to experience a loss of connectivity to services, due to poor IP reputation.

Race Communications knew that by utilizing network forensics capabilities from Kentik, the team would be able to quickly drill down into the incident and determine the root cause behind the formal complaint.

Network Forensics & Visibility from Kentik
The Race Communications network team turned to Kentik’s “Unique Destination Port” metric and was quickly able to see how many ports the address listed in the formal complaint might be hitting, and why it would be considered a scan. When this number revealed only 40 destination ports on average, the team again turned to the insights available with the Kentik platform.

Checking the pattern of all ports that were being utilized, Race Communications discovered that the vast majority of all destination ports were either port 23 (Telnet), or port 37215 (Huawei Remote Procedure Port). The Huawei port was of immediate interest to the network team as this port is part of a long-lived exploit cataloged as CVE-2017-17215. The exploit only requires a single authentication to work, and once it had been exploited, remote code execution is possible on the associated device. (Vuldb has additional information on the life of this exploit.)

The final port that Race Communications observed was port 2323, which was a tiny fraction of the total traffic, and always sourced from port 23 on protocol TCP. This is a potential sign of a Mirai variant botnet C&C traffic, as described in this article, and is additionally associated with CVE-2016-10401; an exploit associated with ZyXEL network devices and escalation of permissions.    The method of exploiting both the Huawei CVE and the ZyXEL are very similar, requiring one authentication first.

In just a three-hour period, Race Communications was able to see a very large Unique Destination IP count-per-port. As seen in the image below, this rate was highly consistent.

Race Communications was also able to leverage the Destination Port information from Kentik to pinpoint backtrace several additional Source IPs on its customers’ networks that were participating in similar traffic patterns.

During this portion of the investigation, while working through the traffic, Kentik and the Race Communications network team found additional interesting and actionable information within the Kentik platform:

  • Most destination 37215 ports were hitting IP addresses in the Asian market regions, such as China, Japan, and South Korea. To the team, this was a potential indicator that the attackers were targeting addresses within a region more likely to contain a Huawei product.
  • Destination port 23 traffic was hitting an evenly distributed traffic pattern between the Asian markets and the United States. To the Race Communications team, this had the potential to imply that the attacker(s) were searching for new devices using Telnet, aiming to find vulnerable Huawei devices in the process, and scripting to additionally hit these devices with a vendor-specific attack.
  • Each Source IP that was on its customers’ networks was reaching an hourly average of 3,500 Unique Destination IPs consistently, for weeks. Each Unique Destination is another potential report, and another hit against this Race Communications’s customer IP reputation.

An Interesting Forensic Finding
Perhaps the most interesting finding for Race Communications was that Kentik’s Spamhaus Botnet and Threat-List data feeds tagged all of the offending IPs that Race Communications had identified manually. Additionally, Kentik’s platform highlighted other suspicious activity among Race Communications’ customer subscriber IPs. This meant that there had been significantly more complaints against its customer IP blocks than Race Communications was initially aware of from the single, formal complaint the team received.

The Race Communications team knew it was not sufficient to wait for another entity online to formally complain. With insights from Kentik, the team could act proactively in order to detect malicious subscriber traffic heading to the larger internet.

Automated Detection
Kentik was able to offer Race Communications with automation in detection, by creating Kentik DDoS-type policies with modifications. Kentik will now alert the provider whenever a subscriber IP communicating outbound via ports 23 and 32715 reaches more than 500 unique destinations.

These insights give Race Communications the option to proactive notify internal operational groups of events that could impact IP reputation in the long term. Additionally, Race Communications can now contact their customers to alert them of a possible infection on one of their Internet-connected devices.

In addition to Kentik’s platform scrubbing partners Radware and A10 Networks and our BGP-based Remote Trigger Black Hole (RTBH), Kentik now offers BGP FlowSpec mitigations. FlowSpec can be utilized to mitigate traffic with low-to-no collateral effects versus RTBH. It excels in situations where non-volumetric attacks like the ones associated with the Mirai botnet are compromising commercial security and reputation.

Summary
With Kentik, Race Communications can protect its business and its online IP reputation by unlocking:

  • Information needed to remediate not just the reported problem, but the entire detectable problem
  • A path by which to contact subscribers and explain specifically why they have been the subject of a complaint (e.g. if they have infected machines on their network, response and action can quickly take place)
  • An automated way of being informed of new IP addresses that form the Mirai pattern or other botnet traffic patterns
  • Faster mean time to diagnose (MTTD)
  • Faster mean time to repair (MTTR)

Kentik is able to offer insights that can help identify almost any malicious or unwanted traffic on any network and provides automatic notification and mitigation capabilities. For more information on network visibility from Kentik, reach out to us or request a demo.

Cybersecurity: How to keep your children safe!

The Internet is a great resource for you and your family. At the touch of a button, you have access to a world of knowledge and entertainment. Sadly, the internet is also a dangerous place to hang out – particularly for children. Nearly 60% of teens have received an email or instant message from a stranger – half of them have replied. With summer approaching, the potential for children to wind up on dangerous sites increases since kids have more free time and that usually means more screen time.

So what can you do? Just to get started, let’s list some things you can do almost immediately to help keep your kids safe while they’re online.

It won’t take a lot of time to try these suggestions, and while we’ll talk later on about setting up parental controls through your Race router, the following steps can give you some peace of mind until you can do so.

    1. Place computers in a common area of the house:
      Don’t allow kids to have a computer in their room. You’d be surprised by how much the mere presence of a parent who may or may not be looking over a child’s shoulder while they use the computer can keep a child in line. They have no way of knowing if your eyes are good enough to see across the room, now do they? Make sure the computer’s screen is visible from other parts of the room and isn’t turned toward a wall.
    2. Set reasonable time and usage limits:
      Set rules about what your child can and can’t do when on the internet. Set time limits on their computer use. If they say they’re researching homework, maybe you don’t include that in the time limits – but make sure they’re using it for homework.
    3. Discuss the dangers of the web with your child:
      Sit down and discuss the dangers of the internet. Talk openly and honestly about what’s out there and the kind of stuff they want to avoid. Forewarned is forearmed.
    4. Teach them to protect their privacy
      While they won’t fully understand the consequences of revealing personal information online, you should still make sure your children know:
      * Never to give their name, phone number, e-mail address, password, postal address, school, or picture without your permission
      * Not to open e-mail from people they don’t know
      * Not to respond to hurtful or disturbing messages
      * Not to get together with anyone they “meet” online.
    5. Keep the youngsters out of online chat rooms, and do your best to reinforce the old rule, “never talk to strangers.”:
      Chat rooms are a popular place for sex offenders to meet their prey. If possible, keep your kids out of chat rooms altogether. Make sure your child knows that no matter how nice an online “friend” may seem to be, they are still a stranger, and may not be who they appear to be. ​
    6. Know Passwords:
      Be upfront with your children that you will need to have their passwords for all of their devices and for all of their social media sites. Once you have the passwords, check these sites regularly to see what your child is seeing and posting.
    7. NEVER let your child upload or download photos without your permission:
      Online predators will often send photos supposedly of themselves or request photos of the child.

Turn your ISP into your ally
Before buying any safety product, experts recommend that you work with what you’ve got, starting with your Internet service provider – hopefully that is us at Race Communications!

Your Gigafy Me router includes free parental controls that can limit children’s access to websites and communication features (e-mail, instant messaging, chat) by the time of day and other variables. If you don’t have a router rental through Race, give us a call to have that added or if you have any questions about these features.

Race Communications wins BroadbandNow Awards!

BroadbandNow recently announced their 2019 Service Provider Awards Program. This program is the culmination of a massive, cross-disciplinary effort to highlight excellence among internet service providers across the United States.

As the only consumer resource investing in data aggregation and validation for independent, regional ISPs, the website is rolling out this awards program in order to recognize providers of all sizes that offer excellent service to consumers across the nation.

We are thrilled to announce that Race was ranked in two separate categories from BroadbandNow and our ‘Gigafy Me’ package was recognized as one of the Top 10 Most Affordable Broadband Plans Nationwide AND as one of the Top 10 Most Affordable Gig Internet Plans Nationwide!

We pride ourselves on providing fast, reliable internet at an affordable cost, and we are so proud of the team behind our network. We want to thank them for all their hard work. We also want to recognize our customers for choosing us, a local provider for their home entertainment needs.

We want to encourage our customers to leave us a review with Broadbandnow (be sure to click on the customer review tab). With your help, Race could be recognized in additional categories!

Race Communications live in Atherton.

Hundreds of residents in Atherton, CA now have access to fiber gigabit internet with speeds up to 1,000 Mbps due to an initiative driven by OpenFiber. On Monday, March 4th, 2019, Race Communications installed the first two eligible homes in Atherton, allowing homeowners to experience what being gigafied really means. One homeowner described the service as “out of this world fast”.

OpenFiber builds and operates an open infrastructure to enable multiple carriers and service providers to hop on the network giving its consumers a variety of choices. The network in Atherton serves schools, public buildings and every home in Atherton.

download_20190307_143318
The installer leaving our first Atherton installation.

“We believe the future is here and OpenFiber provides unprecedented capacity, speed and latency to drive revolutionary consumer experience from smart homes, virtual reality to next-generation HD entertainment and more.” Phillip Clark CEO at OpenFiber. “We look forward to a continued partnership with Race and other providers to provide connectivity that helps meet residents’ personal and business needs.”

Race is delivering Fiber to the Home (FTTH) technology in Atherton as part of its participation in the OpenFiber network, and its commitment to meet the broadband needs of residents and local businesses in California. Race Communications will also provide phone and television services.

“From telecommuting to remote education, broadband access is essential to our everyday life,” said Raul Alcaraz, CEO of Race. “Our team at Race has been blessed to have a successful track record of public and private projects, and we feel confident we can deliver a reliable, fast and affordable broadband system.”

For more information or to order gigabit service, residents can contact Race Communications’ sales team, at (877) 722-3833 or at team@race.com

In case you missed our town hall…

Here is a recap of our recent town hall meeting at the Rood Administrative Center:

20190131_012053265_ios

We know many of the residents in our service area lead busy lives and were not able to attend our town hall meeting on Wednesday, January 30. Don’t worry – if you missed the meeting, you can catch it on Facebook through the Nevada County page or on Youtube through the Nevada County Media page once the video is uploaded.

20190131_022506581_ios
There was a great turn out for the town hall meeting on 1/30/2019

Lead by Race Communications’ CEO, Raul Alcaraz, the meeting provided a lot of helpful information for the community. Alcaraz covered the company’s history, products and pricing as well as the project’s long history and bright future! Alongside Alcaraz, were several of the Race executive team.

20190131_013813362_ios
From left to right: Raul Alcaraz – CEO, Domenic Pacini – VP of OSP & Engineering, Carlos Alcantar – CTO, Jim Miller – VP of Sales & Marketing.

While construction is still a few months away, Alcaraz assured residents that the project would be completed within the timeframe set by the CASF grant. Fiber to the home networks are changing the internet landscape and new fiber networks have quickly become the gold standard for cities across America, and Race is going to ensure that Western Nevada County does not get left behind.

To date, Race has been awarded eleven separate grants through the CASF program to advance broadband adoption and infrastructure deployment in areas such as Mojave, Tehachapi, Phelan and Bridgeport. Of these CASF projects, 9 have been completed and our tenth project is well underway. Bright Fiber will be our 11th CASF project. We have seen a huge benefit from our fiber networks in all of our CASF projects.

It has been a long journey to get to where we are – and there is still work to be done. However, we believe we have the expertise needed to get the project completed within 18 months, if not sooner.

Over 500 residents have paid a $119 deposit for gigabit service, and some deposit payers were wondering about the status of their money. Race will honor everyone’s deposit by applying it as a credit to their first bill. In addition, Race will also be giving deposit payers one free month of data service. In order to apply the credit, Race will create client IDs and provide account and login information to all deposit payers.

We understand that some people may move or simply not be interested in receiving service anymore. For those deposit payers, Race will provide a refund within 30 days of receiving a request. The refund request form can be found on nc.race.com

If you have questions about availability or want to see if your home or business will be in the planned fiber footprint, please check our map on nc.race.com or visit the map directly.