CI/CD Security in Automotive Software: More Questions than Answers

There are many challenges in automotive software development. Not only with the need to deliver fast, while making sure that the software actually works as intended inside the car. Once it’s deployed, even with over-the-air updates, it’s difficult to patch correctly. And as with any element shipped with digital software, cybersecurity needs to be more than an afterthought.

I won’t discuss automotive attack scenarios because most are frightening. If a virus, worm, or direct attacker gains high-level access within the software, the possibilities are unlimited.

Let’s analyze the pain points of automotive security.

Compliance vs velocity

Or, how it looks: guardrails against getting stuff done. Dev teams and management usually want to ship fast, and while one side wants to speed up the process, the other wants to cram in as many features as possible.

The common view is that security and compliance are seen as roadblocks in this process.

From my experience in both development and security, one thing is clear: anything that delays the development and deployment process is generally frowned upon. And even the smallest ‘surgical’ integrations are affecting the speed of either development or deployment.

Regulations

Regulations are always hard: hard to convince people that they are needed, hard to decipher what’s written in the official regulatory documents, hard to translate them in a development lifecycle, and lastly hard to ensure the regulation is actually accomplished after the initial setup.

And, it’s usually not just one regulation to be concerned with. With European markets, software that goes into the infotainment of the car, for example, needs to comply with:

General regulations frameworks:

  • ISO 26262
  • ASPICe

Specific standards

  • UNECE WP.29 R155 (Proves OEMs and suppliers manage cybersecurity across the lifecycle)
  • UNECE WP.29 R156 (Mandates secure processes for OTA(Over The air) updates, rollback, and documentation).

And let’s not forget to mention other adjacent frameworks that usually apply in manufacturing, worth mentioning at least Cyber Resilience Act.

So it becomes a time constrain game of how we manage compliance once, and see what’s left and what overlaps. On a plus side, regulations actually are useful, because it usually contains a lot of lessons learned after attacks, or best practices to avoid similar ones.

Policy-as-code sounds good on paper

Understanding compliance and translating it into code can be tricky because usually compliance is not specific enough and sometimes downright too high level and generic.

However, this is sometimes a plus because, being generic and mentioning “secure by default”, you actually can take the necessary steps to ensure that security is built in layers, and at the start of the project, and possibly surrounding every development process. Yet secure by default can mean doubling the entire development effort if you somehow miss this mark and leave the security as a last step.

So in theory, regulations and standards like ISO 26262, ASPICE, and UNECE WP.29 R155 & R156 can be translated into pipeline logic. And let’s leave it at that for now.

PS: ISO and ASPICE define best practices, while UNECE R155/R156 and the EU’s Cyber Resilience Act (CRA) are binding regulations tied to market access.

The actual complexity

Automotive development is a hybrid world, at left side, we have archaic, hardware-tied software, at the right side, we get containers, cloud, and AI:

  • Legacy embedded systems: best-case scenario, we got C, AUTOSAR, usually hardware-tied
  • Modern workloads: cloud-native, AI/ML models

So we’re dealing with software from a time when the lead programmer is gone or at least enjoying an old-age pension, the programming language is from the Cold War era, and this same software needs to connect with modern cloud systems, including the new trend: AI. And honestly, this is not a rant about old age, and even old programming languages, so evidently about integrating latest and greatest with oldest and greatest at their time.

This merge is sometimes downright impossible. Safety-critical builds will stay on-prem, not only for certification reasons, but due to hardware constraints as well, while simulations and AI workloads run best in the cloud. Good luck developer.

Cybersecurity as a pain point

Everyone loves invisible security as a concept, but hiding it is not that simple.

It’s not overly complicated, there’s a delicate balance between over-securing pipelines with numerous security gates and not implementing enough security measures.

Usually, stacking too many security gates results in slowdown, and security becomes taxing on the release cycles. Not enough security gates? We don’t need to rehash the details of how many security breaches can be prevented, and how many billions or trillions of dollars can be saved by using simple security measures.

Over the air updates: nightmare and blessing

Let’s face it: updating the car is fun. When it works, and you get a UI update, some bug fixes on various glitches and improvements, it’s really interesting.

The nightmare scenario? Especially inside the car, there are too many, and to name a few:

  • Supply-chain attacks and malicious infiltrations of the supply chain. Viruses and worms are a real danger, and they can cause a lot of problems. Specific attacks can change commands, and general ones can still hurt the car by using up resources.
  • Non-intended issues such as an untested fix that actually breaks the OS, or software that runs in infotainment.
  • Remote control or manipulation of critical vehicle functions
  • Sensor and camera feed manipulation
  • Battery management system compromise

Overall, how do we protect against these threats since fewer and fewer mechanical pieces are left in the car, and more and more they are replaced by software? This is the billion-dollar question, I’d argue, not the one that is asked today: how to make it fully autonomous.

Golden pipelines to the rescue

I understand that the focus should not be on adding more security tools. So on the topic of adding tools, the best-case scenario is to add pre-built, reusable pipelines that include security testing, and ideally some compliance checks by default. Hence the name, it provides a golden path for teams, or at least a silver path, or at the very basic least a starting point.

I’ll end with more questions than answers.
And the questions that remain are:

  • Who is making the golden pipelines, security teams or devops, or developers?,
  • How to actually make them reusable between car companies, or at least within the same company.

Photo by Dragon White Munthe on Unsplash.

1 comments

Comments are closed.