SCORE Bot: Shift Left, at Scale!

Objective: Reduce the number of vulnerabilities in our products over time, by building repeatable/sustainable proactive security practices embedded within our Product Life-Cycle.

In the post, I'm going to cover some of the highlights of Vidhu Jayabalan and Laksh Raghavan's great AppSec USA 2018 presentation: "SCORE Bot: Shift Left, at Scale!" SCORE Bot is PayPal’s light-weight, continuous code scanning tool that hooks into their CI/CD pipeline.

This post is will just include the key takeaways. If you want the full scoop, you can check out my full summary here.

If you like this sort of content, I've started a newsletter where I write about AppSec and scaling security, DevSecOps, automated bug finding (static and dynamic analysis, fuzzing, etc.), summarize security talks and papers I like, and share useful security links I come across. You can check out the first issue here and subscribe here.

Key Takeaways

SCORE Bot Architecture: SCORE Bot works by receiving async webhooks when new pull requests (PRs) are created and then scans the diff for PayPal-specific security issues and best practice violations. If found, SCORE-Bot comments on the PR, emails the developer, and stores metrics.

• SCORE Bot’s comment includes what the identified issue is, its code location, a summary of the issue in the context of the code, and a link with step by step details about how to fix it.
• SCORE Bot currently has 25 rules covering PayPal’s custom frameworks, libraries, APIs, logging infrastructure, etc.
• Developers can file an exemption to still merge a PR when required, for example, to fix a P1 (critical) issue.

SCORE Bot’s approach is valuable because it’s fast (developers receive feedback when they’re still in the mindset of the current PR) and automatic (SCORE Bot runs automatically within developer’s normal workflow).

PayPal chose the SCORE Bot approach over SAST tools and IDE plug-ins because SAST tools are too heavy-weight and complex to customize and maintaining IDE plug-ins for the variety of editors PayPal engineers use is not feasible.

SCORE Bot’s benefits include giving the AppSec team end-to-end visibility into PayPal-specific vulnerabilities across all repos and tech stacks, shows vulnerability patterns, enabling security to offer focused training, and helps creates a security culture by scanning every commit, keeping security top of mind.

Unique insights from this talk include:

• A/B test security - small wording or presentation changes can cause a significant difference in developer behavior.
• Maximize security iteration speed - build security tools and processes with an eye towards speeding up the build -> get feedback -> iterate loop.

Stay in Touch!

If you have any feedback, questions, or comments about this post, please reach out! I’d love to chat.

Be notified about all of my future summaries by subscribing to the newsletter or following me on Twitter: @clintgibler.

Have a great rest of your day!