What Shopify has learned from five years of bug bounty programs

As a part-time hacker and full-time security engineer at Shopify, I’ve learned a lot along the way. One of the biggest takeaways I recognized early on was that I kept returning to programs run by security teams that respected me and my time, were responsive to my reports and inquiries, and were transparent in their communications and disclosures.

When I first joined Shopify, we were challenged to scale our team alongside our relatively new bug bounty program. I was excited to bring my insights and improve upon a program that hackers would engage with. Our goal has always been to build upon the success of our hacker-powered security programs with a concerted effort to promote transparency and attract talent. With the extra sets of eyes, we are able to implement more checks and balances to harden our attack surfaces.

We attribute much of our success to our work as an outspoken proponent of working with the hacker community. But hacking is a two-way street, and we work hard to keep up our end of the deal while holding hackers accountable to higher standards. We also put our money where our security is, because our bounty awards are higher than most.

We offer high minimum bounties so hackers know it’s worth their time, even when they find less impactful bugs. Our current minimum is $500. We see that as an investment in attracting talented hackers to our program.

Now, five years into our bug bounty journey on HackerOne — which surpassed $1 million in bounties last year, the fifth public bug bounty program to do so — we’re taking a look at how this program reinforced our belief that transparency is good for everyone.

Transparency helps security

Transparency is the heart of our security program. Disclosures are critical to helping hackers improve their skills as well as improving security around the world. I know it works, because that’s how I sharpened my own skills.

Disclosures create a knowledge repository for others to learn from and use in their own work. It then allows them to level up quickly and bring value to the programs they participate in.

Giving hackers more insight into bugs and the eventual fixes means more trained eyes spotting future vulnerabilities, whether it’s Shopify or another company’s code. By disclosing security gaps, we are helping a hacker or another company find similar gaps somewhere else.

Disclosures also give hackers a quick avenue for testing our fixes to the original bug. A disclosure is a clear signal for hackers to test that functionality. Hackers can see the fix, then look for ways in which it might be bypassed. It can also be used for other companies to read our disclosures in order to check their own systems for bugs. Hackers, and even the other companies, wouldn’t have known about some bugs had we not disclosed them.

Of course, disclosures have a special place in my heart. Prior to joining Shopify, I was an active hacker on other HackerOne programs, but the volume of Shopify disclosures became a reliable source of education and inspiration. Having had no prior security experience, I used those reports to learn how to find security bugs and communicate them to security teams.

A program designed for hackers

Our bounty program has resolved more than 1,150 reports over the past five years, and we’ve thanked more than 400 hackers for their efforts. But those numbers weren’t reached through luck; we’ve focused on making our program attractive to the talent on the platform.  Because we have a lot of hackers on our team, we’re able to shape how our program should ideally operate.

We work hard to explain why a reported bug is or isn’t an issue so everyone understands what we deem to be important. We welcome hackers asking questions about our decisions and see these conversations as opportunities for hackers to improve their skill, better understand impact and set future expectations. By doing so, we’ve seen hackers go from consistently reporting invalid to reporting valid bugs.

Additionally, “security by obscurity” seems contrary to the whole purpose of trust. We want our merchants to know we’re doing everything we can to protect them and their data. We use it to build more trust, not diminish it. We also know that no product is perfect, and when things do fall through the cracks, we’re grateful to those who help us find them.

Nurturing true relationships

When I first started hacking, I would return to programs like Shopify’s that respected me and my effort. When I joined Shopify, I made sure that ethos went into our outward-facing program. Money is attractive, but so is responsiveness, relationships, clear guidance, and constant communication. Nothing is worse than sending in a bug report and then hearing nothing for weeks after. We treat hackers like we treat our colleagues.

We’ve been lucky enough to build real relationships with a diverse community of hackers and are inspired by their dedication to our program. A few standouts are @h13-, @cache-money, @0xacb, @zombiehelp54, @bored-engineer, and @ngalog, each of whom have contributed many significant bug reports over the life of Shopify’s program. For all of these hackers and many others, it’s a big win for us when they take the time to poke at Shopify’s code. We wouldn’t be where we are today without their dedication and collaboration.

Five years strong

Over the past five years, we’ve learned that you have to view hackers as a resource to cherish. Transparent, hacker-powered security should be a key cog in your overall program.

From our internal teams, to the hacker community, and the rest of the technology industry, transparency is what gives our bounty programs real power. We believe that the more everyone shares, the safer everyone will be. Ultimately, we would love to see disclosures standardized across the community. It’s a win for us, freelance hackers, and others who might have similar but unknown vulnerabilities lurking within their code.

Pete Yaworski is a senior application security engineer at Shopify.