Why Tezos was off on Rarible for so long, and why that won't happen again

An explanation—and sincere apology—regarding the Tezos outage on Rarible that lasted nearly three weeks.

Why Tezos was off on Rarible for so long, and why that won't happen again

Gm, Tezos community!

Tezos is finally back on Rarible—and we feel like we owe you an explanation. But before we get to that, we have a more positive announcement, too: To make the outage up to you, trading Tezos NFTs will cost 0% fees for the next 2 weeks until Friday, August 19th.

Now to help you understand what went wrong, we want to share how Rarible integrated Tezos in the first place.

When we saw your vibrant community—all you creative artists and passionate collectors—we couldn’t not have you on Rarible.

But we had neither Tezos experience nor enough resources to do all the work ourselves. Given that we wanted to integrate the art chain ASAP, we asked for help from the Tezos community to support us on that front, and several companies generously gave us a hand.

They did great work, but we just outsourced too much of it. As a result, the infrastructure functioned well, but eventually created complexities that broke down when the protocol was upgraded.

The rest of this article gets a bit technical. That’s because we want to make sure we’re 100% transparent with you, sharing what exactly went wrong and why an incident like that won’t occur again.

How we integrated Tezos

The legacy infrastructure was built and developed by one of the Tezos ecosystem technical partners. They built the indexer and API for us quickly and specifically for Rarible.

Since we needed to move fast, it was developed using the OCAML language (this was the one our contractor was most comfortable with). Unfortunately, we didn’t know this language, so we couldn’t put our own people on it.

This was a problem, but we wanted to fulfill our promise and integrate Tezos by the time we promised to you. In the meantime, another company was developing the initial smart contracts.

This major time constraint forced us to accelerate development. We simply didn’t have enough time and resources to have everything 100% ready in time.

This is why the launch felt less than ideal to many in the community—including ourselves.

We didn’t provide the awesome experience we initially wanted to build. As a result, you saw a lackluster launch with limited features and many bugs. Nobody was truly happy with the situation.

But we wanted to do better than that—and ultimately, we wanted you to love Tezos on Rarible.

That’s why we then spent over 3 months fixing bugs and doing improvements that would help you buy and sell any Tezos NFTs on Rarible quickly, easily, and seamlessly.

That was a massive challenge because we index every NFT on-chain. That means we want any NFT minted on the Tezos blockchain to automatically be available on Rarible.com. Other marketplaces only list their own collections or add selected collections manually.

This makes Rarible.com smoother in the long term and ensures we offer you an open NFT marketplace with great liquidity. But in the short term, it made it much harder to build things—and because of our time and resource constraints, we couldn’t build new infrastructure from scratch quickly.

After 3 months of repairing things, we wanted to do more than fix problems and build the best possible Tezos NFT marketplace. This is why we started creating our own infrastructure and wanted to rely on Tezos ecosystem tooling. As you might know, Tezos has one of the best tool sets amongst all blockchains (and if you didn’t it’s yet another reason to love Tezos!). We built our own set of smart contracts using Edukera’s Archetype language, and our indexer based on Baking Bad’s Tzkt and Dip Dup framework, which is currently the ecosystem’s state of the art.

It then took us another 3 months to finalize the indexer with the help of the Baking Bad team. They provided us tremendous help and guidance. The migration to our new smart contracts and infrastructure was supposed to happen in early July. Our initial plan was to rely on a hybrid solution with both legacy and new infrastructure.

It would look like this :

As the intended migration approached, we didn’t have the resources to keep supporting the legacy API. That’s why we wanted to speed up the migration. We still needed to rely on the legacy infrastructure, because it would have taken too much at the time to completely migrate to our new infrastructure. But to give you a great experience, we wanted to migrate to this new infrastructure quickly so that we can deploy new features.

The reason you saw so few new features on Rarible Tezos is that we focused on laying the groundwork—building infrastructure we could maintain ourselves before adding new features.

Why Tezos on Rarible broke down

On June 29th, after the Tezos Jakarta upgrade, our legacy infrastructure stopped working. And because of its complexity, the problem was hard to solve and would’ve taken significant resources.

Because we wanted to create a smooth transition, our new infrastructure relied on the old system to import existing buy and sell orders. For us, completely removing the legacy infrastructure was a huge change that disrupted our initial plans. So we had to quickly come up with a new solution to make everything work.

It took us a day to define the design that would work without the legacy environment. And it took another several days to implement it.

Since it was quite a huge change, it required a lot of integration testing to make sure that it would not break any of the existing features. And as you can imagine, it definitely did. In addition to that, it all happened during a vacation period, combined with a new COVID wave, which drastically limited the number of resources.

Initially, we thought there wouldn’t be many integration issues. But every time we had fixed a problem and thought we were closer to a re-launch of Tezos, a new issue popped up.

The removal of the legacy infrastructure created a lot of breaking changes that we could not anticipate (since we had to find a new solution quickly), and that we could only detect one by one. In addition to that, we experienced many deployment issues that greatly reduced our ability to quickly resolve problems.

This is what led to the 2 week interruption and the hectic changes you also noticed.

This situation taught us a lot, and it gave us the opportunity to identify many things we had to fix and improve.

Once again, we would like to apologize for all the inconveniences caused by this interruption. As you can see, it was largely the result of resource-constrained work and unlucky timing. But there are good news, too:

What’s next?

Now that we've finally moved to our new infrastructure, we're able to release new features much quicker. For example, bids (including floor bids) and auctions will soon be launched. Cart purchases (like in web2 marketplaces) will also be live in the near future, along with other features!

We designed our new infrastructure in a way that allows us to implement new features easily and quickly, no matter how complex they are. It will also have much better performance and reliability.

For future Tezos upgrades, an outage like this will never happen again, as we are now relying on Tzkt indexer, and all the upgrade work is done on their side by the Baking Bad team, which is one of the best teams in the ecosystem.

As for the communication with Tezos Rarible gang, we know that we haven’t been such a good fren. We weren’t there for you when we needed us, but that has now changed and you can count on us moving forward.

We’ll be there for you guys. A lot of great updates will happen on the Tezos side of the marketplace, and we have never been so bullish.

Again, we sincerely apologize for how things went, and we hope you’re at least nearly as excited as we are about the future of Tezos on Rarible!

With love, Rarible