Open source changed the world

We all know Open source movement changed the industry in the past. Shining stars like Finnish origin Linus Torvalds we followed and heard for years. RedHat showed us how to combine business and open source. For the last years Microsoft has turned towards open source and long gone are the days when open source was a cancer.

Now Microsoft is the biggest single contributor of open source software. They have even entered to the core of open source community bby purchasing Github some time ago. I love open source and by default I share what I do. My sharing nowadays is more like this blog, but did used to write some code as well.

Before the web driven public APIs, code glimpses and applications had APIs. API was not invented during this millenia. The previous APIs were just limited in scope you can use them. They were local APIs in front of or actually part of some code library. I believe object oriented programming expanded the amount of local APIs. Everything was accessed and only with local API. Keep this concept in mind now.

Golden age of web APIs

Then the industry started to boom web accessible APIs. SOAP was the king for long until REST came along providing more lightweight but not standardised option to build and consume APIs. Roy Fielding’s famous dissertation “Architectural Styles and the Design of Network-based Software Architectures” published 2000 is still under heavy discussion. The amount of public APIs started to boom and the most commonly used proof of that is the charts provided by Programmableweb.com

Return of the libraries

Now I claim that we are closing the circle at least for platform-alike services and turning away from web driven APIs and instead leaning towards libraries…again. Most famous examples of library-driven “API companies” are for example Stripe and Twilio, Sendgrid (owned by Twilio).

The main difference is that the library refers to the code itself, while API refers to the interface. The code itself more often has APIs too. When originally APIs driven platform-alike services start to offer libraries via NPM or alike, they are most likely using the original underlying public APIs behind the scenes. Idea of library is just to make the consuming developers life even more simple.

In short, we have come from locally invoked APIs, we took a sail to web driven APIs, and now we are sailing back to the library harbour where APIs are again invoked locally. The difference is that the system providing the functions and/or data are still out there in the Internet.

The download counts alone for NPM tell a story for us. Just look at the numbers for Stripe and Twilio. It must be noted that not all downloads actually mean business or real use. Some of the dl’s are just noobs testing copy-paste code, some tryouts and such. Nevertheless, the huge amount of downloads most likely indicates some level of need and success.

Below is the charts to show how the download count for both has emerged.

Stripe library NPM dl stats 1.1.2016 - 27.10.2019

Twilio library NPM dl stats 1.1.2016 - 27.10.2019

I must stress that web driven APIs are probably the thing for smaller companies and services which precise surgical knifes to specific issues. API -driven service development is not going to go anywhere despite the rise of libraries. APIs are here to stay and keep on offering lego-like blocks to construct services fast in the future as well.

Is open source obsolete?

No. Developers love open source since it makes their life easier, they can build reputation with it, they can contribute to something they see valuable, it’s often also a learning experience and so on. The role of open source has though changed. The rise of API economy has lowered the need to have all functionality locally as open source (to avoid vendor locks to some extend). Now the golden area of open source is in the tooling.

Even the big closed source platforms push their CLI tools to the world as open source. They also embrace the community driven library and tool efforts (in open source). Open source has moved from the core to the next layer towards the consumers.

Are web APIs obsolete?

No, they offer more flexible use of the value proposition behind the libraries. Libraries are there to fulfill the most common needs and make the functions easier to use compared to raw API approach. Library can not fulfill all the needs and access to individual APIs offers means to build desired workflow or process. The success of libraries is explained partly because with them you don’t need to get familiar with low level APIs amount of which depending of the case might be 2,3,4,5…

Now if the individual APIs are designed well and are unified, it’s still effort worth it sometimes. But if the APIs are all unique and you as a developer need to learn them one by one and keep the differences in your head during flow state, it’s a no go. Too much effort and too cumbersome.

Simplicity is key element of great developer experience. Simplicity wins! Offer agility and flexibility with raw APIs.

The companies which have managed to build simple “6-8 lines of code” solutions for developers to overcome huge problems have paved the way for others to follow. They have set the bar for others and that is where new companies aim too. Latest example of this I’ve encountered is Impala. Techcrunch wrote a story of the company which wants to make it easier to interact with hotel data. The startup is building a layer on top of legacy hotel systems to standardize everything with a modern REST API.

In the story it is stated that “Impala wants to be as simple as Stripe, Twilio or Plaid. With a few lines of code, any developer should be able to get started with Impala before diving deeper.”

I guess they need to use GraphQL or go for libraries in the future.

SDKs (loosely used term here)

Boto3 is an example of SDK which contains both library type approach but also low level access to APIs as well “Boto provides an easy to use, object-oriented API, as well as low-level access to AWS services.”

import boto3
# Retrieve the list of existing buckets
s3 = boto3.client('s3')
response = s3.list_buckets()

# Output the bucket names
print('Existing buckets:')
for bucket in response['Buckets']:
    print(f'  {bucket["Name"]}')

The sentence refers to low-level access to AWS services and the examples show that it refers to local library APIs, not the web APIs.

Some more to read from 100 Days DX