Upgrading testing capabilities on Postman
At Goibibo, deployment of new APIs and updates happen every day at 6:00 a.m. To increase efficiency, the team wanted to automate their daily integration testing flow, yet still ensure that every release is of high quality. This was difficult to achieve with their legacy testing solution.
Prior to adopting Postman, Goibibo was using an integration testing tool that had been built in-house. The tool functioned as a microservice within their broader platform; however, it was overly challenging to use and maintain. A few members on the hotel team (part of Goibibo’s larger engineering organization) explored Postman and found the platform suited their needs in multiple ways.
Moving to Postman would allow the team to centralize their test cases across various microservices that make up their platform. Using Newman, Postman could integrate easily with their existing CI/CD pipeline and make test runs fast and seamless. From a user standpoint, the platform was easy to use on a day-to-day basis and it was flexible enough to accommodate the team’s vision for structured collaboration. Finally, Postman enabled well-groomed documentation that could be automatically generated regardless of the coding language used, which saved developers time and allowed them to focus on more important tasks.
“Moving our APIs to Postman allowed us to centralise all our test cases across different microservices, as well as integrate with our CI/CD pipeline using Newman.
Inside the hotel team’s APIs and testing approach
The hotel team consists of 70 engineers working on Goibibo’s hotel booking system, including front-end and back-end developers, QA engineers, and Tech support engineers. The hotel team maintains a suite of more than 400 Goibibo and hotel partner APIs. These are stored as Postman Collections.
Hotel partners integrate the Goibibo APIs into their own systems in order to funnel a range of key hotel data into Goibibo’s booking engine, such as hotel details, room inventory, rates, amenities, and policies. Hotel data is stored in the Goibibo system, scrubbed, and finally passed on to the teams that manage the company’s consumer-facing websites. Every day, Goibibo’s APIs handle over 180, 000+ requests that update hotel data. With high volumes like this, integration testing is critical to ensure that these APIs are all working in harmony. The process combines individual units and tests them all as a group, including conducting schema and positive/negative test cases.On the Goibibo hotel team, everyone participates in testing APIs on Postman. Front-end developers tend to use the platform for ad hoc testing to check particular requests and responses, while back-end developers and tech support engineers are hands-on, writing test cases for every module release.
Structured collaboration on Postman
The hotel team’s rigorous approach to testing begins with their setup on the Postman platform. Postman workspaces allow individual team leaders to easily manage user permissions and version control processes. Using a repo-style model of collaboration, each team member is given their own personal workspace, and a team workspace serves as the system of record. The team’s master set of collections is maintained in the team workspace; everyone has permission to view, but only designated module owners can edit. Members fork the master collections, work on updates in their own workspace, and submit a request to the module owner to merge changes to the master repository on the team workspace.
To keep track of hundreds of requests and test cases, Goibibo follows a meticulous workspace file structure, which includes collections, API folders, requests, and success/fail test cases. Every collection is named after its GitHub repo, which makes integration with their Jenkins deployment pipeline that much easier. When members fork a master collection, they maintain the same naming conventions in their own personal workspace. The team also documented their Postman setup and collaboration process in Confluence to help other teams benefit from best practices.
Standardized testing using scripts and Newman
With a highly organized collection file structure in place, the hotel team used it to create a standard approach to integration testing across all services. Postman’s scripting feature puts the test flow in motion; before an API call is made, pre-request scripts set up environment variables and console logging. Then, once a response is returned, test scripts take these variables and carry out test assertions on response data or pass data between requests.
The test flow executes scripts in the order specified by the collection file structure: first at the collection level, then at the folder level, and finally at the request level. The flow is repeated before and after the API call. In addition, each request is run with two sets of data: 1) “good” data that returns a successful response, and 2) “bad” data that returns a failure response. For example, the team may test hotel authentication by passing either a correct or incorrect hotelID to ensure that the API is working as expected.
Once a collection is ready to test, the hotel team triggers Newman (Postman’s command line Collection Runner) to initiate the test flow. Individual engineers working in their personal workspace use Newman’s CLI to manually run collections on their local machine. To run the master collection on production, and integrate with their Jenkins deployment system, the team uses the Postman API to execute Newman commands. The entire process takes under two minutes to complete.
By standardizing testing on Postman, the Goibibo team has been able to improve their test-driven development (TDD) approach across the organization with greater collaboration, security, and error handling in place. The team is also better equipped to catch API errors, such as sending incorrect HTTP status codes.
The Postman API maintains security during Jenkins deployment
Goibibo leverages Postman’s Jenkins integration to connect Newman with their build system, allowing them to test, verify, and push an update straight into their CI/CD pipeline. As part of this process, the collection URL is passed across the network to Jenkins which, as an external service, makes the URL publicly accessible. With this kind of access, someone could conceivably copy the URL and give it to someone else, thus breaching the security of Goibibo’s API data. As Postman test cases represent 20% of Goibibo’s SOX compliance, this was a significant risk to the company’s compliance status.
Working with the Postman support team, Goibibo was able to leverage the Postman API to develop a solution. First, the team created a Postman API key for their account. Next, they built a command-line script using api.postman.com, the API key, and the collection UID in place of the URL. If needed, they could also pass global variables by including the environment UID in the script. As a result, Goibibo’s sensitive data is hidden and the entire deployment process is much more secure.
One of my favorite features in Postman is how the platform handles security. We could easily use the Postman API to create a command-line script that keeps our API data hidden from view. This helps us drive 20% of our SOX compliance efforts.
Goibibo scales integration testing to over a thousand tests per day
Over time, Goibibo’s hotel team saw positive results from their testing framework on Postman. Tests runs were faster—their setup could execute 212 daily test cases in under a minute. Individuals were more productive thanks to automation, and the overall team was more agile and efficient.
To capitalize on this momentum, Goibibo launched Project Automation, a new initiative focused on automating as many tests as possible on Postman across engineering teams. The company had originally set up two microservices on Postman. Three months into Project Automation, various teams had set up 17 microservices, including one of their largest. These tests run 700 requests that cover 1,700+ test cases during each daily deployment, and the volume is growing every week.
By moving API testing to Postman, Goibibo has been able to make better use of its engineering resources. Because the company does not hire dedicated QA engineers, teams typically have multiple developers doing QA testing. Now, they are able to work up to 10% faster, which results in cost savings for the company. It also frees up developers to focus on other tasks, such as automating more and more tests on Postman.
Automated testing on Postman resulted in a savings of one day in every 10-day sprint cycle, or about 10% greater team efficiency, which is freeing our developers to focus on higher-value tasks.
The hotel team leads Postman training and support
After adopting Postman, the hotel team has become the authority on Postman at Goibibo. More and more internal teams are requesting their own team workspaces, and many more engineers are expressing interest in learning more about Postman which made them form an internal Postman Community.
The hotel team maintains a number of support channels to help their colleagues, including a tips-and-tricks email newsletter, training sessions, blog posts and a dedicated Slack channel. A recent blog post describes the team’s approach to integration testing. The team also provides links to the Postman’s own community and Postman’s issue tracking on GitHub. Postman’s Slack integration even enables users to see automated notifications of system activity while they work.
Most of our developer community is already aware of Postman, but they don't always know what they can do with it. We help them understand the power of automated testing and collaboration on Postman.
Goibibo’s engineering organization plans to scale their use of Postman by applying their integration testing framework and best practices to performance testing. This will help them optimize their AWS infrastructure and further increase the benefits of automated testing at Goibibo.