In this lesson,
we'll discuss Integrations and APIs.
Integrations and application program interfaces,
or APIs as they're commonly called,
are the unsung heroes
behind the seamless interconnectivity
and enhanced capabilities
of today's modern software applications and automation.
When it comes to automation and orchestration,
integrations and APIs
are really the fundamental elements
that drive efficiency, innovation,
and scalability in modern system.
Before we dive into the specifics of each of these,
let's take a moment to define both integration and an API.
And integration refers to the process
of combining different subsystems
or components into one comprehensive system
to ensure that they all function properly together.
An API or Application Programming Interface
is a set of rules
and protocols that are used for building
and integrating application software.
An API will allow your product or service
to talk to another product or service
in a controlled environment
in using a specific data exchange format.
Now, when we talk about API,
it's a library of programming utilities
that are used to enable software developers
to access functions
or features of another application programmatically.
APIs are probably the number one thing
that we use in our integrations
to be able to communicate
between different portions of a microservice
or service oriented architecture.
By using an API,
we can automate the administration,
management and monitoring of various services
and cloud-based infrastructures,
as well as a lots of other applications.
Commonly, our APIs will use either REST
or SOAP to communicate.
REST, or the Representational State Transfer
is an architectural style
that uses standard HTTP methods
and status codes,
uniform resource identifiers, and MIME types
to achieve a simple way to interact in a stateless manner
with a given web service.
REST usually relies on using the JSON format
when transferring data,
which also makes it a very lightweight protocol
that can be easily integrated with existing websites
without needing to perform any kind
of extensive modifications.
SOAP or Simple Object Access Protocol,
is a protocol that defines a strict standard
with a set structure for the message,
usually in XML format,
that includes a header in a body
that is used to transport data
between two different services.
SOAP is known for being more robust than REST
by having some additional security features
and its ability to provide transaction compliance.
This makes it a great choice
for any enterprise level web services
where complex transactions
and regulatory compliance are essential.
While REST is generally more straightforward
and adaptable to utilize,
SOAP does provide us
with a higher level of security
and transactional integrity
that caters to some different use cases and requirements.
Now, when we talk about APIs,
we think about these from a perspective of integration.
APIs allow for integration
between lots of different services,
especially cloud-based services.
Most cloud-based service providers
also allow us to do provisioning,
configuration, deprovisioning,
and lots of other things to the services
by using these APIs.
APIs will also allow us
to have direct integration
of different third party applications
into our own web applications.
By using APIs, we can integrate services together
and make function calls to those services,
instead of having to build an entire service ourself.
With the API,
we can just call on someone else's function
and let their service do the work for us
and return the results back to us.
This is the whole idea of creating a microservice
or service-oriented architecture
instead of building a monolithic web application.
Now, let me give you a great example
of how we use integrations and APIs
in our business here at Dion Training.
To conduct our student support functions,
we use centralized
service management system called Freshdesk.
Freshdesk is a commercial software as a service product
that we pay for every month,
and we use this to be able
to get all of our trouble tickets
or questions submitted by our students,
and then we can track those things
to make sure that you're getting the answers you requested.
So the way we have it configured
is that if you email support@diontraining.com,
it will get sent to our Freshdesk system
so that our team can look at those emails,
which we call tickets,
and then answer the question that you sent us.
You can think about this as a shared inbox,
but since it's actually a ticketing system,
we won't lose track
of any requests we receive from our students
so that we can make sure everyone's questions
are being answered promptly.
Now, we also provide some of our courses on Udemy
and the Udemy students also want to be able
to ask us questions about those courses.
But most Udemy students don't email us directly,
and instead they simply post their questions
inside the question and answer section of that website.
Unfortunately, Udemy Q&A system
was never designed to support instructors
like Dion training
to have over a million Udemy students.
So it is really easy for students questions
to get missed in a Q&A section.
To overcome this limitation,
we would need to be able
to rewrite the Udemy website
to add the ticketing system capability
to our instructor backend of the Udemy system.
Since Dion Training doesn't own Udemy
and we don't have access
or rights to modify their website's code base,
we cannot add a ticketing system
to the Udemy instructor backend.
So how can I get all of those Q&A posts
loaded into my first desk ticketing system
so that we can track
and answer all of our students
without missing any of those questions?
Well, if you guess that we should use an API,
you are correct.
Udemy has created an API
that allows anyone to programmatically read
and respond to any Q&A posts.
So our development team at Dion Training
spent about a week creating a small integration
to curate the Udemy API.
At any time there's a new Q&A post-it,
we read it from the Udemy API,
and then create a new ticket in Freshdesk
using the Freshdesk API.
Now, all of our Udemy Q&As
will show up as trouble tickets in our system
just as if the student emailed us
at support@diontraining.com.
Now, when our team answers the question,
instead of sending the response by email,
since we don't know the Udemy student's email address,
our system runs another automation
that will read the response from our Freshdesk ticket
using their API,
and then post it as an answer to the Udemy Q&A
using the Udemy API.
This is the type of integration that you can do using API,
and in this case, the entire integration program
is less than 150 lines of Python code
running in a serverless AWS Lambda environment.
The entire thing took us about one to two weeks
to create, test and deploy,
and it only cost us about $1 per month
to run this code every four hours to get the new Q&As,
create the trouble tickets
and post the responses back to the Q&A section
once our team answers them.
This integration allows our team
to work more efficiently
since they can answer all student questions in one place
inside of Freshdesk,
and then the answers are sent back to the student
to their preferred support channel,
either email or to the Q&A section,
depending on where they submitted their question initially.
For the student,
they will never know anything is happening
in the background with this integration,
but instead, it looks as if I or Jason
log directly into Udemy
and responded to the Q&A that you posted.
But for our team,
it is easier for us to manage
and allows us to help more students quickly.
Another example of using an API integration
is one that we built that allows our students
to access our hands-on cloud-based labs
inside of any of our courses that you may have purchased
on diontraining.com.
So if you take a course at diontraining.com,
you'll see video lessons, quizzes,
our downloadable study guides,
and our practice exam,
all of which we created ourselves.
But when you get to the Lab-based section,
you will simply click on the link that says launch the lab.
The lab environment will be returned to you
inside of our learning management system
using this backend integration.
When it comes to the labs,
we don't build those ourselves,
but instead we rely on a third party partner
that we work with,
and we use them to integrate their labs
into your course experience,
and we want it to be integrated in a way
so that you don't even realize
that it was a third party provider tool.
In fact, in the old days,
we used to not have this integration,
and instead, our students had to log into the lab provider
the different username and password,
and then select the lab that they needed
to take at a specific time.
By using this API integration,
we remove the need for you having
to use another username and password
and the need to go to a different website,
and the need for you to figure out
which lab for the 20 or 30
that are included in the course
should be performed at that specific time.
But with our integration,
the whole experience is optimized.
So if you're taking a course at diontraining.com,
you'll have access to that full lab environment.
As you're going through the course,
you'll watch a video
and eventually get to the one that says,
"Now here's the lab.
In this lab, you're going to do X, Y, and Z.
Click this link to start the lab",
and that link actually looks something
like you would see here on the screen.
It says something like diontraining.com?Action=
run instance and ID equals,
and the count
and the access key and all those things.
These are the parameters that we're passing
to the API of our lab provider.
This allows our lab partner
to know which of our students is taking the lab
based on their student ID,
which lesson they're at in the course,
which lab needs to be ran,
and the authorization token.
We send all of these details
to our lab provider using their API
and in return, they provide us with the lab information
and the lab launch link
that will be displayed on your screen.
This is actually a magic link
that will let you open the lab
within the next three minutes.
Once you click on that link,
it will load the lab
inside of our learning management system for you.
You may have also noticed
that the URL has an authorization token
including in it as well.
This is because the cloud provider
needs to be able to know
who they're servicing
because we have hundreds of thousands of students,
and by knowing which student you are,
they can give you the right services
that we already paid for on your behalf
so that you're not charged anything additional
to access these labs.
Now that we understand a little bit more
about application programming interfaces and integrations.
You may be wondering how can you test if a given API
is working properly in your existing automation?
One of my favorite tools to interact with an API
is the Curl tool.
Curl comes install by default and Linux and MacOS systems,
but you can also install it yourself on your Windows system.
Curl is a tool used to transfer data
to and from a server,
using one of the supported protocols,
including HTTP, HTTPS, FTP,
FTPS,
SCP,
SFTP,
TFTP,
DICT,
TELNET,
LDAP or FILE.
Now, all of these protocols can be used with Curl
and the one you will use to test the API,
it's usually going to be HTTP or HTTPS,
depending on if you're testing an API
that is using an insecure connection
or a secure connection.
So how do you use Curl?
As a cybersecurity professional,
I can use Curl to test an API
by using the basic command,
curl, space, dash, dash, data space,
the argument that you want to pass,
and then the URL of the API.
So for example, if I wanted to test the httpbin.org API,
I could enter something like curl --data
"firstname=boolean&lastname=world"
https://http.bin.org/post.
In this case, I'm sending the first name equals boolean,
and the last name equals world.
And then I'm sending this to https"//httpbin.org/post.
When I'm sending that information,
what will I receive back from the API?
Well, for the httpbin API,
we will receive something in a JSON format
that has data that includes a list of arguments
and fields that came back from that particular API.
This allows you to send the data over the API
and see what comes back as response,
which is very helpful if you're working
as a software developer
or cybersecurity software engineer
as part of a penetration test.
So remember, integrations
and APIs are used to seamlessly create interconnections
between different services to enhance
their default capabilities.
Integration refers to the process
of combining different subsystems or components
into one comprehensive system
to ensure that they function properly together.
An API or Application programming interface,
is a set of rules and protocols
that are used for building
and integrating application software.
An API will allow your product or service
to talk to another product or service
in a controlled environment
and using a specific data exchange format.
When we are using automation or orchestration,
we rely heavily on the APIs
to create an interconnection
between our different microservices
or services provided by our third party partners.