rollin' onwards with a web application

May 20, 2020

Imported from https://cyberia.club/blog Originally published: 2020-05-20

Forest 2020-05-20

WHAT'S NEW IN CAPSUL?

Capsul has been operated by hand so far, with business conducted via email. Obviously, this isn't the best user experience. If no one is on the other end at the time, the user might feel as if they are shouting into the void.

Ideally, users could pay for service, create and destroy capsuls, and monitor their capsul's status at any time.

So we set out to create an application enabling that, while keeping things as simple as possible. As of today, you can experience it firsthand!

—> https://capsul.org/ <—

WHAT IS CAPSUL? WHY WOULD ANYONE DO THAT?

Capsul started out as a “for fun” project to host multiple VMs with different operating systems on the same physical server.

A cloud compute provider experiment to find out:

How hard is it to build basic compute-as-service functionality that has been mythologized and commoditized by some of the biggest software businesses of all time.
What problems have to be solved in order to do this at a small scale?
And last but not least, how much better-than-the-big-boys can we do? :P

I heard about Capsul and I thought, cool, why not.

At first, I was slightly dismissive of the project — why re-invent the wheel? There are lots of established tools for creating cloud services already out there, surely they would be hard for us to measure up to.

Of course, you could argue, that's not the point. It's all about the journey, popping the hood and learning how things are put together.

But on the other hand, Capsul is something that we want to use, not just a pet project.

Can I depend on it?

/⁀\ _/‾⁀|, (‾‾____‾‾) | xx . .| / \ | [ >)
/ \ | ‶` ‾ |
I WANT TO BELIEVE | (X) DOUBT

Whether excited or doubtful, the tone of the question expresses the real utility and risk associated with DIY.

We have to make our own seat belts for this, an experience and practice that I personally feel is highly under-rated.

I don't want to give up and just leave it to the experts.

I want to build the confidence necessary to make my own systems, and to measure thier stability and efficacy.

(_/) [. .] ==<.>==

“ Anyone can Cook “

It probably helps that I've never seen a friend get hurt because of a flaw in something I designed, but even if I had, I'd like to think that I'd continue believing in the idea that technology is never “beyond” us. I could never make it through Technoholics Anonymous, because I'd never be able believe a Higher Power will restore sanity to the machine and save us from ourselves.

ABOUT THE DEVELOPMENT PROCESS

First step was to chose a language and framework. We made this decision (Python 3, Flask) almost entirely based on which language was the most commonly known in our group. I was the only one who had never used Python before, and I felt up to the task of learning a language as a part of this process.

Next, we had to decide how the system would work.

How would we secure user's accounts? How would users pay for capsuls? Would it be like a subscription, would you buy compute credits, or a receive a bill at the end of the month?

In the interest of simplicity, we opted to use a tumblr-style magic-link login instead of requiring the user to provide a password. So, you have to receive an email and click a link in that email every time you log in.

We also decided to go with the “purchase credits, then create capsul” payment workflow, because it was the easiest way we could accept both credit card and cryptocurrency payments, and we believed that requiring the user to pay first was an appropriate level of friction for our service, at least right now.

I had never worked on a project that integrated with a payment processor or had a “dollars” column in a database table before. I felt like I worked at the Federal Reserve, typing

INSERT INTO payments (account, dollars) VALUES ('forest', 20.00);

into my database during development.

The application has three backends:

a postgres database where all of the payment and account data is stored
the virtualization backend which lifecycles the virtual machines and provides information about them (whether or not they exist, and current IP address)
Prometheus metrics database which allows the web application to display real-time metrics for each capsul.

All of the payments are handled by external payment processors Stripe and BTCPay Server, so the application doesn't have to deal with credit cards or cryptocurrency directly. What's even better, because BTCPay Server tracks the status of invoices automatically, we can accept unconfirmed transactions as valid payments and then rewind the payment if we learn that it was a double-spend attack. No need to bother the user about Replace By Fee or anything like that.

The initial development phase took one week. Some days I worked on it for 12+ hours, I think. I was having a blast. I believe that the application should be secure against common types of attacks. I kept the OWASP Top 10 Web Application Security Risks in mind while I was working on this project, and addressed each one.

Injection We use 100% parameterized queries, and we apply strict validation to all arguments of all shell scripts.
Broken Authentication We have used Flask's session implementation, we did not roll our own sessions.
Sensitive Data Exposure We do not handle particularly sensitive data such as cryptocurrency wallets or credit card information.
XML External Entities (XXE) We do not parse XML.
Broken Access Control We have added the user's email address to all database queries that we can. This email address comes from the session, so hopefully you can only ever get information about YOUR account, and only if you are logged in.
Security Misconfiguration We made sure that the application does not display error messages to the user, we are not running Flask in development mode, we are not running Flask as the root user, the server it runs on is well secured and up to date, etc.
Cross-Site Scripting (XSS) We apply strict validation to user inputs that will be represented on the page, whether they are path variables, query parameters, form fields, etc.
Insecure Deserialization We use the most up-to-date json parsing from the Python standard library.
Using Components with Known Vulnerabilities We did check the CVE lists for any known issues with the versions of Flask and psycopg2 (database connector), requests, and various other packages that we are using, although automating this process would be much better going forward.
Insufficient Logging & Monitoring We may have some room for improvement here, however, verbose logging goes slightly against the “we don't collect any more data about you than we need to” mantra.

If you would like to take a peek at the code, it's hosted on our git server:

https://git.cyberia.club/cyberia/capsul-flask