Hi there 👋

I like programming, SaaS and bootstrapping

@ Upollo


Fast postgres index creation

Fast postgres index creation Postgres is a fantastic database for a multitude of reasons, celebrated for its reliability and flexibility. However, when creating indexes on large tables, it can sometimes feel sluggish. This becomes even more noticeable when your project requires the creation of numerous indexes, potentially slowing down your progress. Configure CPU and RAM available for index creation There are two settings that control the amount of server resources available for index creation, maintenance_work_mem and max_parallel_maintenance_workers....

<span title='2024-03-17 00:00:00 +0000 UTC'>March 17, 2024</span>&nbsp;·&nbsp;2 min&nbsp;·&nbsp;Stephen Nancekivell

Mastering Database Transactions in Go: Strategies and Best Practices

Basics of Database Transactions in Go Before diving into strategies, let’s cover the basics of managing database transactions in Go. In Go, transactions are typically managed using the BeginTx function from the database/sql package. The basic flow for a transaction involves acquiring a transaction, performing operations, and then either committing or rolling back the transaction. tx, err := db.BeginTx(...) // Perform database operations _, err := tx.ExecContext(...) if err !...

<span title='2023-08-04 00:00:00 +0000 UTC'>August 4, 2023</span>&nbsp;·&nbsp;4 min&nbsp;·&nbsp;Stephen Nancekivell

PDFs on the Fly: Programmatically Transforming Webpages into PDFs

In the world of programming, we often encounter scenarios where we need to convert the contents of a webpage into a PDF document. This is particularly common when we want to include a snapshot of the web content in emails, reports, or offline documentation. One challenge faced during this process is ensuring that the webpage is fully loaded before conversion. This post will guide you through some ways to achieve this task programmatically....

<span title='2023-06-23 00:00:00 +0000 UTC'>June 23, 2023</span>&nbsp;·&nbsp;3 min&nbsp;·&nbsp;Stephen Nancekivell

Gopher Wrangling. Effective error handling in Go

When programming in Go, the amount of error handling is something that slaps you in the face. Most API’s you deal with will expose errors. It can become overwhelming, but with a few tips and a guiding principle we can make handling errors easy, keep our code clean and give you the confidence that nothing is breaking in production. A cartoon of a crazy stressed programmer pulling their hair out in front of lots of screens showing a error exclamation marks...

<span title='2023-06-19 00:00:00 +0000 UTC'>June 19, 2023</span>&nbsp;·&nbsp;4 min&nbsp;·&nbsp;Stephen Nancekivell

How to Serve Web Sockets with Http4s

Using Web Sockets, Http4s and FS2 are a great way to send streaming data to a client…

<span title='2022-07-09 00:00:00 +0000 UTC'>July 9, 2022</span>&nbsp;·&nbsp;2 min&nbsp;·&nbsp;Stephen Nancekivell

Simplifying Go Concurrency with Futures

Concurrency in golang can get tricky, now that generics have landed a Future library can help.

<span title='2022-05-17 00:00:00 +0000 UTC'>May 17, 2022</span>&nbsp;·&nbsp;5 min&nbsp;·&nbsp;Stephen Nancekivell

Building Docker Dash - Technical Decisions

Docker Dash is my new SaaS side project, it helps teams manage the security of their docker containers. This post is about some of the technical decisions I’ve made while building it.

<span title='2021-11-18 00:00:00 +0000 UTC'>November 18, 2021</span>&nbsp;·&nbsp;4 min&nbsp;·&nbsp;Stephen Nancekivell

Docker Repository Cleaning

How to clean your Docker Repository

<span title='2021-04-12 00:00:00 +0000 UTC'>April 12, 2021</span>&nbsp;·&nbsp;2 min&nbsp;·&nbsp;Stephen Nancekivell

How to Improve your Scala Compilation Times

How to Improve your Scala Compilation Times

<span title='2021-04-12 00:00:00 +0000 UTC'>April 12, 2021</span>&nbsp;·&nbsp;5 min&nbsp;·&nbsp;Stephen Nancekivell

Improve SQL Join Queries that use Temporary Tables & File Sorts

Eliminate file sort for fast queries

<span title='2021-03-27 00:00:00 +0000 UTC'>March 27, 2021</span>&nbsp;·&nbsp;3 min&nbsp;·&nbsp;Stephen Nancekivell

How to use a Raspberry Pi as a home server instead of a VPS

How to use a Raspberry Pi as a home server instead of a VPS

<span title='2021-02-07 00:00:00 +0000 UTC'>February 7, 2021</span>&nbsp;·&nbsp;1 min&nbsp;·&nbsp;Stephen Nancekivell

Growing Web Traffic at Real Email

How I grew the web traffic to Real Email

<span title='2021-01-25 00:00:00 +0000 UTC'>January 25, 2021</span>&nbsp;·&nbsp;3 min&nbsp;·&nbsp;Stephen Nancekivell

Building a work queue with Scala & FS2

I’ve recently improved the csv parsing in my side project Real Email and built a work queue using FS2. In this post I will explain how to build a work queue with cancellation. The context for my work queue is that im validating csv files from a typical web app. This happens from the browser which then uses http polling to query for progress. The more interesting feature are that I wanted a way to cancel the processing and a way to limit the number of concurrent validations happening at once in case of a sudden spike in usage....

<span title='2020-09-12 00:00:00 +0000 UTC'>September 12, 2020</span>&nbsp;·&nbsp;3 min&nbsp;·&nbsp;Stephen Nancekivell

Adding CSV to Real Email Validations and feature validation

Im pleased to announce that on Real Email you can now validate a csv file using some interactive features. Real Email is a developer focused email validation service, allowing you to check if an email address is real, before sending anything and possibly hurting your email reputation through bouned emails. Its always been possible to possible to validate a csv file with a clever shell script but its not very user friendly, I didnt think it was a very important use case for my market....

<span title='2020-07-31 00:00:00 +0000 UTC'>July 31, 2020</span>&nbsp;·&nbsp;2 min&nbsp;·&nbsp;Stephen Nancekivell

Model driven form builder component in Angular

Model driven components are a great way to quickly build out an app with consitent styling. A form builder component can make it easier to build apps.

<span title='2020-06-30 00:00:00 +0000 UTC'>June 30, 2020</span>&nbsp;·&nbsp;4 min&nbsp;·&nbsp;Stephen Nancekivell

Avoiding Scala Thread Exhaustion in Scalacache

In this post im going to talk about thread exhaustion of the scala global execution context and a curious case where it came up in a old version of scalacache. Its classic scala advice that you shouldnt use the default execution context, in this post we will look into detail why. We will look into how it is then avoided in scalacache while maximising code reuse. Background Use of Await.ready starts a new thread on the default execution context, so if that happens enough you can run out of threads....

<span title='2018-03-15 00:00:00 +0000 UTC'>March 15, 2018</span>&nbsp;·&nbsp;6 min&nbsp;·&nbsp;Stephen Nancekivell

Scala Circe, Argonaut Shapeless, Play Json Compile Times

Slow comile times in scala are a common complaint. This is usually because of advanced usage of implicit resolution and macros. The JSON libraries ArgonautShapeless for Argonaut, Circe and play-json make heavy use of that. I love the convience this approach provides, and I think its better than manually building codecs. Manually codecs can be error prone because you have to be careful about the position of arguments which may not even be defined in the same file....

<span title='2017-07-06 00:00:00 +0000 UTC'>July 6, 2017</span>&nbsp;·&nbsp;2 min&nbsp;·&nbsp;Stephen Nancekivell

Tricks with Futures

This presentation shows some tricks for dealing with Scala Future’s.

<span title='2016-12-10 15:23:43.02 +1000 +1000'>December 10, 2016</span>&nbsp;·&nbsp;2 min&nbsp;·&nbsp;Stephen Nancekivell

How to examine and optimise locking in MySQL

Consider the following queries, each will lock the database in different ways. In this post Im going to show you how you see where each one is locking, and how to optimise it. update TabFoo set bla = ‘bla’ where id = 1; update TabFoo set bla = ‘bla’ where foo = ‘bla'; To see the locks in MySQL innodb, start a transaction run the query, then inspect the innodb status....

<span title='2015-03-07 23:46:00 -0800 -0800'>March 7, 2015</span>&nbsp;·&nbsp;3 min&nbsp;·&nbsp;Stephen Nancekivell

How to write more maintainable code with the angular.js controller as syntax.

The Angular.js team have added a new way to access variables within your controller. When I first saw this I glossed over it and missed the point thinking it was just shortcut syntax, more ways of doing the same thing, but its not. In this post I’ll explain why the controller as syntax is good and you should be using it. The angular controller as syntax looks like this.. ng-controller=“MyController as ctrl1” with access like this....

<span title='2015-01-06 03:30:00.001 -0800 -0800'>January 6, 2015</span>&nbsp;·&nbsp;3 min&nbsp;·&nbsp;Stephen Nancekivell

Scala Testing Techniques

This post is about some of the different ways we can test our scala code. I think this is a bit of an overlooked subject, you don’t see too many scala tutorials that show you how you should test and its often quite different. At the surface this seems pretty basic, but like most code there are lots of ways to do it. No mocking You could test something by running it, and checking that it does what it should....

<span title='2014-12-13 18:03:00.001 -0800 -0800'>December 13, 2014</span>&nbsp;·&nbsp;5 min&nbsp;·&nbsp;Stephen Nancekivell

A Comparison of Angular Routers

When programming using angular.js the routers control the view, they are the link between the page url and what you see, what templates and controllers are shown on the page. In the Angular (1.x) ecosystem there are 3 main routers.* The standard router. angular-route* Angular-ui’s router ui-router* Angular-route-segment angular-route-segment SegmentsOne of the notable features is the ability to change segments of the page. Being able to have nested views, this is useful if you have a dynamic header with dynamic main content section....

<span title='2014-12-08 02:55:00.005 -0800 -0800'>December 8, 2014</span>&nbsp;·&nbsp;3 min&nbsp;·&nbsp;Stephen Nancekivell

Integrating Play Micro Services With Shared Sessions Through Javascript

This post is about how to integrate sessions in Play Framework servers through javascript. Where you have two different servers (micro services) and you want to share the session between them. Because the session in play is just stored in a secured cookie the servers dont need to communicate, they just need the same encryption key application.secret usually in application.conf. This is used to sign the session cookie so if its changed the server rejects it....

<span title='2014-06-11 00:04:00.001 -0700 -0700'>June 11, 2014</span>&nbsp;·&nbsp;1 min&nbsp;·&nbsp;Stephen Nancekivell

URL Shortener

This post is going to be a discussion about url shorteners, what the problems are, and how they could be useful, some potential business ideas. IT people will be familiar with these techniques, but these things are not obvious to everyone. The basic idea is to take big url and make it shorter so it can more easily be shared. In some places its nicer to have short urls like http://stephenn....

<span title='2014-05-04 00:07:00.002 -0700 -0700'>May 4, 2014</span>&nbsp;·&nbsp;3 min&nbsp;·&nbsp;Stephen Nancekivell

Review: Making your own Angular.js

Tero Parviainen has written this amazing article. Making your own Angular.js http://teropa.info/blog/2013/11/03/make-your-own-angular-part-1-scopes-and-digest.html Its amazing how much you can make with frameworks like angular.js without understanding it. But learning how it works is going to help you use it better and make better apps. Im finding with my scala server side programming that I dont need a big framework. With techniques like passing functions Im able to easily remove boiler plate....

<span title='2014-04-24 02:02:00.001 -0700 -0700'>April 24, 2014</span>&nbsp;·&nbsp;1 min&nbsp;·&nbsp;Stephen Nancekivell

My first shot at testing a market

One day I want to start a online business, to sell some software as a service or something. Im not sure what yet, but when I do I'll need to do some marketing. I've never really done that. You cant have a business without customers, you cant have customers without marketing. So I decided to try testing a simple marketing strategy, Adwords. I put up a simple website, a fake online store that sold nuts, an existing market, something I knew people already wanted....

<span title='2014-04-04 04:05:00 -0700 -0700'>April 4, 2014</span>&nbsp;·&nbsp;2 min&nbsp;·&nbsp;Stephen Nancekivell

Hosting Files In S3 Securely

This post is about hosting files securely on amazon S3 and making those available through a web app. It can be a bit tricky to configure S3 through its various caches. Setup the bucket with webapp user that can upload / download make links. First off we need to create a user in amazon IAM for our webapp to login as. Do this through the console and take note of its ARN, use the keys to login....

<span title='2014-03-19 19:10:00.002 -0700 -0700'>March 19, 2014</span>&nbsp;·&nbsp;1 min&nbsp;·&nbsp;Stephen Nancekivell

Javascript apps and rendering big lists

While writing apps with angular.js (which I love) I've come across performance issues when rendering big lists. Like in searchbrew.com. For example if I do this with 2500 docs. It takes a while to render and slows the page down. <tr ng-repeat="doc in allDocs">     <td><a ng-href="{{ "{{" }}doc.url}}">{{ "{{" }}doc.title}}</a></td>     <td>{{ "{{" }}doc.description}}</td> </tr>We can see and measure this performance using chromes flame chart in the javascript cpu profiler....

<span title='2014-03-14 04:11:00.001 -0700 -0700'>March 14, 2014</span>&nbsp;·&nbsp;1 min&nbsp;·&nbsp;Stephen Nancekivell


I made this website the other day. searchbrew.com This article about how I made it. Its a search for hombrew packages. Brew by itself doesnt have a very good search, because they dont want to collect description metadata about packages. Brew does have a link to the homepage of each package. I tried getting descriptions and indexing the homepage for search, but it didnt work very well. Then I came across this project telemachus/homebrew-desc which includes project descriptions....

<span title='2013-12-31 14:00:00.001 -0800 -0800'>December 31, 2013</span>&nbsp;·&nbsp;2 min&nbsp;·&nbsp;Stephen Nancekivell

Tech Meetup's and Presenting

I just gave my second presentation at wellington.js, and a friend asked why? Why present at the meetup? The first question should be why go? Why go?They are a good way to connect to other people interested in the same stuff. Its good to have those people to bounce ideas off. Meet people to start new projects with. You learn new stuff! Its way easier learning about something new when you can just sit back, listen, and ask questions....

<span title='2013-09-12 02:45:00 -0700 -0700'>September 12, 2013</span>&nbsp;·&nbsp;2 min&nbsp;·&nbsp;Stephen Nancekivell