Fast postgres index creation Postgres is a fantastic database for a multitude of reasons, celebrated for its reliability and flexibility. However, when creating indexes on large tables, it can sometimes feel sluggish. This becomes even more noticeable when your project requires the creation of numerous indexes, potentially slowing down your progress. Configure CPU and RAM available for index creation There are two settings that control the amount of server resources available for index creation, maintenance_work_mem and max_parallel_maintenance_workers....

Basics of Database Transactions in Go Before diving into strategies, let’s cover the basics of managing database transactions in Go. In Go, transactions are typically managed using the BeginTx function from the database/sql package. The basic flow for a transaction involves acquiring a transaction, performing operations, and then either committing or rolling back the transaction. tx, err := db.BeginTx(...) // Perform database operations _, err := tx.ExecContext(...) if err !...

In the world of programming, we often encounter scenarios where we need to convert the contents of a webpage into a PDF document. This is particularly common when we want to include a snapshot of the web content in emails, reports, or offline documentation. One challenge faced during this process is ensuring that the webpage is fully loaded before conversion. This post will guide you through some ways to achieve this task programmatically....

When programming in Go, the amount of error handling is something that slaps you in the face. Most API’s you deal with will expose errors. It can become overwhelming, but with a few tips and a guiding principle we can make handling errors easy, keep our code clean and give you the confidence that nothing is breaking in production. A cartoon of a crazy stressed programmer pulling their hair out in front of lots of screens showing a error exclamation marks...

Using Web Sockets, Http4s and FS2 are a great way to send streaming data to a client…

Concurrency in golang can get tricky, now that generics have landed a Future library can help.

Docker Dash is my new SaaS side project, it helps teams manage the security of their docker containers. This post is about some of the technical decisions I’ve made while building it.

How to clean your Docker Repository

How to Improve your Scala Compilation Times

Eliminate file sort for fast queries

How to use a Raspberry Pi as a home server instead of a VPS

How I grew the web traffic to Real Email

I’ve recently improved the csv parsing in my side project Real Email and built a work queue using FS2. In this post I will explain how to build a work queue with cancellation. The context for my work queue is that im validating csv files from a typical web app. This happens from the browser which then uses http polling to query for progress. The more interesting feature are that I wanted a way to cancel the processing and a way to limit the number of concurrent validations happening at once in case of a sudden spike in usage....

Im pleased to announce that on Real Email you can now validate a csv file using some interactive features. Real Email is a developer focused email validation service, allowing you to check if an email address is real, before sending anything and possibly hurting your email reputation through bouned emails. Its always been possible to possible to validate a csv file with a clever shell script but its not very user friendly, I didnt think it was a very important use case for my market....

Model driven components are a great way to quickly build out an app with consitent styling. A form builder component can make it easier to build apps.

In this post im going to talk about thread exhaustion of the scala global execution context and a curious case where it came up in a old version of scalacache. Its classic scala advice that you shouldnt use the default execution context, in this post we will look into detail why. We will look into how it is then avoided in scalacache while maximising code reuse. Background Use of Await.ready starts a new thread on the default execution context, so if that happens enough you can run out of threads....

Slow comile times in scala are a common complaint. This is usually because of advanced usage of implicit resolution and macros. The JSON libraries ArgonautShapeless for Argonaut, Circe and play-json make heavy use of that. I love the convience this approach provides, and I think its better than manually building codecs. Manually codecs can be error prone because you have to be careful about the position of arguments which may not even be defined in the same file....

This presentation shows some tricks for dealing with Scala Future’s.

Consider the following queries, each will lock the database in different ways. In this post Im going to show you how you see where each one is locking, and how to optimise it. update TabFoo set bla = ‘bla’ where id = 1; update TabFoo set bla = ‘bla’ where foo = ‘bla'; To see the locks in MySQL innodb, start a transaction run the query, then inspect the innodb status....

The Angular.js team have added a new way to access variables within your controller. When I first saw this I glossed over it and missed the point thinking it was just shortcut syntax, more ways of doing the same thing, but its not. In this post I’ll explain why the controller as syntax is good and you should be using it. The angular controller as syntax looks like this.. ng-controller=“MyController as ctrl1” with access like this....

This post is about some of the different ways we can test our scala code. I think this is a bit of an overlooked subject, you don’t see too many scala tutorials that show you how you should test and its often quite different. At the surface this seems pretty basic, but like most code there are lots of ways to do it. No mocking You could test something by running it, and checking that it does what it should....

When programming using angular.js the routers control the view, they are the link between the page url and what you see, what templates and controllers are shown on the page. In the Angular (1.x) ecosystem there are 3 main routers.* The standard router. angular-route* Angular-ui’s router ui-router* Angular-route-segment angular-route-segment SegmentsOne of the notable features is the ability to change segments of the page. Being able to have nested views, this is useful if you have a dynamic header with dynamic main content section....

This post is about how to integrate sessions in Play Framework servers through javascript. Where you have two different servers (micro services) and you want to share the session between them. Because the session in play is just stored in a secured cookie the servers dont need to communicate, they just need the same encryption key application.secret usually in application.conf. This is used to sign the session cookie so if its changed the server rejects it....

This post is going to be a discussion about url shorteners, what the problems are, and how they could be useful, some potential business ideas. IT people will be familiar with these techniques, but these things are not obvious to everyone. The basic idea is to take big url and make it shorter so it can more easily be shared. In some places its nicer to have short urls like http://stephenn....

Tero Parviainen has written this amazing article. Making your own Angular.js http://teropa.info/blog/2013/11/03/make-your-own-angular-part-1-scopes-and-digest.html Its amazing how much you can make with frameworks like angular.js without understanding it. But learning how it works is going to help you use it better and make better apps. Im finding with my scala server side programming that I dont need a big framework. With techniques like passing functions Im able to easily remove boiler plate....

One day I want to start a online business, to sell some software as a service or something. Im not sure what yet, but when I do I'll need to do some marketing. I've never really done that. You cant have a business without customers, you cant have customers without marketing. So I decided to try testing a simple marketing strategy, Adwords. I put up a simple website, a fake online store that sold nuts, an existing market, something I knew people already wanted....

This post is about hosting files securely on amazon S3 and making those available through a web app. It can be a bit tricky to configure S3 through its various caches. Setup the bucket with webapp user that can upload / download make links. First off we need to create a user in amazon IAM for our webapp to login as. Do this through the console and take note of its ARN, use the keys to login....

While writing apps with angular.js (which I love) I've come across performance issues when rendering big lists. Like in searchbrew.com. For example if I do this with 2500 docs. It takes a while to render and slows the page down. <tr ng-repeat="doc in allDocs">     <td><a ng-href="{{ "{{" }}doc.url}}">{{ "{{" }}doc.title}}</a></td>     <td>{{ "{{" }}doc.description}}</td> </tr>We can see and measure this performance using chromes flame chart in the javascript cpu profiler....

I made this website the other day. searchbrew.com This article about how I made it. Its a search for hombrew packages. Brew by itself doesnt have a very good search, because they dont want to collect description metadata about packages. Brew does have a link to the homepage of each package. I tried getting descriptions and indexing the homepage for search, but it didnt work very well. Then I came across this project telemachus/homebrew-desc which includes project descriptions....

I just gave my second presentation at wellington.js, and a friend asked why? Why present at the meetup? The first question should be why go? Why go?They are a good way to connect to other people interested in the same stuff. Its good to have those people to bounce ideas off. Meet people to start new projects with. You learn new stuff! Its way easier learning about something new when you can just sit back, listen, and ask questions....