006: Design discovery part 2 — Survey Results

— 4 minute read

I’ve been running the user survey for about a week now. It’s been good leaving it to do its job while I wrapped up some other more pressing work. I’ve got just over 100 responses, so there’s enough data to work with for this context.

The first thing I did with an export is chuck it into Notion into a table. Notion is super smart and laid out the data in a way that I could consume it really easily:

A screenshot of Notion with the survey responses in a table. Colours are used to help me to see the difference between certain answers.

Analysing the responses permalink

I was quite surprised by the results, so I’ll break each question down again, just like last time.

How easy was it to subscribe to Piccalilli? permalink

As I mentioned before, I had some reports of people getting trapped in a CAPTCHA loop or just never receiving emails, so I was very surprised to learn that 85% of respondents found it very easy to subscribe and the other 15% found it reasonably easy.

The issues still exist and probably always will (email is a nightmare), but it’s good to know that I can keep signup simple, as that appears to be working.

If you answered “Difficult”, can you give a short summary of why that was? permalink

No-one answered “Difficult”, but as anyone who’s conducted survey-based research will have guessed, I still got an answer or two that I didn’t expect.

One answer stood out and is as follows:

RSS feed should be easier to find.

I didn’t even realise there was an RSS feed on Piccalilli. One thing I want the new Piccalilli to be is a web-first setup, rather than an email-first setup, so this answer, albeit in the incorrect field has provided a bit of food for thought.

What best describes the environment that you read the Piccalilli newsletter? permalink

Surprisingly, more than 30% of the respondents said that they read Piccalilli in the browser. This is good to hear but also poses the question: “Does this mean that the newsletter doesn’t render well via email?”. Luckily that’ll be answered in the next question.

Most of the other respondents reported that they read Piccalilli in a web based email environment, which is most likely going to be Gmail. I need to keep my eye on how the new design performs in that environment.

How does the newsletter load in your environment? permalink

The majority answered with “well” or “reasonably well”, with only 7% answering “bad”. The percentage of users that report a “bad” doesn’t match up to the 30% that read Piccalilli in the browser, so I can trash that early assumption.

Why did you subscribe to Piccalilli? permalink

This question wasn’t as helpful as I would have liked it to have been. I probably shouldn’t have made it a multi-option response in hindsight.

That aside, the most popular answer was “To see cool animations and interactions”. A surprise for me was that “To read the weekly CSS pattern, trick or utility” featured in around 40% of the responses. This has made me re-consider slowing that aspect down a bit.

If you have any further comments about Piccalilli, please add them here permalink

This question brought in some truly lovely answers that made my cold-dead heart warm up a bit. Folks are very kind indeed and reading responses like that got me through the last week of a very large client project!

Some chosen responses that stood out:

Always went through to junk... Had to actively dig it out and usually the junk folder works really well.

Like I’ve said before, you’ll always get a little nugget of information in these responses. The junk situation could be anything, but I immediately point the blame at images. I might have to think about them in depth for this redesign and do some research on common reasons why emails land in junk.

One note on reading environment, I almost always have my first read in iOS mail, but will eventually re-read or revisit on my desktop (via Postbox) and sometimes at the website

This is an interesting bit of feedback. It reminds me that the content needs to be equally accessible for all environments (email, web, rss).

Wrapping up permalink

This exercise has been really useful, so I’ll wrap up this part of the discovery by answering the questions I outlined in the last post.