Build with Local LLM AI Models - Setting Up

I have been prototyping a number of LLM-powered AI digital products in the past couple of weeks to further understand how to work with and integrate AI models into my own projects and ideas, including running AI models locally, using AI to organise and make connections between my personal notes, creating a desktop app that reads my emails and draft replies in batch, AI as a dungeon master in a rogue-lite game, and using AI to continuously create new landing pages for A/B testing for my GMAT book. I’ll write about each in separate blog posts. But first, let’s get some AI running on my device that I can use to test my projects.

Running LLM Models Locally

I want to try running LLM models on my local machine. I have a Mac Studio (Apple M2 Max, 32GB RAM). It’s not the most powerful machine you can build to run AI models, but I wanted to know how capable a typical computer you might find on your desk can be. I have found these local models to be sufficiently capable in most instances for the typical writing related tasks, idea generation, and analysis.

Setting Up

The easiest way I found is to simply use Ollama, which is a desktop platform that allows you to run and interact with AI models locally on your device.

Criteria for Deciding on Ideas

Mike, my co-founder at markd.ltd, and I are both the kind of people that has many random ideas, and often find ourselves diving right in after we’ve come up with something new or novel or just quirky.

A few months ago we decided we should have a process to help us think about our ideas. Building new ideas is all fun and game, but at the end of the day, we’ve got a company to build, one which I hope can continue to be creative, fun, and useful.

So this is the criteria we came up with. This is not meant to be a general purpose criteria for filter ideas, but one that we decide to use because of our skills, experiences, personalities, and resource limitations.

My New Favourite Toys (creative apps, web apps) August 2019

Unfollow tool

Tokimeki Unfollow is a fun web app that helps you trim down your Twitter following. Without showing you the bios of people you are following on Twitter, you have to decide to keep following someone or not based on their recent tweets. I liked the personality of the app (too few social media tools have a personality!)

Olive video editor

TalentSearch.cc bombed on ProductHunt

So TalentSearch.cc bombed on ProductHunt.

It didn’t get nearly as many votes or as much attention as ColdEmailTemplate.cc or Markd.co, both were featured on the front page. I was overly confident in how useful TalentSearch.cc is for people (who wouldn’t want a search bar to find talents on multiple social platforms?!) and how popular it will be for the PH community, which also meant I spent way more time on it than ColdEmailTemplate.cc.

Tidy Up Your Following — The 100-Following Rule

Why We should KonMari Our Social Follows

—

A number of years ago I read Kevin Kelly’s essay on 1,000 True Fans. The idea that if you can find and grow an audience of 1,000 true fans that love what you do, you can be financially independent as a creative maker.

I liked this theory, and decided I will put it to practice by reducing the number of people I follow on Twitter.

Mensch Patterns

At Business of Software conference last month, I sat in a talk by Derek Sivers and learned about Mensch Patterns, his antidote to the inescapable Dark Patterns so prevalent on the web today (tricks that make users do things that they didn’t mean to).

Marketing Sucks

Marketing Sucks.

Marketing sucks, big time.

You push and pull and try to get known. You obsess over views and votes and all the likes you didn’t get.

Feeling Anxious? Try Writing

A couple of weeks ago on a Saturday, I was feeling anxious. For no apparently reason, it was one of those days when you couldn’t think of anything to feel down about, but feel down nonetheless.

So naturally to “fix” this, I made coffee. One cup of coffee, didn’t work. Two cups of coffee, didn’t work. Green tea? Didn’t help either. By mid afternoon, I was high on caffeine but low on mood.

Moving My WordPress Sites to Jekyll

Towards the end of 2018, I had the urge to “tidy up” my mental space to make room for Markd, and I decided to close down the numerous websites I’d had created and accumulated over the years.

For the sites that I do want to keep, I wanted them to be smaller sites, simpler projects.

For years I’d been using Wordpress for websites. It’s a powerful CMS that allowed me to get set up and running quickly. The massive plugin eco-system meant I could add the latest features fo these sites without technical knowledge. The themes let me create websites that looked nice without worrying about designing them myself.

But it also came with a cost. Plugins and themes constantly required updating. And dealing with spam was always an issue for some of my bigger sites. Maintenance became a bit of a headache, and with so many different plugins for different sites, it got overwhelming.

Blog is blog

Ten years ago, I had a blog. It was in mandarin and hosted on a popular blogging platform in Taiwan.

I wrote anything that came to mind. I wrote about the passing of my grandmother. I wrote about life in Japan. I wrote about new learnings. I wrote personal thoughts. I wrote poems.

Then I left my job to start a business. And all of a sudden, blogging becomes part of that. I thought I needed to write about business, technology, and whatever else good for SEO / personal brand …etc.