
Ask HN: What automation tools have you used to replace mundane activities? - beatthatflight
For my travel site side hustle (beatthatflight.com.au&#x2F;), I find and publish cheap flights for Australians. Sounds simple, but even if say, someone gave me the flights each day, there&#x27;s posting the, sometimes to deals sites, as well as then editing it and posting it on my site, and out to social media, and then to specific mailing lists on mailchimp, depending on the source city of the deal, or type of post.<p>A LOT of that becomes tedious. Mailchimp can auto post to FB&#x2F;twitter&#x2F;instagram etc, but even converting the blog post into mailchimp emails gets tiring, as you need to choose a feature photo, the specific mailing list etc.<p>I&#x27;m a selenium guy, so have built some scripts to minimise some of it, but the rush each time of finding a deal still pains me when I think of the tedium ahead of publishing it.<p>Ideally, my goal would be to submit [to][from][when] to a script and the whole process would be automatic.  I can see that selenium <i>could</i> do it, but my goodness, it&#x27;d be slow and potentially flaky all via the web UI for those sites.<p>I&#x27;d love suggestions, examples of automation you&#x27;ve done, and tools used?
======
iKlsR
I have a slack workspace that has several bots, some listen for specific
keywords and or phrases around the web and post to the relevant channels.
Other bots remind me if I'm slacking off (maybe stayed a little too long on
youtube), remind me of appointments, to drink water, if a stock price falls
and some more. Also have filtered rss feeds for business and finance news etc.
All of these bots also have custom identities that make it much more engaging
like Axelrod is my bot for finance related news.

Others tell me if my server is getting swamped, forward important emails like
renewing a cert, a deploy failed, bug in prod or something else that requires
often immediate attention. Chat widgets from sites in production also go
directly to Slack so I can reply quickly if needed. (Gmail, Sentry, Freshchat,
Ploi etc, integrate them all and filter out what's critical).

Everything generally operates on a 15 min interval or so it's not noisy and
I've pretty much tuned it to only receiving specific stuff I want to see.
Having a central assistant like hub to go to is very useful and time saving.

~~~
kryptn
Oh cool, I didn't know if others were doing this. I've recently just started
going down this path with some zero effort IFTTT triggers but I'm not happy
with how infrequent they seem to trigger. My next step is to build my own set
of listeners/scrapers.

~~~
bluehatbrit
I posted the same thing on a similar thread yesterday but if you're looking to
build some scrapers and listeners and make it easy for yourself I'd highly
recommend Huginn
([https://github.com/huginn/huginn](https://github.com/huginn/huginn)). I set
it up a few months back and I've been using it to scrape all sorts of info and
alert me on things. IE:

* A change in GBP -> NOK exchange rate, to indicate if it's a good time to buy ready for my next trip to Norway.

* Alerts on flight prices to help me purchase flights when they're at their cheapest.

* Daily digest email with content I'm interested in from HN and some web comics.

* A summary of new content coming into my RSS reader.

It works really well for this sort of stuff and I found it fairly easy to get
up and running.

~~~
mark_l_watson
Thanks for the link! huginn looks cool and useful (good combination!).
Impressive work by 179 contributors.

I have had numerous simpler projects automating my digital life but I made the
(probable) mistake of always starting from scratch and writing stuff all on my
own.

~~~
bluehatbrit
I was in a similar boat but refused to build my own system. Huginn was a
perfect solution for me and I really enjoy using it. You can write your own
agents if you need to but I haven't had to so far.

Since I set it up a few months ago I've been using it a bunch, it's event
driven system is great for when you want to trigger off multiple streams half
way through a flow. You can also go back through past events if you want which
is really powerful.

------
buzzy_hacker
I use YNAB ([https://youneedabudget.com/](https://youneedabudget.com/)), which
requires inputting every one of your transactions by hand or importing them
from your bank. The downside of the import is it takes a few days for the
transaction to show up and you need to give YNAB your bank username/password,
which I'm not comfortable with.

I wrote this [https://github.com/buzzlawless/ynab-live-
import](https://github.com/buzzlawless/ynab-live-import) to import credit card
transactions instantly with no need of giving up my bank credentials.

The whole stack runs on Amazon Web Services. Simple Email Service receives a
notification email from the bank that I've made a purchase, saves it to S3,
and triggers a particular lambda function tailored to whichever bank the
notification came from. The lambda function retrieves the email from S3,
parses it for transaction data (account, payee, amount, date), and writes that
data to a DynamoDB table. The table has a stream enabled, which triggers
another lambda function when the table is updated. The function reads the
transaction data from the stream and posts the transaction to YNAB using their
API.

I've mentioned this on HN once or twice before and got some positive interest
with people even submitting pull requests, which is awesome :) Going to find
the time soon to review those and maybe add more features

~~~
bathtub365
That sounds a lot more complicated than just downloading bank statements and
uploading them to YNAB and adds a lot more points of failure

~~~
ralmeida
It does, but it also keeps YNAB more in sync with your bank account, since
transactions will typically take a couple days to show up in the statement.

~~~
bathtub365
Sure, but a database? Isn’t that part of what YNAB provides?

~~~
aphit
One of the key features of YNAB is using it to actually inform you before you
make purchases. By having it near instant (manual, or by this method), your
decisions are fully informed when you make them.

------
defterGoose
I used to work at an architecture firm that specialized in design and
manufacturing. One of our more popular products was facades made from aluminum
panels with "graphic perforation", a pattern of holes made to form an image at
a distance. As you can imagine, for larger projects the number of unique
pieces to be cut was into the hundreds. This represented a lot of time for our
CAM programmers because the (quite mature and industry standard) CAM software
we used was incapable of determining the proper cut orientation without manual
input. This meant clicking holes a bunch of times per panel to switch which
side of the curve was cut. No Bueno.

One of our primary CAD suites was Rhino which is very nice and has a great
python API. So I wrote a full-fledged 2.5D CAM processor for that. This
allowed us to batch process hundreds of these parts with a single click.

------
lostintangent
Playwright is a new alternative to Selenium, and besides being cross-browser,
is focused on performance, developer ergonomics and reliability (death to
flakiness!). Might be worth checking out?
[https://github.com/microsoft/playwright](https://github.com/microsoft/playwright).

Disclaimer: I work with the Playwright team at Microsoft.

~~~
rumanator
Hi thanks for mentioning your project. Are there any plans to use it for
automated UI testing for desktop apps? Selenium supports it (more or less)
with WinAppDriver, which is another project from Microsoft.

[https://github.com/microsoft/WinAppDriver](https://github.com/microsoft/WinAppDriver)

~~~
lostintangent
Desktop support is definitely on our long-term roadmap! We’re simply starting
web, since it’s an area we think we can make big improvements to, before
moving on to the broader developer landscape.

As we build out Playwright, we hope to learn a lot about how to make
automation more reliable, ergonomic, and enjoyable for developers. Ideally, we
could then apply those principles/learnings to desktop, mobile, etc.
automation too. Stay tuned!

------
dhruvkar
>>my goodness, it'd be slow and potentially flaky all via the web UI for those
sites

Generally, I've had better luck using undocumented APIs for this kind of
stuff. I was a heavy user of Selenium to automate many a task (I work in
wholesale construction goods, tons of automations needed everywhere).

But then discovered that using internal APIs (which surprisingly don't change
much) is far easier than trying to exception manage changing UIs.

Open up dev tools on your browser and watch the GETS/POSTS as you complete
your daily tedium. I use Python (usually just requests and beautifulsoup is
enough) to mimic the calls and I have yet to find a use case where I wasn't
able to automate something online.

I'm currently developing an SDK for our 18th-century ERP system, which has no
API. I've also automated getting shipping container tracking data from a
various shipping lines and railroad company websites, many with complicated
login processes.

Happy to chat, email in profile.

~~~
ctrl4
Yup. I did this with a local Android app, that tells you the gps location of a
public bus. I sniffed the http packets, found the requests i needed and made a
python script that pinged me at slack when my bus was 5 minutes from my bus
stop. Useful when I worked at night.

~~~
spikej
I'm familiar with the web dev tools, but no clue about mobile. Can you point
me in the right direction on how to do the same?

Were you using an android emulator on desktop or there's some way to easily do
this on android?

~~~
dhruvkar
I use mitmproxy or charles proxy with an android emulator, with an older
version of Android so it allows for a man-in-the middle, without being finicky
about certificates.

------
cpclermont
I have 141 custom scripts in my $HOME/bin. Most of it is in bash, but there's
a couple of node.js scripts as well.

\- I _mostly_ automate my bookkeeping with a set of recurring & dependent
taskwarrior tasks and scripts as annotations that I run with taskopen[1].
That's creating a bunch of folders, turning some emails in mutt into PDFs,
gathering PDFs from emails, fetching bills with selenium, moving files from
$DOWNLOADS into the appropriate bookeeping folder, putting a date on some
files, turning the whole thing into a zip file, and sending it to the
bookkeeper with mailx.

\- I automated the email send of my daily summary to my clients with mailx (so
I can send it directly from vim)

\- I automated turning screen recordings into thumbnails+mp4 link (since
GitHub only supports gifs)

\- I automated making before/after screen recordings for when I do noticeable
performance improvements (page load/animations)

\- I automated booting/killing my development servers

\- I automated making PRs with `hub pr` (finding the origin/upstream, putting
labels, etc.)

\- I bound to a key combo switching to the logs of specific development
servers

\- I turned my client's time tracking (tempo) into a CLI because I got tired
of using the UI to say I worked X hours on that ticket and 7.5 - X on the
other. Now I only do `tempo log $ticket1 2h $ticket2 3h $supportTicket rest`

[1]:
[https://github.com/ValiValpas/taskopen](https://github.com/ValiValpas/taskopen)

~~~
YourMatt
How do you like using Node for CLI? I started playing around with it a couple
years ago, and at this point, I don't even consider BASH scripts if they
involve the slightest bit of logic. Part of me feels like by going all-in, I'm
going to lose touch with my ability to write direct BASH scripts, which are
likely to work literally forever. Meanwhile, I expect my Node scripts to have
a relatively short lifespan, particularly in that I'm leaning on NPM. It's
been so nice though, I've decided it was a worthy tradeoff.

~~~
cpclermont
I've been doing pretty much the same thing. If the logic gets crazy, of if I
want lots of subcommands, I'd write it with node & commander. There's only so
much you can do with `jq` when it comes to processing JSON.

------
martin_a
I run Home Assistant with Zigbee devices for automatic lighting at home:

\- switch on when I get home and sun is set

\- switch on when I'm home and the sun sets

\- switch on in the morning, full brightness, coldest color to help waking up
(really helps me a lot and is not as stressful as an alarm clock)

\- automatically turn light off when I (my smartphone) leave home (not visible
in the WLAN for 10 minutes)

Will look into more automations there. Perhaps something about turning the
heat up in the bathroom in the morning when the windows are closed (there are
sensors for that).

I ran NodeRed on a Raspberry Pi too for some time, but I found that helped me
only little. Mainly did logfile analysis through that but did not look into
other use cases for me.

For work I've got the de-facto standard for the printing industry, I'm doing
everything possible with that.

~~~
mnbbrown
Interested why you use the coldest colour for waking up? I do the same, but
use a sunrise simulation instead that goes from warm to cold.

~~~
martin_a
Hm, maybe it's self inflicted torture. I just feel uncomfortable with the
blueish light and will get up quickly to leave that behind me. Maybe I will
give your sunrise simulation a try, might work just as well, not sure.

------
htk
I manage a Family Office and do monthly performance presentations with
PowerPoint, the slides have an unique design, and updating them every month
became extremely tedious and error prone.

I was already looking for an excuse to learn Python and this was perfect.

Instead of recreating the slides I created templates (just the last set of
presentations) and update them with info directly queried from the database,
using python-pptx and pyodbc.

This isn’t exactly an example of a “tool”, but I think the most important part
isn’t really the tool, but to be able to identify what you can automate. How
you do it might range from DIY to paying someone to code for you, but the
feeling after getting rid of manual processes is great!

~~~
simbas
Check out revealjs.com

~~~
excitednumber
Reveal is good but very hard to recreate ppt slides.

------
benibela
I spend a lot of time with web automation

Especially for public library catalogues. There the webpage shows when the
borrowed books are due, and you need to check it every week, or pay late fees.
And you cannot set a bookmark, since you need to login first, which is
tedious. So I automated it to renew all books and show me a warning.

And because that needs to run silently everyday, I decided to make it as fast
as possible (and because I only had a single core cpu with less than 1gb ram).
No browser, no javascript, no selenium, no python, no java. Only viable
languages would be C or Pascal, and for memory safety I implemented everything
in Pascal.

Because it is also tedious to build HTTP requests in native code and recompile
it all the time, I then wrote a scripting language for it, all the webscraping
is done in the script, but it is still fast because the processing is done in
native code.

To read data from the HTML, I use pattern matching, e.g. <a href="{.}"/>\+
would read all links, and <a href="{.}">foo</a>\+ would read all links with
text foo.

Nowadays I do not even visit public libraries anymore, but the automation tool
has become an open-source project at
[http://www.videlibri.de/xidel.html](http://www.videlibri.de/xidel.html) .

I also started playing some browser games, but they are boring, so I stopped.
But I want to stay in the highscore list, so I wrotes bot for them. Because
they have anti cheat detections, you cannot send the http requests directly.
For javascript based games, I wrote a bot with Greasemonkey. For flash games
that does not work, so I have decompiled the flash file, added the automation
functions to the flash, and then recompiled it.

------
JshWright
I use Home Assistant to automate a lot of stuff around my house. The one I use
at least daily is a trigger based on the light switch in my bedroom. A double
click down triggers an automation that turns off all the lights in the rest of
the house, makes sure the garage doors are closed, and that the rest of the
doors are locked.

~~~
patrickk
One future Home Assistant project that I would love to implement is a whole
house lighting system that gradually transitions all the lights from
white/blue in the morning, to yellow/orange in the evening to aid natural
melatonin production. Such a project would have to be fully automated,
otherwise I'd forget to set it each day. Blue LEDs are terrible for sleep and
throw off your body's natural sleep pattern. I got the idea from the excellent
book "Why We Sleep"[1]. It's possible that a similar HA project already
exists.

[1]
[https://en.wikipedia.org/wiki/Why_We_Sleep](https://en.wikipedia.org/wiki/Why_We_Sleep)

~~~
huseyinkeles
I use Circadian Lighting [0] component for this one. Works pretty well all
around the house.

[0] - [https://community.home-assistant.io/t/circadian-lighting-
cus...](https://community.home-assistant.io/t/circadian-lighting-custom-
component/61246)

~~~
patrickk
Wow, very nice. This looks like exactly what I had in mind.

------
dana321
You mention selenium, i use Selenium IDE

Its a browser plugin that allows you to record and playback anything that you
do in your browser.

[https://selenium.dev/selenium-ide/](https://selenium.dev/selenium-ide/)

There is a right-click action to record hovering over a section if it is
hidden by css, its a well-known gotcha in wordpress admin listings and other
sites that love to hide until you hover over a section.

I've still to look into whether the file format that is saved from selenium
ide could be used within a virtual browser like phantomjs or the like.

For direct scraping, i tend to go straight to xpath with dom.

Its far more reliable than trying to get stuff out via regular expressions,
that i only use when i need to get a certain part of the node value from a
text node.

I used to use php for that, but i wrote my own programming language that uses
a library called htmlquery (its a go library)

[https://github.com/antchfx/htmlquery](https://github.com/antchfx/htmlquery)

If i have to login, i fake each request with the correct headers. Plenty of
libraries around that help with that for every language.

------
Oldham-Made
I threw together a quick python script which scrapes listings from cinemas
local to me, along with each movie's IMDB score, and compiles the results into
a weekly Friday-morning email which I send to myself and my SO.

It's nice to know what movies are out, and when we can book, without having to
remember to check or wait for pushy emails from the cinemas themselves.

I'm planning on doing the same to email us the latest menus from our favourite
restaurants/cocktail bars so we can see when they change.

~~~
gurjeet
Would love to see this turned into a website that does recommendations based
on zip code.

------
clement_b
Probably off initial topic, but:

\- FileJuggler on Windows, to automatically clean Desktop, Download folder and
archive everything into a date-based folder structure in Drive.

\- AutoHotKey to insert text templates into everything (e.g. current date,
formatted Jira issues, annoying things to type)

The problem with automation is that one will want to break even quickly [time
saved > (time spent finding/setting/maintaining)] otherwise automation is just
a glorified delayed action. Time saving is not the only reason for automating
but I assume it's n°1's when it comes to personal use.

~~~
patrickk
Some more items that I've setup in AHK:

\- Launching commonly used apps, like Notepad, Windows Explorer or Sublime
Text

\- Launching commonly used websites in a browser, e.g. gmail

\- Paste from the clipboard into a language translation site (languageX to
languageY via URL parameters)

\- Open a to do list in Excel (hard coded file path)

\- Text expansion for personal and work email addresses

\- With one keypress, open the train schedule on the website, type in the
'from' and 'to' stations, search

\- Text expand to symbols that I always forget, eg. euro symbol, hash sign etc

\- Google/Wikipedia search clipboard text

~~~
jaclaz
Hopefully you are prolific enough in AHK to use it directly, for all other
people I suggest Lintalist:

[https://lintalist.github.io/](https://lintalist.github.io/)

which is a AHK based tool, very easy also for the "casual" user (like I am).

~~~
patrickk
Thanks for linking this, I hadn't heard of this tool before (I've been using
AHK for many years).

Another very useful tool that I recommend for automating PixelSearch and the
like is Pulover's Macro Creator:

[https://www.macrocreator.com/](https://www.macrocreator.com/)

It spits out AHK code after recording actions (e.g. a long mouse and keyboard
sequence), it's a great time saver for more complex scripts.

------
h4waii
Take a look at Huginn [0], it's very powerful with the existing agents for
taking input, parsing, and outputting data to different locations.

I use it mainly for when I can't do things in IFTTT or have sensitive things
I'd prefer to keep in a self-hosted system.

0\. [https://github.com/huginn/huginn](https://github.com/huginn/huginn)

~~~
pricci
Now that you mention IFTTT. I've recently visited their website and found out
it has a web hook "if". An http endpoint where you can GET/POST data and
trigger actions on it.

------
nickjj
Too many to list here, but one of my most useful ones is a Python script to
help pay out affiliates.

It crawls through monthly CSV files provided by my payment gateways, finds
sales and returns by a specific affiliate, calculates what I owe them and
returns a dollar amount, a sales break down for each course, how to send it
(paypal, wire, etc.) and who to send it to.

It's nice because it takes around 2 minutes to payout everyone each month with
a very high confidence level in the results. Prior to the script, it was a
stressful work flow with many chances of human error. I only did it once
before automating it.

Another script I use helps me invoice my freelance clients each month. It's a
Bash script and is open source
[https://github.com/nickjj/invoice](https://github.com/nickjj/invoice). It
saves me a ton of time every month since I can calculate how much I need to
invoice folks in a few seconds with it.

------
RocketSyntax
[https://smmry.com/about](https://smmry.com/about)

Summarizing long text-based articles! Found this on HN. Now when my family/
friends send me stuff they haven't actually read ... at least I can reply
somewhat intelligently.

~~~
PenguinCoder
I'd still love to be able to self-host this.

~~~
ajot
It's development has stalled long ago, but you could try OpenTextSummarizer
[0]. It is available on the Debian repos [1] and you can try a web interface
[2] to see if it fulfills your expectations.

[0] [https://github.com/neopunisher/Open-Text-
Summarizer/](https://github.com/neopunisher/Open-Text-Summarizer/)

[1]
[https://packages.debian.org/search?keywords=libots](https://packages.debian.org/search?keywords=libots)

[2]
[https://www.splitbrain.org/services/ots](https://www.splitbrain.org/services/ots)

------
ksahin
I have many cron jobs in AWS Lambda functions for many tasks.

I wrote a blog post about how to monitor a competitor website using Python /
Lambda and serverless framework for deployment:
[https://dzone.com/articles/monitor-your-competitors-with-
aws...](https://dzone.com/articles/monitor-your-competitors-with-aws-lambda-
and-pytho)

For lots of workflows we also use No-code/Low-code tools like Zapier /
Integromat.

~~~
slightwinder
How much does that cost?

------
ankit_it09
I used google scripts and google forms for some of my automation like:

1 - Notify me when Movie tickets are available to book. Used google scripts
that run every minute and hit a URL to check if tickets are available now.

2 - Book a class when it's available, I have joined a Gym where you have to
book a class before going, and the Zumba is very famous there the time it
opens it fills up in 30seconds and there are limited seats, so if you are late
it's gone. I wrote a script that checks and books the slot for me and then
adds the same in my calendar. I extended the same using google forms when I
want to book another class at some specific time which is currently not
available if somebody cancels it will be available. The script keeps on
checking until class start time, if available books and notify me.

3 - In my team, everyone plays Foosball and after lunch, it's everyday
discussion who will play first and with whom. I wrote a script which will
decide the matches and players in each team. I used the google app engine to
deploy it, which is still running and just by hitting the API will sort the
things for us.

There are a few other automation I did using IFTTT.

~~~
netsharc
I wonder who else is refreshing the class registration page and filling it up
within 30 minutes? Or do the special people get advance notice that
registration is about to open?

I remember reading a wild tale of how someone made a program for a used car
salesman that would check a particular car auction site to make sure they get
a good deal (I wish I could remember the details, or better yet, find the URL)

~~~
Throwawayxx001
You're looking for "Defcon 21 - How my Botnet Purchased Millions of Dollars in
Cars and Defeated the Russian Hackers"

[https://www.youtube.com/watch?v=sgz5dutPF8M](https://www.youtube.com/watch?v=sgz5dutPF8M)

~~~
netsharc
That should be it. Thanks, Deepthroat!

------
lyime
Automated investing and wealth reporting.

I use a few services to automatically invest money into a few different
financial vehicles at a regular basis

* Vanguard finds (bi-weekly transfer into various ETFS)

* Titan Invest (robo advisor bi weekly)

* IRA and 529 (monthly)

I use personal capital to connect see view my cash flow and returns across all
my accounts.

I have been doing this for the past 3 years and my returns have increased
significantly.

I am 34 and I wish I had started doing this when I was younger.

~~~
shostack
Have you figured out a way to automate exporting of data from Personal Capital
into somewhere else (spreadsheet, DB, etc)?

~~~
lyime
That’s a great question. I haven’t tried it but seems like you can access
their private API.
[https://github.com/haochi/personalcapital](https://github.com/haochi/personalcapital)

~~~
shostack
I've been building something that looks very similar to this for Mint in Ruby
but am still implementing exception handling for 2FA. I'll have to check it
out--thanks!

------
lsiebert
I use tasker on android along with a plugin to automate turning down my screen
brightness in the evenings, turning off screen rotation in a particular
orientation for specific apps (IE I prefer to text in portrait mode since I
can see what I'm replying to).

I've semiautomated setting up a new laptop using a private repo with my dot
files with some bash scripts. Started with linux, now works with Mac os x (but
won't handle linux well and I haven't gone back enough to make it worth case
by case stuff).

I still have to generate a public private key pair, and some things on macs
aren't perfect, but I have a todo list for the non automated things in the
repo. I get going less on new macs nowdays since the keyboards are such crap
(fingers crossed this year will bring better 13 inch keyboards), but everytime
I do it I add something.

------
arafsheikh
Last week I built an automated expense tracking dashboard on Tableau using
SendGrid and Google Sheets. I use a Gmail filter to forward my credit card
transaction emails to an Inbound Parser setup on SendGrid, which makes a POST
call to my home server with the content of the email. A regex then captures
the details from the POST body and pushes them to a Google Sheet.

Next I use Tableau Public to read data from Google Sheets and publish a
dashboard to Tableau Public Online. Tableau Public automatically refreshes
data on the server side from Google Sheets every few hours.

The good part is that all of this can run automatically without any manual
intervention whatsoever and I get a beautiful near real-time dashboard for my
daily expenses that I can access from anywhere (dashboard is publicly
accessible but hidden).

~~~
jesterson
Are you comfortable sharing your personal expenses with so many public
sources?

~~~
arafsheikh
Tableau Public is the one that makes me uncomfortable. If this dashboard
continues to work well and be useful I'll look into self-hosted alternatives.

~~~
j45
Metabase may be of interest as an alternative

~~~
arafsheikh
Yeah, I've used it in the past. Problem is my home server is a tiny RPi Zero
and Metabase is too heavy for it. I'll look into running it on AWS/DO if I
absolutely have to.

~~~
j45
I’ve given in and bought an energy efficient homelab server for a few hundred
off eBay. Intel makes the nuc, Lenovo has tiny thinkcentres, and HP has the
elite desk among others.

The strange thing is how many odd lxc container projects I’ve stood up just to
play with. Proxmox is sweet.

------
s4ik4t
Better Selenium alternatives would be - puppeteer, cypress, testcafe etc.

For Windows automation AutoIt is good. VBScript for MSOffice automation.
Python/shell scripts for mostly everything.

------
Cyph0n
Ansible!

I’ve been working on a “mail to cloud storage” project on the side for fun,
but also as an excuse to learn Rust.

As part of this effort, I decided to try out Ansible. Man, it is one great
tool!

For development, I have all of my server “groups” pointing to the same host,
so my Ansible playbooks install and setup everything on a single machine.
Makes testing much easier tbh.

~~~
cybrox
I'm currently migrating a system at work to using Ansible and I'm absolutely
in love with it.

My next project will be creating Ansible Playbooks for my workstation, so I
can easily re-setup my Arch workstations on Desktop, Laptop, private and at
work.

------
jaggs
A bit more basic, but I use two free desktop PC automation tools which I now
can't do without.

1\. TyperTask
([https://typertask.en.uptodown.com/windows](https://typertask.en.uptodown.com/windows))
to automate common tasks like adding different email signatures for different
addresses etc etc.

2\. ClipX ([http://bluemars.org/clipx/](http://bluemars.org/clipx/)) for
managing a huge clipboard history file. This saves me SO much time it's
unbelievable.

~~~
johnflan
It surprises me how many people don't use a clipboard manager. I use PasteBot
on the Mac.

~~~
Nannooskeeska
I've had Ditto [0] on my PC for almost a year now, and I could probably count
on two hands the number of times it's come in handy for me. Can you explain
what is so great about having a clipboard manager? I feel like I'm missing
something, and if it would become truly useful to me by using it a different
way I'd like to learn.

[0] [https://ditto-cp.sourceforge.io/](https://ditto-cp.sourceforge.io/)

~~~
gcarvalho
It depends on your workflow. I sometimes find myself alt-tabbing multiple
times to copy stuff over. Othertimes I accidentally overwrite my clipboard and
lose some text that I was writing. Being able to recover it from the history
is nice.

I use Pasteapp on macOS and FastKeys on Windows, which comes with other useful
stuff, such as a shortcut for controlling volume by using the mouse scroll
wheel on the right edge of the screen, macros, and a text-expander. The ones I
use the most are:

ddate = 2020-02-18 ttime = 03:40:26 nnow = 2020-02-18 03:40:28 @@ = my email

------
omneity
I just commented on something similar:
[https://news.ycombinator.com/item?id=22345215](https://news.ycombinator.com/item?id=22345215)

Disclaimer, it’s a project of mine, but if the websites you’re watching are
compatible that will save you some hassle trying to extract data.

Use a lot of google sheets as well, it’s super helpful to make sense and use
of the data you scrape.

You can check it out here [0], and feel free to ask if you need something.

[0]: [https://monitoro.xyz](https://monitoro.xyz)

------
TrueNomad
Autohotkey on windows is my goto tool. I don;t know much about it but I can do
the basics like move mouse to x.y coordinate and click it, or send a string of
keystrokes to a form etc.

~~~
mackrevinack
Autohotkey is such a beast. I really miss that now that I've started using
Linux. i spent nearly 4 years writing scripts to automate all sorts of things
in windows but to recreate them on linux might take me much longer since i
would have to learn maybe 6 or 7 different programs just to get the same
features. but alas Windows was doing my head in so I had to jump ship!

~~~
zo1
Autohotkey is also a tool I sorely miss since I moved to linux so many years
ago. Sadly, there is no linux-equivalent and no one seems keen on porting AHK
to linux.

------
kd5bjo
Make is an incredibly powerful tool for managing any automated process, not
just software compilation. It doesn’t know how to do any of the individual
steps itself, of course, but it lets you break down a process into indivdual
small pieces that can be automated separately.

------
rolandog
'make all' the things with (GNU)Makefile!

From pandoc pre-processing with a great tool called 'pp', to generating my CV
in different languages and paper sizes, to ensuring my repositories have the
latest .gitignore files.

I feel it requires a significant effort to setup, but once you're there, it
eases your development flow and enhances reproducible outcomes, in my opinion.

~~~
vorpalhex
I use makefile to front each project in a consistent way (whether it's my
blog, a golang project or a hardware flashing setup, `make start` will
probably work..)

------
mentos
We use Jenkins to run .bat files that package our video game, upload to Steam
and spool up Gamelift matchmaking servers on AWS every night. Before this I
did it by hand, now I wonder how I could ever have operated with that
inefficiency!

~~~
ieRei6ae
Jenkins? .bat? That's some masochism.

~~~
smolder
Can you elaborate on how that's masochistic? The game likely needs to be built
on Windows.

------
ghaff
In a somewhat similar vein to the OP, some of the details associated with
publishing a new podcast can be rather manual and error-prone. This is
especially annoying in the context of simple interview/discussion podcasts
that don't really take all that much time to edit/post.

A fairly simple Python script can automate the fiddly things like creating XML
for a new podcast and uploading things to wherever you're hosting. (I used to
use a script to add intro and outro audio too but I now do that as part of the
audio leveling process with Auphonic.)

~~~
xcubic
It looks like you could build a small SaaS around this...

------
Ayesh
I build a web site [https://visalogy.com](https://visalogy.com) with help of a
keyboard/mouse automation tool.

It was too simple to just automate and control my laptop with scrolling to
certain web pages, xpath the elements, and copy-pasta data to a CSV file.

Nowadays, I use bash scripts to automate some scraping. Cron scripts in Travis
CI are amazing, this [https://visalogy.com](https://visalogy.com) web site
practically runs by itself thanks to them.

~~~
bhy
My kid just drop by when I was looking at your site and pointed out the flag
you have for Sri Lanka seems a bit wrong - the lion in the flag is missing. :)

Anyway, great work to build such a useful site!

------
nikoksr
Greatly simplified, accelerated and automated the creation of project
directories.

I have always been annoyed by creating directories for new projects. Always
the same procedure. Always the same commands. Always the same source files. So
I wrote a small shell function which created C++ and Python projects for me.
But after a few months I started to learn golang. And there I was again.
Creating directories and files myself. But at this point the function could
hardly be extended.

So I started to transform my shell function into a powerful and expandable go
application. Learning go by starting a new project and solving a personal
problem at the same time? Perfect!

Now a few months have passed and proji has become much more powerful and
diverse. The templates it uses, which are called classes, are not bound to
languages or anything like that, you can create a class for really everything.
No matter how complex the class is, proji creates the project in seconds.
Class configurations can be imported and exported, making it easy to share
them with other users.

With the latest version of proji there is a new feature that takes proji to
the next level. Proji can copy the structures of repositories on Github and
Gitlab and import them as a class which you can use locally to create your own
projects.

Additional features: Classes support shell scripts, template files to minimize
the boilerplate code you have to write, ...

[ 1 ]: [https://github.com/nikoksr/proji](https://github.com/nikoksr/proji)

------
mast4461
I jut begun to do some partial automation of website interaction with the
extension Tampermonkey for Google Chrome to inject my own scripts on certain
websites, often creating an additional control panel UI for custom actions
such as scraping or data input. I use Tampermonkey in such a way that the code
managed by Tampermonkey are only loading scripts that load my actual scripts
from GitHub and other sources, such that I always get the latest published
versions of my scripts.

~~~
terminaljunkid
would be interested to see why you use tampermonkey instead of GreaseMonkey. I
tried both but stuck to GreaseMonkey as it was open source.

~~~
fbcpck
Not OP, but I decided to switch to Tampermonkey after GM version 4 which broke
almost all my scripts due to the GM_## api changes. I no longer remember the
exact details but it was poorly executed/communicated, and it’s easier to
switch to TM instead of updating all the scripts.

The dashboard interface in TM is a lot nicer than GM too

~~~
aepiepaey
When Firefox stopped supporting legacy extensions, the Greasmonkey developers
saw that as a chance to redesign their API (to use promises).

Tampermonkey used to be open source, but unfortunately isn't any more. I still
use it though, as it has better UX (e.g. nicer dashboard, prefills @match when
creating new scripts, better editor, etc).

An alternative to Tampermonkey that is open source and still uses the old
style of user scripts is Violentmonkey (which also lacks in the UX department,
if i recall correctly).

------
dankohn1
If you travel regularly, Tripit is a great service for organizing your flights
and hotels on to your calendar (Google Calendar also does this with Gmail but
it doesn’t work reliably, especially if you book some of your travel via your
personal email account). You can set up a free Tripit account and either
forward your receipts to it or let it look into your email for them
automatically. Add the calendar of your Tripit trips to your work Google
Calendar. These now show up in as a second calendar but are not visible to
your co-workers. Create a free Zapier account and use a Zap to automatically
copy all events from your Tripit calendar to your work calendar, which your
colleagues can see. If you want to be shown as available while traveling, you
need to change the multi-day event that Tripit creates from busy to available.
And if you change your flights, you need to manually delete the old ones. But,
people will no longer book you into meetings when you're on a plane.

Zap: [https://zapier.com/apps/google-
calendar/integrations/google-...](https://zapier.com/apps/google-
calendar/integrations/google-calendar/10039/copy-new-google-calendar-events)

------
zze
* Home Automation, like opening shutters at sunrise and closing them at sunset or turning on LED lights at night when movement is detected and turning them off after a certain amount of time since the last movement. Z-wave relay modules control most devices, and the command-center is Raspberry Pi + Aeon Z-Stick (aeotec.com/z-wave-usb-stick/), which allows for a vast amount of flexibility compared to standard Z-Wave controllers.

* an extensive collection of custom made /usr/local/bin scripts which automate things such as video recording and encoding for youtube (recordmydesktop + avconv), backups, and certain time-taking and repetitive operations on websites (like downloading invoices; through selenium-webdriver - github.com/SeleniumHQ/selenium/wiki/Ruby-Bindings)

* zaps on zapier.com for pushing data between multiple cloud services that would otherwise have to be moved by hand (i.e., Gmail email with matching subject to a task in kanbantool.com)

* autokey on Ubuntu, which allows me to type phrases like "!thx" and have them automatically expanded to i.e., "Thank you for your email!"

------
rasikjain
Written couple of scripts related to storage space in Shared Drive Folder and
Database storage. If the storage goes below certain threshold, then the
notification is sent to the group email.

Using the google rss notification (e-mail) for keywords which I am interested
in (e.g my name, stock ticker and company name etc).

I use www.hnreplies.com for alerts when someone responds to my HN comments.

------
derefr
The script I get the most personal use of is for one of the _most_ mundane
things you can imagine—automating the repetitive process of keeping everything
on my computer up to date. By "everything", I mean:

• my OS packages

• my OS-overlay packages (e.g. Homebrew, Nix)

• the versions of any standalone SDKs I use, that aren't maintained as OS
packages (e.g. Rust from rustup, the Google Cloud SDK)

• any globally-installed packages for the package ecosystem of each language I
install such stuff from (not libraries, but standalone utilities, e.g.
youtube-dl in pip, or Foreman in Rubygems)

Here's the script:
[https://gist.github.com/tsutsu/270e09c68690ec85c51dbd054e22b...](https://gist.github.com/tsutsu/270e09c68690ec85c51dbd054e22b9ef)

I think automating this might help a lot of people, because people tend to
forget that they even _can_ update a lot of this stuff, or they don't know
there's a command to get certain things to happen at all. (E.g. did you know
that `brew cask upgrade` will switch out your application-bundles installed
via casks with newer versions?)

I never polished it up, though, because it's still frail in some ways. (I
still don't exactly trust the way it does pip package updates, for example;
sometimes pip decides to upgrade packages into an inconsistent state where a
package's deps get too new for it to use.)

But the idea is that you run this _interactively_ , first thing in the morning
when you sit down, in its own fresh terminal window, right after maybe letting
your computer restart to update to a new OS version. It's like putting your
workspace in order in the morning. It's doing the grunt work, but you're still
making the decisions and fixing the exceptional conditions.

\-----

I feel like this _could_ be polished into a universal tool that'd literally
update _everything_ for _everybody_ with one well-known command.

But, better yet, the problem could be reversed, and a standard for registering
installed package ecosystems and their respective update/clean/etc. commands
could be created, that installed ecosystems could register with by placing
files in a directory (like e.g. pkg-config or bash-completion) such that this
command could outsource its smarts to the ecosystem creators themselves.

------
polishdude20
I work at a local college and was recently tasked with collecting and
aggregating instructor workload requests. We are in the planning stages of
which courses each instructor will teach in the fall 2020 semester and my
department wanted an easy way for instructors to submit their requests as well
as be able to see what courses other teachers have requested and which are
available.

Manually, if have to receive each spreadsheet or email of choices from each
instructor and then add it to an Excel spreadsheet of some sort. This wouldn't
have been live at all.

So I came up with a system.

I made an excel proposal template each instructor fills out with their name,
school ID and the courses they want to teach as well as the hours they'd have
available for each course. They then send it to my email which has a rule on
it to send excel attachments to a Gmail account I own. This is because I can't
access the API of outlook so I need to get outlook to send this stuff to my
Gmail account. My Gmail account has a watch on it so whenever an email comes
in to it, I get a push request on my server. My server reads the contents of
the excel file and sends it to a database.

From the user side, if a teacher wants to access the full list of what courses
each instructor has chosen, I've made a website in react with an excel like
row and column layout where the columns are each teacher and the rows are the
courses. Where the two meet we have the hours the teacher wants to teach for
that course. When this page is visited, it pulls from my database and
populates the whole thing with the latest data. The site can also export all
the data to an Excel file to be implemented once all the teachers have made
their choices.

I learned a lot with this project. Learned react, setting up https, basic
authentication, postgres, running a server and routing, and a while lotta
other stuff. Super valuable to me in my learning even if I could have spent
less time doing it manually!

------
simcop2387
I've started playing with some things with node-red + open hab to control my
lights, but more in line with what you're talking about I've also played with
Huginn for automating some small daily actions and webscraping. So far it's
been going well but I'm not doing anything terribly difficult yet.

------
caduar
Hey, I recommend you RPA (Robotics Process Automation), there some open source
option such as Open RPA or community options as UiPath. Actually I work as an
RPA developer and you can automate whatever you want. For example read an
email, capture that information in an Excel and fill web forms based on the
excel.

------
faeyanpiraat
Be careful, my antivirus blocked the site when I visited the "Privacy" page
from the top menu.

[https://www.virusradar.com/en/HTML_ScrInject.B/description](https://www.virusradar.com/en/HTML_ScrInject.B/description)

------
msadowski
I run a robotics themed weekly newsletter as a side project.

At one point I decided that it would be nice to add header images to every
issue. I decided to do it through a quick and dirty python script that adds
the text for me.

Another thing I've just recently automated is the python script calling
mailchimp API to show me the most popular link together with unique opens. I'm
currently planning to embedd this info in every issue.

The next step that I'd like to automate is the campaign creation. With the API
I don't suppose this will be a huge deal, just need to put some time on this.
When I have it then I'd like to create a commit hook that will do all of these
things automatically after pushing changes to the repo.

------
loufe
In the engineering office (non-software) I work in, I've done wonders with
classic scripting for me and my colleagues. Some Excel VBA, lots of
Powershell, AutoLisp, etc. Doesn't need to be shiny to be effective.

------
theshadowmonkey
What about headless chrome and some cron jobs to run tasks for you. It'd be
similar to selenium. But, on a headless browser. There are also some scraping
services that let you get data on a set interval with a phantomjs scraper. I
have one that uses apify and then notifies a zapier hook on success and then
zapier posts to my google sheet. My webapp then calls a GCP instance REST API
endpoint which internally calls the google spreadsheet through an exposed api,
cleans the data and sends back to the webapp.

------
RMPR
This one is a shameless plug (I created the app) while it's still in early
stages of development, the core features are working, I use
[https://github.com/rmpr/atbswp](https://github.com/rmpr/atbswp) to automate
demos, whenever I have to present a demo in front of people, I just record it
before and replay, but the app can have many use cases beside this. You're
only limited by your imagination.

------
sufyanadam
I use a collection of tools from this repo to automate mundane developer
activities such as posting release notes slack messages containing linked
pivotal or Jira story titles:
[https://github.com/sufyanadam/pivotoolz](https://github.com/sufyanadam/pivotoolz)

There's a bunch of stuff in there that saves you seconds each time you have to
work with a user story or cut a branch or merge etc... Saved me a bunch of
time over the years!

------
boomeranked
Airbnb experiences will not add an API, so if you run any kind of tours, you
have to go the App to add/remove/change dates/prices/starting points.

Selenium takes care of it now.

~~~
spectramax
Everytime I use Selenium, even for legitimate use cases, I feel dirty deep
down under. My soul writhens and I want to do something else in life other
than writing Selenium automation - perhaps chop some wood.

------
steveeast112
I've used IFTTT a bit, along with Zapier. Biggest issue is trying to do
multiple actions when you need to automate tasks for multiple rows of data at
a time.

~~~
paulmendoza
Look at Integromat. I used to use Zapier but have moved mostly over to this
instead.

------
jsonbourne
Usually, my download folder is quite messy. I wrote a script to re-organize
any such folder using a simple CLI. It also has a config that can be updated
to account for different file types or categories.

[https://github.com/functioncall/neat](https://github.com/functioncall/neat)

------
paultopia
slightly mundane, but I'm an academic and my cv needs updating a _lot_. I
finally got latex to work on netlify, and now all I have to do is update a
yaml and push a commit and my cv is updated.*

* At least on one of the three locations my cv lives... the next step is to point all the other domains at my new CI/CD home...

------
roarkry
I manage two calendars for work and often have to make sure my calendar events
exist on both so I don't get double booked. I wrote
[https://syncmycalendars.com/](https://syncmycalendars.com/) to automate and
simplify that process for myself and others.

------
staticautomatic
I started with selenium and still use it when I have to but try to just make
http requests and parse responses directly whenever possible. Probably my
favorite travel-related script was one that watched for an opening at a hard
to book hotel and snatched it up when it became available.

~~~
beatthatflight
and to parse the responses (Assuming they're HTML page sources, not json/xml)
- you using scrapy or something else?

~~~
staticautomatic
I pretty much always use lxml. It has an html method but I typically just use
etree.parse(). It's very fast and has excellent XPath query support.

------
batirch
I use telegram, ifttt, Google sheets to calculate expenses.

At my last job, I used Zapier & Zoho CRM to automate creating leads from email
requests which saved me hours of time.

~~~
wastedhours
Was at a startup that was super low-overhead (and as dysfunctional as a 4
person team could be), we used Zapier to automatically create Gmail drafts as
replies to new lead enquiries on the website and follow-up responses as
needed. We could then spice up the response if needed, or just bulk run
through and hit "Send".

Appears personal to all email systems and spam filters, but actually was semi-
automatic marketing automation.

------
Havoc
Maybe check out huggin

------
peteretep
Unfucking the many many ways that banks screw up exporting financial
transactions into well-formed OFX files that MoneyDance can consume

------
pravins
Have you tried [https://pipedream.com](https://pipedream.com)? You can run
Node.js (with any npm package) on webhook/http requests, emails and schedules
for free. It also manages auth (including oauth token generation and refresh)
for popular apps like Google Sheets, Discord, Slack and Airtable -- just use
the auths object to reference tokens and keys in code.

~~~
unixhero
Looks very promising! However the pricing is opaque. What will this cost me,
if my business becomes successful and hit their free tier rate limits?

[https://docs.pipedream.com/pricing/](https://docs.pipedream.com/pricing/)

~~~
dylburger
Hi, I'm a co-founder and engineer at Pipedream.

Paid tiers are coming soon, and if you hit our daily compute limit before
that, we're happy to raise your limit to make sure your workflows run without
issue.

Let me know if you have any other questions! Feel free to reach out to me
directly at dylan [at] pipedream [dot] com.

------
buboard
auto delete certain messages from gmail after 30 days . i use filters to tag
messages as "autodelete" and create a google script that runs periodically to
delete them . It shouldn't be so hard to do something so simple.

~~~
Klathmon
I'm fairly certain this is how the "trash" in gmail works by default.

Anything that gets sent to there stays there for 30 days then is automatically
deleted.

~~~
buboard
i mean, for my own reminders and messages that i forget to delete.

