The immigration website of Saskatchewan province opens up randomly to apply for immigration. I missed it many times because there is no indication other than the "Apply" button being enabled and a small text in their homepage which says "Applications are now 'open' ". They will close the application intake when they have reached X number of applicants. So timing is very important.
So I hacked up a script which diffs their home page every 10 minutes for "open" regex. When there is an "open" keyword in the diff, the python script calls twilio API to make a phone call to me along with an SMS.
So this script was running in AWS for many weeks and one day I got the call. Logged in to Saskatchewan's immigration homepage and applied. Now I am in Canada as a permanent resident. Thanks to Twilio.
edit: add H1B to make clear which type of immigration is broken IMO.
(This comment is from my previous HN account, for which I lost my pwd.)
Here's a Postman collection of all possible options with the API https://app.getpostman.com/run-collection/620bbe44b6ecc6c2e3...
I used it to notify me of new Hololens events, listed on this page https://www.microsoftevents.com/profile/web/index.cfm?PKwebI...
Just set it to check for the presence of a word on the page, etc.
I was hired to scrape some economic related pages and build an excel file and email it, I got that done but not sure if I should try to host this and turn it into a service or just set it up for the client and let them deal with it. It's just personal use on their part.
The problematic part is when you scrape data off of websites, and the owners don't want you to do it; as in, they would not even be happy if you manually copied that stuff into an excel file for fun or profit.
So for me I'll set it up for them and them run it.
How did Google deal with it, when they started their search engine business?
I kinda felt the same way about how Netflix started out but I think they probably had a deal to pay a portion of revenue from dvd's that they rented out.
I can’t see anything else with that name. All the references in this thread are great tips, so I’d like to track this one down too.
London Meetup fill up within minutes that many data science ones allot tickets “randomly” via lottery.
Surely this is a market opportunity?
No need to write code, you only need a shell account on a box somewhere. Put a line like this in your crontab:
20 9,12,15,18,21 * * * curl -s http://example.com/special-deal-page | grep 'something that needs to appear on the page' | mailx -E -s "it's happening!1!" $MAILTO
Not sure why you choose to do diff? You could have just searched the dom for the existence of 'Apply' button? Am i missing something?
Still that aside, great ‘hack’ and congrats on getting your Canadian residency!
Its broken because it was really intended for exceptionally skilled workers. But currently its a lottery. USCIS just takes the word of the company applying for it and grants the visa. This loophole is being exploited by numerous body shops. That is reason why there are low quality H1B workers. There are various other problems like employee mobility, spouses of H1B is not free to work in the land of the free. Country of birth based quotas for permanent residency, rather than skills or points based quotas.
Saddle the lawyers, we ride at dawn!
Inspect the page,find the structure, isolate or if it is easier to just dump the entire plaintext and use strpos to find a specific word.
> I did, ...Had one sale though
Yeah, was about to say, it's too small of a niche. Plus the risk that sooner or later, Saskatchewan would update their site to provide the same feature.
Even allow people to create presets that would then be added as a generic thing. You could use it for camping all sorts of intermittent online things (web site specials, online ticket pre-orders, etc.)
First I thought to ring a loud alarm connected with a Pi, but then it will be limited to my apartment. So I went with twilio to give me a phone call. Because I should act immediately and time was of the essence. Emails or SMS might go unnoticed.
There are a lot of Software Engineering/Development jobs in Toronto or bigger cities so people in those fields wont leave these bigger cities.
Due to the volume of the pot, it would be difficult to get a true moisture reading, so I place the sensor directly under the spout in order to have noticeable changes.
which can use a number of different sensor makes, but http://www.decagon.com/en/soils/volumetric-water-content-sen... seem pretty good
Please take care of my plant | https://news.ycombinator.com/item?id=15453974 (100+ comments)
And the link: http://www.pleasetakecareofmyplant.com/
If you have any more questions feel free to post them in the sub and I can explain more!
How it functions is a daily yes/no vote that triggers a watering event for that day.
Edit: Oh, but to I think the real question -- No. It's not my plant anymore as far as I'm concerned. The community can care for it as they see fit. I won't interfere, except to add features, or put up seasonal decorations :)
2. She also wouldn't answer the door when during kids pick-up. The same app texted her when I was outside.
3. To disprove her allegations I wasn't involved in the kids school and activities I used Android's Locale app to trigger geofenced log entries.
4. Python and Matplotlib came in quite handy automating timeline generation from PDF docs. Never got that one quite perfect though.
I'm not sure I could have prevailed in getting more time with my kids without the time savings automation gave me. I've seen other fathers stopped cold in Court who were less prepared.
Considering that close to a MILLION Americans get divorced every year, at a 50% rate i.e. half of marriages are ending up in divorce in the US.
May you and your children be blessed.
Still, that doesn't prove that the individual points weren't fabricated at the time.
There don't seem to be any criminal charges here.
Still a lie, even in the blockchain
One night she stayed over, set the phone down, got the text, and thought it was hilarious and said she did think it was weird that I said the exact same thing every night.
The next day I updated it to pick from one of 10 different text messages!
It took about a month or two for her to notice, though I believe that what ultimately did me in was that time when I was talking with her on the phone when the SMS task executed...
I just assume you have to rent a number/sms service or put a usb cable into your phone to use your existing phone.
Would be cool to interface with it without usb.
Anyway that is pretty cool what it does.
Similarly, I have a self-hosted instance of Tiny Tiny RSS set up with an array of custom scraping plugins to pull all the web comics I follow into one feed, which I consume with the Android client. I'd push this through my Kindle delivery system, but then I'd be stuck reading black-and-white versions of color comics.
Along the same lines, there are a few YouTube channels I subscribe to whose content can be enjoyed nearly as well in audio-only form. As a university student, I do a lot of walking most days to get from place to place, and I fill that time listening to audio content. The same server which runs my news- and comic-gathering systems also watches those YouTube channels, pulls down new videos, converts them to audio, and publishes the results as podcast feeds which I can subscribe to through Pocket Casts on my phone.
I wonder how effective it is though these days, I'd say 70% of my RSS feeds are either truncated forms of a full article (with a 'click here to continue reading!' link), or just summaries.
Is anyone aware of any repositories where the customization required to obtain the important content is maintained by a community?
If you are interested in working together please reach me out!
You can check what we did here:
SciShow (and the sub-channels: SciShow Space and SciShow Psych)
PBS Idea Channel (no new videos, but old content is good)
I'm also developing something similar to your news scraper but with different end goals. I'd love to talk with you about your approaches in that domain as well.
My Python script reads my Google Contacts csv extract to identify relevant people, the lxml library parses my mobile phone call and sms logs, the mailbox module reads my email inbox and Sent Items to extract relevant messages, PIL resizes attached images. Then I use docx to reassemble the results chronologically into a Word document suitable for submitting in court proceedings.
The resulting Word document is an intimidating 130 pages long. I have shown my builder enough excerpts for him to accept liability for most defects. I won a County Court judgment against him last week. Now for the financial settlement!
I thought about putting my script up on github, though haven't had time to scrub some personal information from the source code...
I know of a homeowner that informed his contractor(s) that he blogs daily about the progress of his renovation. The fear alone was sufficient to ensure that his project went smoothly. Plus if the contractors do a good job then they get free PR.
If you could integrate that into an application (Google Contacts integrated message history with Email/SMS/Phone/GV message history) that I could run on my own device, I would pay money for that. I would only be interested if I could run it on my own though, eg specifically not as a third party service; privacy issues.
It's actually one of the first apps I always install after I discovered the hard way that Android used to delete corrupted SQLite databases and I lost the SMS DB on my phone.
The new Firefox is actually faster than Chrome, on my PCs. Plus of course the mobile version actually supports adblockers. I'm switching back to Firefox, personally.
i won't spam you, promise. will notify everyone there about updates like the firefox port
I tried but can't (same thoughts as you, without verifying it myself, not gonna sign in...)
Hitting F12 or Ctrl+Shift+C on the popup page do nothing.
EDIT: chrome on linux Version 59.0.3071.86 (Official Build) (64-bit)
getting signed out happens for a variety of reasons, probably unrelated to the extension itself. ill keep an eye out though to see if there is a systematic reason
thks for the feedback mate!
eg. 拓展 expand
I guess you will want to keep using your homegrown program, but maybe this can give you some inspiration for features to implement.
1. I have a script to automatically buy small amount of BTC every day
2. For the more knowledge-dense books I read, I write summaries of them (https://piszek.com/books/). I have a script that puts a random book review to my pocket for reviewing every week
3. I have an instagram account of lego minifig (https://www.instagram.com/le.traveller/). I wrote a script that likes other profiles to get traffic
4. I have a script that parses my bank e-mail statements to fill up my spending spraedsheets
5. I also have a script that parses incoming email for invoices. That system basically does my taxes
6. My GTD methodology revolves around Evernote. I have cron jobs that throw me "checklists" with stuff to do around certain times (yearly taxes, etc)
7. Using Twilio and verified number, my calendar sends personalized SMS with birthday wishes to my family that appear as they are from my number
8. Also on Twilio I have DIY voicemail that is aware of where in world am I and either routes to my current SIM card or takes a message. I also have a US number that routes any SMS to my current SIM
I recently moved, so I have to rebuild all my Smart-Home hacks. I am currently trying to automate my Intercom at home to play pre-recorded message to postman and let him in automatically
Source: doesn’t work for me.
This one gets bonus points for being MIT-licensed:
To the best of my knowledge it is not possible to send SMS messages in an automated fashion on iPhone, you might as well use calendar reminders.
Anyway, yeah I've got that problem right now wrote code for a specific stack need to just drop it into a server that's not mine/easy for client to use.
I'm kind of curious about #1 regardless of the price you buy? What if it was like that day the ICO was banned and it dropped like $800 or whatever.
I'm looking to implement a GTD-like system myself, and any automation is helpful.
I started an escape room business for fun, and have been enjoying the heck out of writing custom software for the rooms. Both in-room and for administration.
It does full window management and all the regular UI stuff so it feels super normal to the players. My favorite part is that I don't use jQuery. That was just a little challenge I created for myself for fun.
Now my rooms have a login prompt of any kind, and I can then create windows with any kind of HTML/JS/CSS content for solving puzzles, extra clues, etc. I can use an RPi and control maglocks from the computer, light up LED's, or even communicate with wireless props.
I run a kiosk add-on for the browser and physically hack the keyboards so certain keys don't even work in the rooms. No F11 or Ctrl-Alt-Del without access to the netbooks or RPi's locked away in hidden compartments.
I also created a lot of room management software for timers, sending hints into the rooms on tablets, etc.
I wonder if there's a market for re-selling experiences to other locations.
I captured video input with a simple python+QT script and emitted a button-press whenever the screen flashed. The best part was that the script didn't interfere with my controller - I could run around the area opening chests and battling random encounters, all the while getting closer to the 100-contiguous-dodges prize!
Sure - a memory editor would have had the same effect in 5% of the time - but this was _way_ more rewarding.
Turns out the low-tech solution of using an insanely fast car (Escudo), taping down the controller's gas button and just letting it do laps while grazing the wall worked. Some time later I looked back and had won.
This made your character run into a wall, but keep running on the spot, thus levelling up your "athletics" stat.
On PC, you could use a keyboard script to repeatedly press the dash key every 500ms and steer with the mouse. Something that was pretty difficult with a controller.
When the laundry basket gets full past the painter tape line, I do a load.
I load the dishwasher after dinner, and unload it in the morning regardless of the volume. Got a roomba that runs every other day and cries when it's full of dirt.
I have NFC stickers that link to start webapp timers in my smart phones browser (just a url to googles timer web app, pre populated with time, I get push notifications and all that) tea brewing timer, Laundry+ Dryer, cooking, all cheap NFC sticker+ webapp
I also wrote a bot that alerts me when 3D printer deals from trusted retailers are going on. I just bought two for $160 each, with free shipping!
(Or are you talking about separating things like sheets, towels, etc?)
I wear a lot of white sports socks. All of them are pure white after years of having them.
So I write those links to the NFC stickers (click one on mobile)
Write on them with a sharpie,
When I go to use it, I tap my smartphone(stock android) and it automatically launches the browser to the webapp
You have to get around the DRM, and the software it uses to make 3d print file from an STL runs on Windows or MacOS, closed source, no Linux support. I run Windows in a VM for the slicer.
There's DRM on the filament and it's PLA only, but you can buy the key to unlock and rewrite the DRM NFC chip. I use an app on my phone to rewrite the values
I call it DRM because you can't edit it without a reverse engineered NFC key.
Another side to the DRM thing is that if you use inferior input material and the printer has a quality issue, customers are going to blame the printer and not the filament.
Value added DRM I guess
These printers are nice to prototype with before sending it to a more costly and less time investment forgiving production printer, or quick around the house jobs/play.
Feel free to reach out if you want help getting started: my username @gmail.com
- User pokes immigration website repeatedly to submit his application as soon as the site starts accepting. Obviously if everyone did the same thing, no one would benefit. The thing that needs fixing is an immigration system based (partly on) quotas and first come, first serve.
- User automates sending amorous messages to his girlfriend, so long as she doesn't know they're automated. If enough people did it, girlfriends would eventually find out and the desired effect would be lost (or worse, they'd feel tricked).
- User automates saving of all store coupons to his loyalty card without ever looking at the coupon. This defeats the idea of coupons (to encourage you to try something you would not have otherwise bought). If everyone did it, coupons would cease to exist.
- User automates getting into desired university class by hammering the registration site repeatedly. Needless to say, if everyone started doing the same thing, no one would benefit.
- User automated complaining to the water utility about a problem in front of his house. Once again, it might work for one person, but becomes completely ineffective if everyone does it for problems in front of their houses.
- User automated late delivery complaints to post office to get compensation. If his script becomes too widely used, the monopoly post office will simply raise prices or stop offering compensation.
That sounds like success to me, though not everyone will agree.
> - User automates getting into desired university class by hammering the registration site repeatedly. Needless to say, if everyone started doing the same thing, no one would benefit.
This was circumventing a bad process; people would benefit if everyone did it, because it would demonstrate a need for a better process.
> - User automated complaining to the water utility about a problem in front of his house. Once again, it might work for one person, but becomes completely ineffective if everyone does it for problems in front of their houses.
If everyone does it then perhaps they’ll fix things—or admit defeat and stop giving people false hope by allowing them to report such things when they have no intention of doing anything about it.
> - User automated late delivery complaints to post office to get compensation. If his script becomes too widely used, the monopoly post office will simply raise prices or stop offering compensation.
Or, just maybe, fix it. One can always hope.
Not having coupons/loyalty cards is a good thing. Hammering the immigration website, hammering the university website etc might end up improving the process (hopefully?) that is also a good thing.
In a larger context, how is this different from using one's good looks, parents' connections etc to get an edge over others?
That is true, but I don't think anyone is claiming to make the world better. The question was more how individuals improve their situation, gaining a competitive edge if you will. That will always come at the cost of others, but in this case the hacks are really things almost anyone can do, so it's hard to object on moral grounds.
I think what is being talked about is something along the lines of if everyone became a minimalist then most people would be out of a job.
Except for mine. it’s a win / win when people hang out together