f-log

Very quickly, I wanted to note how the SpamDater Python code got to GitHub.

I had discussed Git a few years back.

May2013.html#030520132124
May2013.html#030520132150
May2015.html#p160520151101

But I could never work out how to take an existing project and add it to GitHub.

Step one; Create local git repo
git init
git add -A
git commit -m "first commit"

Step two; Create new repo on GitHub and do not create any auto files.
Step three; Tell GitHub about the new code.
git remote add origin https://github.com/robgithub/spamdater.git
git push -u origin master

easy peasy!

Really should do it more often ;)

There was a massive data breach a couple of weeks ago and I decided to do some analysis on the spam I was getting.

I tell my ISP to not filter spam for me so SpamAssassin does all the work, shuffling it off into a special folder.

Most of the emails start with "Your account has been hacked" and "I wrote you this email from your account.". There is some variation, but not much and the FROM and TO addresses are the same.

As anyone who has dealt with emails on a technical level will tell you, it is very easy to spoof the FROM address. The protocol is designed to facilitate it. e.g. Company marketing Droid 1 sends you and 1000 others a spam/not-spam email, they will set the FROM address to be the company not themselves.

I had been seeing a lot of spam over the last few years degrading my email addresses. Dropping the first character, doubling the last, general bit rot. So I decided to do some investigations.

Started with bash script
for F in *; do egrep "^(Date|Envelope-To):" $F | sed -re 's/(.+)[^0-9]:(.+)/"\2"/' | tr -d '\n' | sed -re 's/""/","/' | sed -re "s/$/\n/" >> stats.lst; done

Which did a good try but missed lots of messy emails, so I wrote a Python script.

This started off very simple but more and more complex until I felt it needed it's own GitHub repo, SpamDater.

The idea was to get the email addresses and the dates the emails were sent. So, it had to collect all the unique email addresses to my domain, learned about Python sets and also the likely date the email was sent.

Dates are never easy but with a little regex magic and Python logic I was able to extract a reasonable date to pair with the email addresses.

and then...

Well it was a bit boring. The Massive data breach turned out to be more of a web site scraped email address list and the emails appeared to be minus bit-rot.

From 2011 to date, I was basing my research on 54849 emails (marked as spam)
of which my SpamDater found 63331 emails
of which 1735 were unique.

One had a date of 1899 and a few hundred were just RANDOM-CHARACTERS@ my domain.

Did have fun writing the Python though;
:easy to understand command line arg parsing
:uses sets for uniqueness
:regex search and findall
:file reading line by line
:file writing line by line
:writes to stdout when no file requested.

We went to see Harry Potter and the Cursed Child Parts 1 and 2, which is a short six hour play continuing on from the last Harry Potter book/film.

The tickets had been booked a year in advance and to get special pricing the family was split up. I was in the Balcony at the top of the theatre.



I found this view a little un-nerving. While the house lights were up I felt serious vertigo. Mainly because of #1 in the photo above. The ceiling was below my eye line and really emphasised the drop.

However, once the house lights dropped I had no problem. There were opera glass to hire, but my eyesight was good enough.

The levels #2, #3, #4, #5 were all below us and gives you a taste for how many visitors were there (it was packed) and how high we were.

#6 is the visible part of the stage, ignoring the bars and peoples heads. #7 is the bit of the stage completely out of view, that was gratefully used only a couple times and I do not think I missed anything.

The main tag line for the play is "Keep the secrets" and I knew nothing going in and I am happy to #keepthesecrets to aid future visitors.

It was very very good and the programme had details from the books/films that visitors should be aware of to help them understand what was going on and who was who.

The six hours was split into two performances each with a 20 minute interval and each section was so involving that time whizzed by.

Very very worth while going (even being stuck at the top of the theatre)!

Then I made the fatal mistake of trying to work out what was going on.

I ran
/opt/vc/bin/edidparser sv-3.edid.dat
/opt/vc/bin/edidparser sv-3.edid.dat


and got
HDMI:EDID found monitor name descriptor tag 0xfc
HDMI:EDID monitor name is Acer_AL1916W

Which is the exact VGA monitor I am using!

Where did it get this information and, more importantly when will it lose it?

In the last post I laid out the basics of my plan, process and issue. This post is going to get into all the technical details.

So, plugging in the SV-3 to the Raspberry Pi either pre or post boot did not work.

Plugging in the Raspberry Pi pre or post boot to a VGA monitor did work.

Unplugging the working VGA monitor and then plugging in the SV-3 did work.

Because of the last one, I know the settings are correct and the hardware was all compatible and working.

The settings I tried in config.txt

Safe mode: this overrides a number of settings and should give us the best chance.

# uncomment if you get no picture on HDMI for a default "safe" mode
hdmi_safe=1


SV-3: No signal
Monitor: Signal
SV-3: Signal

---STOP THE PRESSES---

As I was writing this up, I added a comment that the same result was obtained when cold booting the Pi rather that resetting it. But I had not done that, so I set it the scenario up expecting the same failure and ... it only worked.

So, all that palaver, including a massive posting to stackoverflow was a bit of a waste.

Got to think how I can use it now :)


For reference I previously used the following tools to try and resolve and diagnose the previous issues.


/opt/vc/bin/tvservice -m CEA

/opt/vc/bin/tvservice -s

/opt/vc/bin/tvservice -d edid.dat

/opt/vc/bin/edidparser edid.dat

get-edid

xrandr -q -d :0


Then these are the settings I tried in the config.txt


hdmi_ignore_edid=0xa5000080

hdmi_edid_file=1    #file added to /boot/edid.dat

hdmi_group=1    #CEA
hdmi_mode=1     #VGA 640x480
hdmi_mode=2     #480p
hdmi_mode=3     #480p H


hdmi_group=2    #DMT
hdmi_mode=4     #640x480 60 Hz


Back near the turn of the century, before kids, I purchased a rather unique item. A SV-3 Instrument Viewer.
I even created a couple of pages for it, this was before the b/flog had tags.

The SV-3 Instrument Viewer is a VGA 640x480 monitor embedded in a pair of glasses with a mirror to view the results through a single eye (can be moved from right to left). Think Google Glass but without the computer or the camera.



It has a rather strange particularity in that it is 6bit colour. If you try and give it 8bit colour the colours in the extra two bits are a bit scrambled and I am sure they used to flash, angrily.

I did not do much with it, but always had great aspirations. I did create a Javascript based Japanese app that showed different symbols in black and white in a random slideshow. The idea was to learn subliminally and I had a spare port on my Linux setup. X was more than happy to have an extra monitor that was tiny and I has scripts that used the standard X geometry parameters to locate it correctly on the extra screen.

To be fair, I had packed it away and had forgotten all about it.

So I came across it in a spring clean attempt and decided that a tiny Raspberry Pi would be ideal to make it portable, my number one bug-bear with the SV-3.

Plan

1. get a Pi Zero W(ireless)
2. install new Raspbian on SD
3. attach adaptor to VGA
4. tell the Pi to run in 640x480
5. Profit!!

The first one was easy as I am not doing anything with my Pi Super-computer.
Downloading and installing Raspbian is straight forward.
VGA adaptor ... The Pi Hut has a nice HDMI->VGA adaptor, but the Pi Zero has a Micro HDMI. So I end up with a rather ugly Pi->dongle->dongle->VGA cable setup.
I expected to be telling X about the resolution, but the Pi handles that in the config.txt file.

and ...

no signal

try all the settings and read lots of very old posts.

no signal

Now I am convinced the SV-3 is broken, so I grab an old VGA monitor and plug it in.

Works perfectly...

Just before I pack everything away, I just plug the VGA cable back into the SV-3...

And it WORKS! The standard Pi background looks perfect no flickering or anything.

technical details next time.

Back in September 2018 I came across Blender Appreciation Poster linked from https://www.blendernation.com/2018/09/18/blender-appreciation-poster/ and now the latest version includes my image!



can you guess which is mine?

That said, the project organiser is not too motivated to take the idea any further.

I got a new Google Pixel 3 phone, it is not Blue but it was free (for work).

There is not much to say about the changes from the Pixel Blue original. The transfer by cable was good and I was able to update credentials or all the apps it automatically installed/copied.

The one I was dreading was the Pebble app controlling my Pebble Time watch. No watch faces, not even the ones that were already installed on the watch :(

But I knew there was a community that had been working to allow the Pebble to continue now Pebble now more/is FitBit.

http://rebble.io

The website is not the most user friendly but if you dig through there is a link to https://boot.rebble.io/ and after you have logged in the on a browser on your phone with the Pebble App it all starts working by magic!

Well, not quite that simple, but the instructions explain how you may need to shut the Pebble App and re-open it and then Magic!

Then all the Apps and watch faces just work. There are so many different ideas/designs and they are nearly all free.

There are also lots of new Apps and watch faces that have been added in the last couple of years.

Amazing work and I will be able to enjoy my watch for years to come.

Skydiving (tunnel) [CHECK]
Surfing (with un-aided standing in wave-rider) [CHECK]
Jumping from 125ft tower [FAIL]

Skydiving was a lot of fun and I found it surprisingly easy to float unsupported.

I had done surfing at the Wave-Rider before, but never managed so much as a single second standing on the board un-aided. This time I found it quite simple, but I could do no more than standing.

I got into all the climbing gear and jump harness and climbed all the tiny maintenance ladders and got clipped into the rapid decent mechanism ... and then I chickened out and had to climb all the way down.
The climb itself was quite an experience and the view at over 125ft was amazing, so I consider that a win (small win).

And that was my Christmas present.

The voucher I was given even had the price and at around £20 per activity it was a very good deal. Staff were particularly helpful through out the booking and then very friendly for the activities.

Recommended.

http://www.twinwoodsadventure.com/


not worried about facial recognition getting anything from that ;)