2 servers, 1 cup

Actually its 1 IP-adress, but 2G1C jokes never get old!

Issue: I, as most others, only have one public IP.  I also got plans for several services I need to be able to connect to from outside my home and network. As mentioned in former posts, I’ll be running multiple virtual machines.

Solution: Reverse proxy. Yep, that is a expression I learned about a couple days ago.

Simple topology of a reverse proxy

After some intense googling (there seems to be absolutely no complete guides for this, unless you know exactly what to look for), I got it working. I used an old Raspberry Pi 2 Model B I had lying around. Its only purpose in life is pretending to be a real server…

Step 1 – Install Linux

Since I’m using a Raspberry Pi I installed raspbian, because its Debian based and I like Debian.

Step 2 – Install updates and Apache web server

Step 3 – Enable proxy and configuration

This will enable the proxy-module. Next up is configuring the config file.

I used this config:

The just restart the apache service.

 

This makes the Raspberry Proxy a landing site web-server. I directed a few subdomains to my public IP, and whenever I use one that is NOT defined in the config I get the proxy web site. Perfect!

Getting ownCloud up and running

Hello and happy new year. Things have been slow lately due to christmas and other things, but I hope to get more time working on various projects in the future.

Installing ownCloud

Next up on the web server agenda is getting the ownCloud up and running. In my last attempt I managed to break the whole server, but luckily it was easy to revert back to a working version due to VirtualBox. I also found out I was installing it the hard way, by manual installation. The actual preferred installation is a semi automatic one listed here. I still have to do the security stuff that broke it the last time, but at least the installation itself is a bit quicker.

This will install ownCloud to the /var/www/ directory. The next step is to make a user and give ownCloud access to the database.

Setting up the admin user

And the ownCloud web site is once again up and running! Uploading files works smoothly. This is a good place to create a snapshot with VirtualBox. That way, if (read: when) I mess up again I can go back to where it works.

Securing ownCloud

This is where everything went wrong the last time. The goal is to secure the installation to prevent unauthorized access. As this server will be visible from the internet, which is needed for syncing outside my home, I need to be sure it is as secure as possible. The ownCloud installation manual got all the steps needed, now I just have to execute them without breaking anything.

I made the bash script as shown here, and called the script secure.sh. I changed permissions on the script, to make sure I could run it. This is where something went bad last time.

The last command shows who got the rights for this file. For me it listed the root user. Good enough I guess, since its my server I like to think of myself as root… The I ran the script.

Nothing happened of course. This, I think, is a good thing. Last time I remember getting a lot of weird warnings and errors due to permissions. So no news is good news. When I try to go into the ownCloud directory I get “permission denied”. This is also a good thing, as the script makes the “www-data”-user the only user allowed to access the files. Testing the web site from a browser seems ok, and I can delete and upload files without any errors.

Testing the system

After installing and securing the ownCloud service there is some testing needed. This is supposed to replace Dropbox, so I need to be sure everything works. I do not want to loose 500 pictures of my kid just because I messed up something. But the installation looks good, and it might be time to try it on the actual server, download the app and test it out in parallel with Dropbox for a while. There is probably some more settings I need to change for ownCloud, make a few scripts so I can easily update the install, this will be done at a later time. Right now I’m just happy it works.

 

VirtualBox server simulation

Instead of going straight to installing Linux on a server I want to do some test runs, just to see if everything actually works. I am not experienced enough to make any real security checks, but will try some guides I have found online for securing the server. On a virtual server I can mess up as much as I want, and just recover from an earlier snapshot if I want to revert back to where it worked.

Simple flowchart to show the process of using a virtual machine to test and troubleshoot an installation.
Simple flowchart to show the process of using a virtual machine to test and troubleshoot an installation.

Continue reading VirtualBox server simulation

Preparing for the Linux server

As mentioned earlier I am going to set up a Linux server for running different services, like the ownCloud “getting rid of Dropbox” experiment and other things. I have some experience using Linux, but usually end up doing stupid stuff that I cant find my way out of. This time however I plan to be a bit more systematic about it, and documenting the changes I make to make troubleshooting a bit easier.

Choosing Linux

Choosing between the multitude of Linux flavours can be hard, as it is a few to choose between. I have tried a few different ones, and for me the biggest differences lies in the GUI (which I wont use) and how stuff is installed.

The server of choice will be Debian. No particular reason really, but I like the package manager APT. I also really dont like Ubuntu, as I felt it was full of bloatware last I tried it. So a command line, only whats needed,  Debian server is what I trust to take care of my web server.

understandingdebian

Choosing the web server

Up until just a few days ago I thought apache/php/mysql was the way to go when you wanted to host a web server. After googling around for read-ups on security settings and other information around web servers I discovered NGINX. Its supposedly a lot faster than apache, and more future proof. All the facts and numbers they present to me had me convinced that this was the place to go. The ownCloud installation manual recommended apache, but in the same breath they write about requirements for up to 150 users. My installation will have 2 users, so I dont think I will notice much of a speed difference. Actually it would probably not matter at all if I went apache or NGINX for any of my applications, but sometimes I just have to do what I have to do.

I will also need to run a ruby on rails server. I am hoping this is as easy as it looks in the different installation manuals I have checked. I may have to read up on developing with ruby. There is a great resource for this here, but I have yet to find the time to dive into that.

Other than that I will most likely follow the install instructions for the services I want.

The web services

Here is the initial list of services, or web sites, I want installed on the server

  • ownCloud – This is for a local run Dropbox variant. All files will also sync to an offsite server for extra safety
  • Piwigo – This is the photo library of choice. It works fairly well, got lots of add-ons and is open source. All photos will be synced to an offsite server.
  • Thingspeak/data.sparkfun – This is for the data logging once I get the ESP8266 projects up and running. As of yet I have no preffered choice between the two, so both will be installed and tested. Also I dont think there is anything stopping me from logging data to both services simultaneous.

The hardware

The Linux server will be running on an old computer I got lying around. I was actually looking into buying an old real server, rack mounted with dual xeon processors and hot swap disks and what not. What I realised though was that those rack servers are about 70cm in depth, and my rack cabinet is about 45cm. Simple math then proves that this might not be as feasible as I first hoped. Until I can find a spot for a full size rack cabinet, and a spot to put it, I am stuck with my old desktop computers.

Full specs of the server:

  • Core 2 Duo E8200 2.66GHz
  • 16GB RAM
  • 120GB SSD + 2x1TB HDD
  • Single Gb ethernet

Definitely not a powerhouse of a server, but with 2 users I do believe it is more than enough to host a webserver. The harddisk space will be upgraded as needed.

So thats it for the server. Next up will be installing the software on a virtual machine, testing and documenting all the way. Hopefully I will be a little bit wiser about Linux when this is all done.

What all true nerds want. This is from facebooks datacenter in Sweden.
What all true nerds want. This is from facebooks datacenter in Sweden.

 

 

A simple night light

In northern Norway it is dark almost all day in the winter, and I had an idea to make a simple night light that would turn automatically when ever it was needed. Perfect for a smooth transition between the pitch dark bedroom and the flood lighted bathroom. A simple motion detector connected to a low power LED should be able to light the hallway sufficiently to avoid any danger related to stumbling around in the dark.

The parts

The night light should be as simple as possible. I considered building my own motion detector, using a micro controller and IR LED and receiver. This, as I found out in so many projects, was overkill and simple solutions already exist. I bought a PIR sensor (Passive Infrared Sensor) from eBay, paying $2 for two of them.

PIR sensor

I also bought 10 1W LEDs for about $2.

LEDs

I am not sure how much power the PIR can deliver, so I might need a transistor to run current through the LED. The BC337 is rated at 800mA continuously, which should be more than enough for the 1W LED.

BC337 Transistor

The casing is a small plastic box I had lying around.

The encasing

And I needed a battery pack to power the whole thing

Batterypack

Planning the setup

During engineering school I was taught how to use OrCad. This corporate, pay-till-your-fingers-bleed, awful program that had all kinds of backward solutions. A program where Ctrl-z was the shortcut to mess up something important with no shortcut to get it back. Where you had to spend 2 hours drawing a new component because it spat out error messages that might as well have been in Arabic because it gave no sense.

Luckily, the open source community has made a awesome, free, easy to use, alternative. They even have the support and love of CERNKiCad was used extensively  through the bachelor thesis, and I came to love it.

A quick schematic drawing in KiCad gives the project an overview, and simplifies any troubleshooting. If you are a bit rusty it also helps getting the right component in the right place. Those NPN/PNP always gets mixed up!

PIR sensor technical drawing

Testing the setup

Instead of gluing and soldering the whole thing and hoping it works I prefer to test the setup in a breadboard first. Sometimes, especially when you are very sure you know how stuff works, it does not do what you thought it would. A breadboard test is a good way to sort out any brain farts you had while thinking of the solution.

The breadboard testing is a crucial part of any project like this. A missed connection, a short, LED connected backwards... Any small mistake is easy to fix while nothing is soldered or glued in place.
The breadboard testing is a crucial part of any project like this. A missed connection, a short, LED connected backwards… Any small mistake is easy to fix while nothing is soldered or glued in place.

After connecting 4,5V to GND and ground to VCC, realizing that I made that connection mistake and fixing it, it was time to turn on the power supply. The LED turns on when the PIR sensor detects motion, and turns off again shortly after. The current draw at 4,5V was about 110mA with the LED on, and 0mA while off. This of course is raw numbers read of the power supply, but this kind of project doesn’t really need 100% exact measurements.

One problem discovered was the ON-time for the LED. The PIR sensor got two pot-meters that I believe should control sensitivity and time ON. The sensitivity one seems to do the job at any setting. The timer function though seems to be “on for a second” or “on forever”. I will need to test this a bit further, or test with the other PIR sensor in case this one is flawed. Either way, as long as it is motion in from of the sensor the light will stay on.

Next up is assembling the thing, and testing it in the hallway at night.

 

Dropping the Dropbox

It has been years since I started using Dropbox. It lets me sync my files between my devices, while keeping a copy in the cloud. Perfect for everything you dont want to loose. They even had different “activities” you could do to gain more disk space. Right now I got 9.4GB, which is more than enough to keep all my school work, previous job applications, my resume and other documents. I even use it to sync the pictures from my phone. This however is where the problems begin. When dealing with pictures and other, bigger files, the 9.4GB is to little. I could upgrade to an unlimited plan of course, it doesn’t even cost much. Less than 10 Euros a month for 1TB of space. But I already pay for Jottacloud, which is about 50 Euros per year for “unlimited” space. I dont use Jottacloud for the syncing however. With Jottacloud I can backup selected folders, with no need to sync everything back and forth wasting bandwidth. So I need a system to sync my phone and computers, which is also capable of backing up selected folders to Jottacloud.

I periodically need to copy my pictures elsewhere and delete them from my dropbox account to not run out of space.
I periodically need to copy my pictures elsewhere and delete them from my dropbox account to not run out of space.

Enter ownCloud. Dropbox running on my own server, with unlimited space and all the same possibilities. It is open source and seems to have a ton of addons. It runs on a webserver and got a good manual. Running the cloud on my own server also gives me more freedom with what to do with the files. As mentioned, I want to select specific folders for backup to the Jottacloud. With administrator access only my skills and fantasy sets the limit here.

This is not a “no work needed” kind of deal though. I currently have a windows server, running a web server and the Jottacloud backup program. The reason for this is because Jottacloud is Windows/Mac only. No Linux support. I have no problems running a windows server, but I really want to learn Linux better. Luckily I have a few functioning computers lying around, so I have the possibility to run both Linux and Windows servers. In the future I might look into using a Python script to do the backup, but this is not as straight forward as using the program running on a Windows machine…

I will write about the installation process of Linux and ownCloud in a later post.

F.A.I.L.

Just a small update on the project. Most DIY’ers would probably have finished a dozen weather stations by this time, lucky for me it is not a race!

Some of the reason things take time is distractions. I can search around for solutions for my current project, stumble upon a exciting new project and de-rail completely. Another reason for the long time is when my planned solutions simply just fail. According to the internet FAIL just means “first attempt in learning”. Well, with the ESP8266 I should be about to NAIL it. That means “Nth attempt in learning”.

This doesnt look like a Python prompt to me. The latter part of the screen kept looping forever…

Trying to get MicroPython running on the thing is the easiest thing in the world. Thats what the internet says. I have spent many hours trying to follow every guide out there and not getting the same end result. That means that at the moment I am at square one.

So instead of running MicroPython I will give Zerynth a go. This is Python packed in a virtual machine, ready to run on the ESP8266. I gave it a try with some sample code, and it worked straight out of the box. It’s not the real thing, and a bit less DIY than intended, but for a first edition of the weather station I guess it will suffice.

3 different ESP8266 kits. Upper right was the first one. After hours of troubleshooting I managed to flash it. It has been dead ever since. Upper left is the one I am currently fiddling with. No luck at flashing that one with MicroPython either. The bottom one is the one planned to go into the real projects. It is not very breadboard friendly, so I have not tried getting it to work. Yet.
3 different ESP8266 kits. Upper right was the first one. After hours of troubleshooting I managed to flash it. It has been dead ever since. Upper left is the one I am currently fiddling with. No luck at flashing that one with MicroPython either. The bottom one is the one planned to go into the real projects. It is not very breadboard friendly, so I have not tried getting it to work. Yet.

I also looked around to find some way to display and/or store the data. Making a database is easy enough I guess, but there are a ton of ready made web apps out there. So why reinvent the wheel? Thingspeak is actually a free service offering (I think) what I am looking for, and even better: It is open source! This means I will be able to run it on my base station for the weather station.

To wrap it up: The project is not dead. It’s just slow. Also, CivVI came out this weekend, and I’ll continue my project right after this next turn…

Technological advancements…

Making electronic devices today seems to be hard work. As soon as you got a solution planned out, something new, faster, more versatile and cheaper comes up. As I mentioned in my last post the library I was planning to use for the wireless data transfer was outdated. While researching the new solution I came across the ESP8266, which obsoleted the old solution completely.

Enter the ESP8266:

This little thumb sized gadget changes everything.
This little thumb sized gadget changes everything.

The ESP8266 is nothing new really, it has been on the market for a few years already. It was released without any documentation in english, and hackers around the globe has slowly figuring out its capabilities. Today the possibilities with the device seems endless, giving true WiFi to any project. And this is the thing that will power my weather station.

The ESP8266 has, for my purposes, the same capabilities as an Arduino, but with the added benefit of integrated WiFi. This means the outdoor device will update the database with log data by itself. I can have several indoor devices to log data from multiple rooms. And the base station can just pull all the data from the database to display all kinds of stuff.

Python!

As much as I want to learn C/C++ I do like Python a lot better. According to this YouTube video running Python on the ESP8266 is no problem. And this is really good news, since it will ease the code-writing part. A detailed guide for making this work can be found here.

When?

I mentioned in an earlier post that I already got one of these devices. At the time I obviously didn’t know the possibilities. I have done some quick tests with it, though without much luck. There seems to be a few quirks to sort out. I also ordered a bunch more from eBay, which will arrive i 2-8 weeks or so.

A few resources.

Mostly for my own benefit, a few links to sites about the ESP8266:

I might update the list with more links as I find more.

Back in business!

Wow, its almost been half a year since my last post. The time available for project just wasnt there between my paternity leave, summer vacation and other stuff. Like procrastination.

But here I am, back in business, ready to continue with the different projects I got in my head. First off; back to the weather station.

The temperature reading is done, but as I mentioned in another post I ran into some hiccups when sending and receiving the data. After investigating potential solutions I discovered that the library I used, VirtualWire, is no longer supported*. It is instead replaced by a new, and possibly better, library called RadioHead. Meaning half of my code is obsolete and needs to be redone. At a glance it looks similar, but after 5 months without writing a single line of code I need to get my head around it. So next step will be to figure that out.

I did however find a nice link for getting the arduino talk to the Raspberry Pi. The RPi is the true brains behind this project, and will be connected to a 7 inch display to show all kinds of cool data. I hope. Using the quick guide found here it turns out the Pi just needs a serial python library to be set. The rest is like reading the data off of the serial monitor in the Arduino software. Which should make the data easy to use with Python.

So what is the plan?

  • Fix the wireless communication between the Arduinos.
  • Connect the humidity sensor and the barometer
  • Clean up the code for the Arduinos.

When all this is done its time to get the RPi connected to the display and start thinking of an interface. But this is most likely some time in the future. Hopefully I will be posting more updates in the coming days and weeks.

 

*Edit

Seems the VirtualWire library was obsolete long ago, but I failed to realize it until now. Oh well.