Mike's Notes

Just a quick post to confirm move from mrdw.co.uk/blog has happened – decided that wordpress probably have more experience than myself in hosting a blog, (saves me from having to update wordpress install etc.)

I’ve completed the last todo list for the running club message board, including a few extras that I added on the way, but have also added many more things to my mental (as in held in my mind, but maybe also MENTAL) todo list, so nows I good time to write them down so I can tick them off as I go.

1. Tidy up code, not aiming for it to be perfect but having the luxury of just hacking away with getting functionality working quickly as the main driver it could do with being tidied up before attempting anymore work on it.
2. Templating: At the moment theres hardcoding, so will only work for one messageboard, but I want to set up another one for testing, one for demonstration, and also another running club have one that is still on the old PHP code so would be nice to update that, shouldn’t be too hard, be nice to just have a properties file that I can switch.
3. Use Github: I’ve been using Codesion for some subversion hosting, but the pricing doesn’t really make sense for small projects that aren’t expected to make any money, so having used up my allowance of projects I’ve resorted to backing up a zip file. Github looks good, I’m happy to make the code public, and also feel like I should make the effort to try Git.
4. Allow login through other services – google/twitter/yahoo etc., I don’t like Facebook so be good to  not be encourageing people to use it even more.
5. Make the login system create actual users and generally be better, it works as it is but its about as basic as you can get – you’re logged in or not, but everyone effectively logs in as same user, but just gives a different name to their post.
6. Admin users – need to be able to have administrators that can delete posts etc.
7. Allow editing of posts/replies you’ve just created,  not sure about going back and editing old messages, but should be able to use the session to allow editing of something that’s just been posted, which is when people normally spot typos.

That will do for starters, got loads of other stuff in I’d like to do, be good to start using it as a basis for trying / evaluating different technologies – and brushing up things I’ve got on my CV. For example using JPA, Spring, Shiro. Then be good to try a Ruby version, a google app engine version and things I’ve only just heard of Node and Mongodb …

A Facebook app that’s worse than Farmville is also on my mental list.

Got Facebook login working yesterday.

It’s been a while since I posted so thought I should do a quick one whilst I’ve got time.  I’ve got an idea to go back over the various tasks involved in getting the message board running and write them up as proper “how to” blog posts that might actually be useful to someone, or at least me if I forget how I did it.

Anyway, I got facebook login working, after a bit of googling and reading through the facebook developer documentation. I have resorted to dropping the Spring login stuff, which I’d used to allow ajax logins to work with standard J2EE security. As I couldn’t find away of getting the standard J2EE security to work with logins from Facebook (not facebook specific – would have had the same problem with any OAuth login I think). I did try the new(ish) HttpServletRequest.login method available in Servlet API 3.0 – Tomcat 7.0, but found it only seemed to login that request, not the session, so when the response redirected the user to the message board after the facebook login they were still logged out as far as the session was concerned. So I’ve done what feels like a quick hack and just stored a “logged in” flag in the session.

I will investigate Apache Shiro, http://shiro.apache.org/ , an alternative to J2EE security. I’m not convinced J2EE standard is flexible enough for most uses I’m likely to come accross in the real world, but maybe I’m missing something, but it seems as though the creator of Shiro thought the same.

So in the past I’d used Twitter4J to work with twitter, and it had proved to be very simple, and just work.

For example the give a userid and password for twitter I could just write code as follows to do an update.

Twitter twitter = new Twitter(twitterID,twitterPassword);
twitter.setSource(“mikewarrenstwitterprogram”);

Status status = twitter.update( responseString );

Seems things have moved on, and there’s a whole load of configuration required to be able to authenticate programatically with Twitter. I’m not sure why exactly, because if I use a twitter client all I need to give it is username and password, but I’m sure there’s a good reason  … just need to find out how and why, which means what I thought would be a 10 minute bit of work will be as long as a piece of string – that’s computing for you.

Looks like I need to start with https://dev.twitter.com/docs/auth

Update 22/09/2011:

Just a quick note to say I got it working after jumping though a couple of hoops, using Twitter4J still, but you’re now forced to use OAuth to interact with Twitter.  Basically you need to authorise your app with the account you want to post to, and then set up access tokens and secrets etc. I’ll write it up as a nice “how to” blog post when I get round to it.

Well almost, I want to make a tweak or two before I make it available for use , but the work I originally listed back in April is done

• Switch to use the live DB currently being used by the PHP app. – DONE
• Make links automatically out of URLs in messages – DONE
• Clean up html so it looks presentable (this could be a big one) – DONE
• Either set up email notifications, or some alternative, e.g. posts to twitter – DONE

I mainly want to change the URL that it’s using so that once it starts being used people can favourite a URL and not have it change a week later. Also although I’ve got email notifications, I’d like to add twitter posting as I know that’s pretty easy, and gives some advantage from a user’s point of view to the new version.

I’ve been tinkering with adding facebook “like” and google “plus 1”, but easy to do for the whole page, but not working for me when I try to make them refer to an individual post, so that’s another thing to add to the list.

Part of me can’t quite believe that it’s taken so long, but then it has been the just the odd bit of time here and there in between working, sleeping, and various recreational activitities, so part of me is pretty pleased to have done it. Blogging has be useful as a reminder of what progress I’ve made – important when it’s so slow, and also an incentive to at least try and do something every week or two.I intend to continue, documenting adding various features, refactoring and maybe trying out different languages/frameworks, keeping a low threshold on quality of what I write in the hope that will mean I do actually post something.

Now I can treat myself to a waterproof camera housing, although having spent this long I’m not sure how much I really want one …

Not much to report again, in the little bit of time I’ve managed to grab I’ve been looking at the ake links automatically out of URLs in messages task.

I’d already done this in PHP, and planned to do it in Java. After implementing the code to do it in a getter on my Message class I realised the serialization done by com.google.gson.Gson.toJson wasn’t using the getter. After stopping to consider I decided it made more sense to do it in the code doing the actual display anyway, i.e. the javascript.

So looking into Javascript regular expressions and regular expressions for URLs I found this http://daringfireball.net/2010/07/improved_regex_for_matching_urls which looked pretty promising. But had trouble converting it into a javascript regexp (I think mainly due to some none ASCII characters, and the escaping of certain characters in Javascript), and had a feeling that it would actually be more lenient than I really want – e.g. I don’t want to match ftp.
So resorted to just converting what I was doing in PHP, which is a bit inelegant, but seems to work. It uses three separate replaces to cater for URL’s starting with “http://” (but not www’s), then strips off any “http://” from www’s and then sorts out remaining urls starting with “www.”. I know one bug is that it seems to fail to match the url after a “www” on its own, and also with a bit of regexp skill I should be able to combine the 3 separate passes into one regular expression.

As a quick test I used this javascript: ( I definitely need to sort out WordPress displaying code)


var data = "blah http://google.co.uk blah www.bbc.co.uk www www.mrdw.co.uk http://www.bbc.co.uk blah www.bbc.co.uk blah http://google.co.uk /mb mike gahsj asghja";
var reg1 = new RegExp("http://[^w][^w][^w][a-zA-Z0-9|_+-/?&=.%:,~#]*","g");
var data1 = data.replace(reg1, '$&');// deal with non www addresses that have a http://
var reg2 = new RegExp("http://www.","g");
var data2 = data1.replace(reg2, ' www.');// 1st strip off any http://www to just www so next rule will work
var reg3 = new RegExp("www.[a-zA-Z0-9|_+-/?&=.%:,~#]*","gim");
var data3 = data2.replace(reg3, '$&');
document.write("

"+data1+"

");
document.write("

");


So as is traditional a quick reminder for myself of what my todo list was looking like:

So, as is traditional, I didn’t get to spend much time on it this week, if only I didn’t have any family or freinds or leisure activities I might have more time to write a crappy message board application. Oh well.

But have managed to prove that I should be able to use a gmail account to send notification emails from – not sure if this is an advantage or not over setting up an email server, but in the spirit of just getting something working in whatever way is easiest it does the job.

Also a bit of tweaking of nginx config seemed to be having no effect, despite “ctrl shift delete, delete all cache” in firefox, which stumped me till I found a default file in /etc/nginx/sites-available which was what seemed to be taking precedence. Or maybe nginx wasn’t reloading the config like I thought it was when I ran “nginx -s reload”. Anyway, it’s working now, but need to experiment a bit to be confident that I’ve got any sort of a clue as to why.

Code for emailing from Java using gmail (mostly from http://www.velocityreviews.com/forums/t141237-send-smtp-mail-using-javamail-with-gmail-account.html
) , sorry wordpress doesn’t format better …:

package uk.co.mrdw.util;

// Setting the Subject and Content Type
msg.setSubject(subject);
msg.setContent(message, "text/plain");
Transport.send(msg);
}
}


(Not sure if this is actually the 3rd post on the message board – but will say it is for the sake of having a title)
So just a quick reminder for myself of what my todo list was looking like:

First one is pretty much done, although I haven’t tested what the performance is going to be like. I might to just copy the database over to the new server if it’s terrible, but would be nice to be able to run the old PHP app.

Number 4 (domain name) is done – got http://www.msgbrd.co.uk from 123-reg.co.uk , I didn’t do a lot of research on how domain name’s work, but it was pretty easy to set up. I’ve got a pdf or two I might go back and read if I feel like I really need to know more.

For number 5 (webserver) I’ve taken my brother’s advice and used nginx, seems it’s smaller and faster than apache and is designed to act in the way I want it – i.e. passing on requests to tomcat. I’ve got it installed and running, and can edit the config to prevent the processing of requests, or just display a welcome to nginx screen. Need to do a little more reading to see how to get it to forward to stuff on to tomcat.

I haven’t had much time again to spend on it this week, so pleased to be making progress in the right direction, and feels like I’m at a point where anytime I’ve got a spare 30 minutes I can sit down and do something useful on it, which is good.

Blogging like this is working well to just give me a little bit extra motivation. I’m pretty much sure I could safely use this blog as a place to store my passwords as I’m the only one reading it, which is how I like it at the moment. Although there are loads of spammers commenting, which is a bit annoying (I’m not accepting their comments , so they shouldn’t be visible), maybe I need to upgrade wordpress to get a better captcha.

So another week over and what have we done ?
A new one just begining, I hope it is fun.

Well, I’ve remembered two other things I need to do, so my new list is as follows:

Remaining tasks before I consider myself eligible for the underwater housing treat are:

Switch to use the live DB currently being used by the PHP app.

Make links automatically out of URLs in messages

Clean up html so it looks presentable (this could be a big one)

Either set up email notifications, or some alternative, e.g. posts to twitter

Setup a domain name.

Setup a webserver, which stand infront of tomcat (which is running on port 8080) so can use port 80

So that’s progress of a sort, but basically the wrong sort.

One thing I have done is do the majority of the first task,so it should now be very simple to switch over to use the live database that the PHP app uses. This is important so I can have a period of them running side by side, and then move over to a database hosted on my memset server when I’m confident it’s working.

One thing I can say is that my coding so far is very rough and ready, and not something I’d want to show anyone else. So maybe there’l be a post in a few weeks, bemoaning the fact that I didn’t do it properly in the first place. Or maybe there will be one saying how it was more important to have just done something, and have got it out there. At the moment I feel like the second is more likely to be the case, at least in terms of keeping me motivated, but we’ll see what problems my slap dash approach throughs up in the future, maybe.

Otherwise my ankle seems to have recovered well from a sprain, so various races and club training nights provide an excuse for so little progress.

So over a year since my last post, I’m going to reduce the quality threshold on what I post and see if I can post more regularly.
One of the posts has actually been useful to me to come back and remember what I was doing, so as long as it’s ocasionally useful to me then it’s worth doing.

So my current home programming project is converting the message board that I’ve set up for a couple of running clubs, e.g. Wrekin Road Runners, currently hosted here, http://www.mrdw.co.uk/wrr/noticeboard.php , from a crappy php application, to a crappy Java one (although it will be allowing ajax posting of messages and logging in) once I’ve done that and got it up on line I’m going to treat myself to an underwater housing for my camera http://www.camerasunderwater.co.uk/ewa_marine/slr/ub-series.html

So far I’ve got a Java app running on tomcat on a Memset virtual server ( https://www.memset.com/dedicated-servers/virtual.php ) using mysql that roughly does what I need. I went with Memset partly because of there claim to be green and a vague desire to support a UK company. Linode ( http://www.linode.com/ ) seems to be the default choice for this sort of thing at the moment, and I might try them for a comparison at somepoint.

Remaining tasks before I consider myself eligible for the underwater housing treat are:

Once I’ve done that, there’s loads I could do, but think it might be a usefull base project for trying out different web frameworks, got a plan to learn ruby on rails, so maybe writing it as a ruby on rails project will come. Also from point of view of making the message board useful, allowing logons from Facebook, twitter , openid etc. would be good, and an rss feed is pretty much essential I think.

I’m undecided as to whether there is really any point running clubs having message boards instead of just using something free like facebook, or a google group. There does seem to be something in not having to force people to create another logon for a little message board for people who all know each other (currently everyone posting uses the same shared password) , and also for not forcing people to sign up to something like facebook if they don’t want to. (Plus I don’t really like facebook). Also as it’s dedicated to just providing a simple message board it can be less cluttered and I don’t intend on having adverts. (I don’t really like adverts). Anyway, as long as it is useful as a practical learning project for me it being of any use to anyone else is a bonus.

Right, now I’ve blogged about it, maybe I’ll do some work on it.