WHS and SmugMug- Architecting the Add-In

The structure of an Add-In is important to what we are trying to accomplish here.

I could put all my code as part of a Console Tab and have people manually getting it working.

That would no doubt work, but its kind of lame and falls short of expectations.

So we want an application that reacts in real-time to folder and file changes, but is also able to be scheduled ( say for 3am after a defrag or back-up session).

This requires a number of different application blocks, each an application in its own right.

To see how an Add-In works in code, the WHS Photo Uploader for Flickr is hosted on Codeplex. Unfortunately its a totally different application for a totally different service. But I did get a good look at the Console and Settings Tabs to see what the code would/should look like.

If you want to download and run it, I’d download the WIX Installer XML for Visual Studio 2005 to allow the  Add-in’s installer to be recognised by VS and compiled.

So the lesson is that Console Code is run from the Console (doh!) and any file monitoring code will only work while the console is running.

So we’ll require a service to be running to monitor our gallery folders and record the events to our Master XML file. Hence this application will consists of XML code and our FileWatcher Object. It will need to use the Windows API to allow it to be installed and run  as a service.

Then, we want to schedule our uploads so that they are not taking over your broadband connection. We could use System.Threading to calculate the number of seconds remaining and have the timer call our upload method once the time is reached.

However, we want to keep things as simple as possible. This means using windows’ Scheduled Tasks to call our uploader (an exe file). Scheduling Tasks is for some reason not in the .Net Framework 2.0 so we’ll use a library to wrap all the C++ Interop code. There is a CodeProject article here on it with the library. This also means we can run tasks with a users username and password (i.e “Administrator” and whatever the password is) so tasks will run, in theory, even when no one is logged in.

So we have three applications here, all rather trivial:

• The Console Tab and Settings Tab with code for usernames/passwords as well as rescheduling uploads and manually starting the uploader. I have the idea of getting album stats and displaying the charts by using the Google Charts API, but that’s for version 1.1.
• The Service app that will take file and folder change events and writes them to our Master Settings file. This runs independent of any user being logged in.
• The Uploader that is scheduled to run at such and such a time to execute the events in our Settings file on the server using he SmugMug API. Also runs independent of any users being logged in.

Slowly but surely the Add-in progresses.

File Sharing- Oh the Humanity!!!

Via Hugh MacLeod - who did a cartoon about this.

The IFPI - the International Federation of Phonographic Industries - is the global music industry organisation whose very name tells you how long ago progress overtook it. On Thursday it published its digital music report for 2008, which says boldly that “the spread of unlicensed music on ISP networks is choking revenues to record companies and investment in artists, despite a healthy increase in digital sales in 2007, up approximately 40% on the previous year”. (If you’re wondering, those sales were $2.9bn (£1.45bn) for the year, including ringtones.)

The IFPI’s solution? Sort it out at the internet service provider level. “ISP cooperation, via systematic disconnection of infringers and the use of filtering technologies, is the most effective way copyright theft can be controlled. Independent estimates say up to 80 per cent of ISP traffic comprises distribution of copyright-infringing files.”

You know what I say? Damn right. Let’s get ISPs to stop copyright infringement. But, um, music people? Better form an orderly queue. You think you were the first to suffer from your content getting ripped off and spread to the four corners of the earth? Get to the back of the line, bud. There’s a few people ahead of you.

 

Read the rest of this fine article here.

Quote of the Day

Yes, we have to divide up our time like that, between our politics and our equations. But to me our equations are far more important, for politics are only a matter of present concern. A mathematical equation stands forever.
  - Albert Einstein

WHS and SmugMug - A Word About XML

XML is a markup language in the same family as HTML. This means you have nested elements in a  structured document.

Its structure makes it ideal to read and write data quite easily and in  a logical format. Since the elements all have names, the documents are usually human readable. This is of course a disadvantage for a proprietary file format.

Reading and writing XML is quite easy with Visual Studio and the .Net Framework. You can use an object called XMLSerializer that will do it automatically, but it depends on you structuring you data for it. It is also great for recurring data. In the example below we are reading in multiple FileEvent object.

The way I prefer doing it is using XMlReader and XMLWriter. Like so:

reader.Read();

 for(int loops =0; loops<count;loops++){
                    FileEvent temp = new FileEvent();
                    reader.ReadStartElement();
                    temp.FilePath=reader.ReadElementString();
                    temp.EventType = reader.ReadElementString();
                    reader.ReadEndElement();
                    mastersettings.Events.Enqueue(temp);

}

reader.Close(); 

Line for line, the read and write methods of your application is exactly the same as your final XML document, unless you use a loop.

Once you know how to use XmlReader and Writer, its easy to use. Essentially you have to ensure that your elements open and close at the right points it the code to ensure that you get a properly formatted Xml Document.

Second, make sure you are reading the right data type. I use strings as they are quite easy to use, even for dates and such. Most of my errors come from, that.

Third, make sure that you read and write methods read and write the exact same data at the exact same points ( you don’t want to be reading a name when you expect a date, etc).

Fourth, you need to use the WriteStartDocument method to start writing an Xml file and WriteEndDocument for ending one. thing of them as a huge node that encapsulates rest of the document. Just remember, one document per file.

And that’s all there is to it.

More later on how I’m using XML as a data source.

Related Post: WHS and SmugMug - Keeping Track of Files

WHS and SmugMug - Keeping Track of Files

The question that has to be asked at the beginning of every software project, hobbyist or commercial is: “What are we trying to achieve here?”.

For this project, the aim is to move all the contents of one folder to a gallery on SmugMug - and keep them up-to-date. This means that modifications ( tags, edits, etc), additions and deletions are replicated on the gallery in question.

This requires two things:

• The program to be aware of what is happening to those folders
• And for the program to be aware of what has already been uploaded.

Number one requires a FileWatcher Object from System.IO.

The Filewatcher throws an event for creations, deletions  and modifications. This allows us to determine the nature of, say, a create event and decide if its a file or directory that has been created. Based on that we can take other action.

We do this by calling System.IO.Directory.Exists(string path) that returns a boolean. You could use the File.Exists(string path) method as well.

Second, a running tally has to be kept of what is going on. We do that by maintaining a master file in our root directory (say \\Photos\SmugMug)  and child files in each directory. The actual directory may or may not be exposed as a setting via the Console.

The master file is essentially has a queue of file events to execute on the server the next time its syncs. This stops large file  ( i.e. large numbers of files) operations overloading the server (resulting in multiple create events) as Demigrator.exe is working on those files as well. So better  to queue the uploads to take place when the disks are idle. I’m using XML and handwriting the read and write procedures - you’ll understand why at the end of the post.

The second requires a file in each directory with the SmugMug Gallery ID and  a list of each file and its SmugMug ID in that Directory. This allows us quick access to files instead of iterating over the lot and comparing each file to the one on SmugMug.

As far as scheduling the uploader is concerned, I haven’t gone into that in too much detail. I could schedule an exe to run using the task scheduler. Its not a bad way of doing things. Norton Internet Security does just that for its scheduling. That, of course has a disadvantage of spitting up the add-in into a logical front-end and back-end.

And by the way, I’m writing this in Visual Studio 2005 using C#. I’m going to do a second version in C# Express 2008 (to use LINQ-to-XML, mainly) for a comparison.

WHS and Smugmug 1

The first thing I looked for after signing up for SmugMug was a WHS Add-In since there is one for Flikr. Lo and behold, there isn’t one.

So last night I started writing an application architecture post when it occurred to me that the best way of explaining it was to write a program from scratch.

Enter the SmugMug Add-In idea. So I’ll be doing that in a short series of posts.

First things first, I haven’t decided whether to use Visual Studio 2005 or Express 2008.

Second, the WHS SDK is already in your servers’ Program Files\Windows Home Server folder, so don’t bother looking online for it. You’ll need these two files added as references in Visual Studio ( or Express):

• HomeServerExt.dll
• Microsoft.HomeServer.SDK.Interop.v1.dll

Third, the Smugmug API can be downloaded from SmugMug.com. And it looks to be rather straight forward to write code against it (Note - I always say that looking at an API for the first time, then regret it later). And you’ll need an API key from SmugMug. And you’ll need to add the API in as a reference in Visual Studio or Express.

For a quick look at it, Evan Leventhal’s .NET wrapper SmugMug.NET, is a good start.

Fourth there the way the program will work is pretty straight forward :

• The folder name can be the name of the gallery OR an XML file in the folder will identify which gallery it corresponds to (that is the SmugMug Gallery ID) ala SyncToy.
• Since I keep far more photos than I post in \\Server\Photos,  it would be a good idea to create a separate folder containing all the gallery folders for syncing.
• As far as checking the folders themselves, the .Net 2.0 has a FileWatcher object ( or something to that effect).
• A settings tab will have a list of folders in the gallery folder that will be synced by default unless you uncheck it.
• As far as a console tab goes, I have no idea - stats, gallery browser, etc.
• And scheduling of some sort, could do somthing ala SyncToy (and no, I’m not french).

I’m writing this primarily for myself  as I’m fed up with having to Remote in and use the Send To SmugMug shell app but it will be packaged neatly and on Codeplex once its finished, I imagine.

Blog Directions

So, this is supposed to be a tech blog (along with the few hundred thousand of them already out there). Thing is that I’ve only used it for two things lately: Youtube videos to do with Apple and various complaints mostly to do with Google et al.

I need to narrow the scope down a bit. not quite sure exactly what on. Lets go through it.

SQL Server? I get the basics but I’d rather use Subsonic any day of the week.

ASP.net? Yes, but 2.0. I’ve used the Visual Studio 2008 and it rocks for web design, so a tour of Visual Web Designer Express 2008 sounds good.

Visual Basic 8.0? Again, I haven’t got round to using it, at least not with .Net 3.5. I’m rather good at VB, if I do say so myself.

C# 3.0? The C version of VB ( I’m kidding ), but yeah getting to grips with it might prove entertaining.

LINQ? Haven’t touched it, though I’ve diligently read Scott’s posts on the topic. It sure rocks so I might take it for asp on some Commerce Starter Kit databases I have lying around.

Virtual Server? I actually had a draft post about that lying around somewhere. I’ll get it up as soon as I’ve completed taking VS for a test drive ( always handy to have a copy of Windows Server 2003 and SQL 2005  lying around).

Networking?? I’m laying gigabit throughout the house to a central 24-port switch. So details on that later.

What else? Java??? Seriously, although I can see the languages’ power and potential as well a pretty safe choice when undertaking any software project, I wish Microsoft would do a deal with Jonathan Schwartz and include it as part of Visual Studio. With a .Net interop??? And Sun, please make your Java download page a wee bit more understandable to a  (relative) newbie.

Programming in general?? Perhaps something slightly more advanced than Hello World. Toolkits I use and so on. I mean, application architecture is a huge subject, as is project planning (UML, etc). Plenty we could go over in there.

At the end of the day, its good opportunity to consider moving any future photography posts to their own blog.

You never know.

iPhone AD

Well, not really:

Are you Ready for AIR

Check the Macbook Air out:

Many thanks to FSJ for passing it on.

Quote of the Day

Let not the sands of time get in your lunch.
  - National Lampoon