Click to close the image preview

Showing results for 
Search instead for 
Do you mean 
Most Liked Users
User Reps Count
1
Labels
Announcements
New from the Blog: Tour of your MozyHome Account
Join us for Part 1 in our series where we provide a detailed tour of the MozyHome account settings.

May's Webinar: How long does my data stay on Mozy servers?
This months webinar on 05/31 will look at the recent changes to the Mozy retention policies which gives users access to older versions of files for longer. Register today!

Looking for Support?
Create a new post with your question and our community members and moderators will be happy to help answer it for you!
Ted_Haeger
Moderator
(260)

Stash 0.14 - Better, Stronger, FASTER!

Mozy has begun to roll out the most significant update to Stash since its debut. The update looks and feels just as before, so don't look for any big change to the user interface. The biggest feature in this update is under the hood: we have finally completed the long awaited package of bandwidth & speed improvements to which I have alluded in the forums for many, many months. Before I get into explaining the new features, let me provide some background on how we are making this update available.

 

Who Gets It & When?

Please read this section. Many will be very eager to get this update, and we make it available to all Stash users over the course of a couple months. However, the significance of this change warrants a careful rollout. It goes out first to a select group of users that includes the members of the MozyHome Customer Advisory Board, certain members of the somewhat larger Mozy Terabyte Team, and select other MozyHome users. After we have confirmed a solid rollout for the first group, we will progressively extend the update to more and more Stash users. So if you have not gotten the update yet, please be patient. It's coming.

 

New Features

The official release notes for 0.14 are attached to this message below, providing you with a summary of many of the smaller fixes in this update. I'll spend the rest of this post diving a bit deeper on three of the bigger features it contains, starting with the simplest.

 

Linked Notifications

When stash pops up a notification about a file getting updated, you can now click the notification to open the file's parent folder, with the file selected in it. To test it out, try an uploading a picture from the Mozy mobile app, and then click the notification when it arrives on your computer. Voila! Your file browser opens to show your uploaded picture.

 

Lazy Sync

If you are one of those brave people who puts Quicken, Quickbooks or Outlook files in your Stash (despite all our recommendations not to!), this one is for you. The Stash sync engine now keeps a list of file types that typically get very large, and stay open when in use by their associated app.

 

When the Stash software detects a change to one of these files, it checks to see whether the app it belongs to still has it open before trying to synchronize it. If the file is still in use, Stash waits 30 seconds, and then tries again. It will do this until two hours have elapsed, and then sync the file even if it cannot get exclusive access. So, if you put a .pst file in a computer's Stash folder, Stash won't upload it if Outlook is still running, or at least two hours have elapsed.

 

The point of Lazy Sync to keep Stash from chewing up bandwidth by frequently uploading large files while they're in use. Changes to such files typically contain very small updates, and they happen so frequently that it's inefficient to continuously update the file while it's in use. Another benefit of Lazy Sync is that it limits how many versions of such files you have to sift through when you want to restore a past version. Coupled with the bandwidth & speed improvements described below, this should vastly improve the handling of such files.

 

We still recommend that you don't keep such files in your Stash because they tend to cause sync conflicts between computers. But many of you are out there pushing the envelope anyway, so if you are one of the bold, please let us know how Lazy Sync works for you.

 

Lazy Sync File Types by Filename Extension

These are the file types that this first version of Lazy Sync addresses.

  • qdb, qdf - Quicken and Quickbooks
  • pst - Microsoft Outlook
  • mdb, ldb - Microsoft Access
  • mny - Microsoft Money
  • apdb, db, dbf, db3, sqlite - common databases types
  • tc - TrueCrypt virtual encrypted disk 

Faster Syncing…Way Faster Syncing

The biggest update to this new version of Stash is the efficiency with which it uses the network. Two new major components make a noticeable increase sync speed: data compression and a new type of differential patching. If you're not interested in the geeky details, suffice to say that Stash is now much faster and skip the rest of this section. But for those who like to understand how things work by taking them apart—computers, clocks, cats—read on!

 

Let's start with data compression. As of this update, file data gets compressed before going over the wire for both uploads and downloads. If you're familiar with file compression programs like WinZip or Gzip, then you know that a compressed file can be drastically smaller than its uncompressed counterpart. For example, text files (which includes HTML, XML, JSON and many other common file formats of the Web) typically compress to less than 1/3 their original size. So with compression, Stash ends up sending much less data for files that would benefit from compression.

 

To be sure, compression does not benefit all types of files. The data in most photo (such as .png, .jpg, .gif), audio (.mp3, .wma, .ogg) and video files (.avi, .mov, .mpeg) is already so highly compressed that Stash compression adds little benefit. Nevertheless, enough file types do benefit from compression to create a significant performance improvement.

 

The second optimization Mozy created for this update is a new form of differential patching. We got some help from EMC's Data Domain group, a team who create amazing technology for on-premise backup and recovery systems. Their guidance helped Mozy engineers devise a way for Stash to send and receive just the updated parts of a file when you make changes to it, even if the file has shrunk or grown in size.

 

To understand how this works, let's step back and talk about how Stash would sync changes in previous versions. Whenever you changed a file, the sync engine would detect the parts of the file you had modified, and send just the parts of the file that you changed instead of the whole file. However, prior to this release, if your changes altered the size of the file, Stash needed to send any data in the file from wherever in the file the data had changed.

 

To use a grossly oversimplified analogy, consider a printed 10 page document. I'm talking about real paper here—10 pages with a paperclip to hold them together. If you were to insert a new page before page 4 in a 10 page document, Stash would upload everything from the change—the new page 4—to the end of the document (that is, pages 4 - 11). But in this analogy, the data for pages 5 - 11 didn't really change. They just shifted position, so Stash really should not have needed to send those changes.

 

Continuing with the same analogy, Stash 0.14 would handle the page addition much more efficiently. Instead of sending pages 4 - 11 in the changed document, Stash would now just sync the new page 4, and note that the other pages have all shifted one page higher. So in this extremely simplified example-by-analogy, Stash would now sync 6 fewer pages. (And, you can certainly extrapolate more complex examples that the new differential patching would handle just as elegantly. For example: "I updated page 2, inserted a new page 4, deleted page 6 and updated page 10." Stash 0.14 sends the just the changes.)

 

Of course, Stash syncs files, not paper documents. So now that you have the basic concept, let's use a real world example. Stash engineer Joe Cline conducted some interesting tests on MP3 files. Let's examine the tests he ran on one example file, Pink Floyd's "Time" from Dark Side of the Moon. Because we're dealing in small changes, we'll work in bytes. The original file size was 10,239,548 bytes.

  • Joe first changed some of the text-based tag metadata on the file, a change that did not affect the size of the file. As expected with such a minor change, both Stash 0.13 and 0.14 synced a very small update. Stash 0.13 synced 4,120 bytes. Stash 0.14 synced 2,476 bytes. The clear winner was Stash 0.14, but with a difference of less than 2K you probably would not notice it.
  • Next, Joe added album art to the file. Since the file did not already contain any album art, the added image grew the file by about 12,000 bytes. An important detail here is that MP3 files typically store tag metadata at the beginning of the file, before the audio data. (That means that Joe's change was like adding a new page 1 in my paper document analogy.) Stash 0.13 had to send 10,260,200 bytes to sync the update—effectively the whole MP3 all over again. But Stash 0.14 only sent 32,020 bytes. So in comparison, Stash 0.14 synced a fraction of a percent as much data for this particular change. (Of course, when I saw this, I immediately started down the track of, "Okay, and since Dark Side has 10 songs on it…." and, "And then, if I have a program modify my whole music library….")

MP3 files make a particularly powerful example for how 0.14 differs from 0.13 because of how MP3 files are structured, with tags at the beginning of the file (as noted in the test result above). Certainly not all types of files will produce such a dramatic result. In fact, similar to what we discussed with compression, there are certain types of files that cannot benefit from differential patching. In general, if the file's data is already compressed, differential patches don't help because the file data gets re-shuffled each time you save the file. For example, when you save your changes after recoloring a JPEG picture file, so much of the file gets re-written that any differential update is pretty much moot. Nevertheless, there are so many types files that do benefit from differential patching that overall Stash 0.14 should make a noticeable performance difference for those of you who rely on Stash swiftly sending and receiving file updates.

 

As a final thought, if you're still not clear on the difference between compression and differential patching, Joe's MP3 tests provide a good example for that, too. Unlike what we see with text files and other highly-compressible data, Joe found that the initial upload of MP3's got no significant benefit from compression. That's because the bulk of an MP3 file is the audio data, which is already compressed quite efficiently. So compression yielded no gain on MP3's, but when it comes to changes to tags on an MP3 file that is already in your Stash, the new differential patching algorithms yield a major efficiency gain.  

 

Conclusion

As I mentioned at the start of this lengthy post, Stash 0.14 completes a large body of work by Mozy R&D. We're eager to hear how it works for you. If you run any tests and find interesting results, please post a comment and reply. Also, you can find out about even more improvements in the release notes attached to this post.

 

As always, be safe and happy Stashing!

 

--Ted

Ted Haeger
Mozy Product Management
Wayward
Overlord Level 1
(648)

Re: Stash 0.14 - Better, Stronger, FASTER!

Ted,

 

Thanks for this detailed post concerning the Stash 0.14 update features.  As you probably know from previous posts, I'm one of those who forged ahead with (non-recommended and probably troublesome) Quicken .qdf files plus some other fairly large database files that update frequently.  

 

These files are a critical part of my Stash sync need.   I have seen a few sync problems but was generally able to work around them.  I look forward to working with the new release and the improved method of handling this type of file.  Thanks for the continuing Stash improvement ...

Wayward

"Five out of four people have a problem with fractions." ... unknown
0
Ted_Haeger
Moderator
(260)

Re: Stash 0.14 - Better, Stronger, FASTER!

Thanks, Wayward. We eagerly anticipate your reports on whether Lazy Sync and VBI improve performance for syncing Quicken files.

 

--T

Ted Haeger
Mozy Product Management
0
Owyn
Emperor Level 1
(734)

Re: Stash 0.14 - Better, Stronger, FASTER!


  • apdb, db, dbf, db3, sqlite - common databases types

 

Is "sq3" there too? For sqlite3 databases I use

 

 


It will do this until two hours have elapsed, and then sync the file even if it cannot get exclusive access.

but but... will it be successful? Cat Embarassed I mean if it can not get access to it

0
Ted_Haeger
Moderator
(260)

Re: Stash 0.14 - Better, Stronger, FASTER!

Owyn:

 

The operative word there would be "exclusive." When Stash cannot get exclusive access, it will wait for two hours, and then upload the file to ensure that you always have at least a backup from 2 hours ago.

 

Should we add sq3? There may be a registry value that you can use to add it to test it out. Interested?

 

--T

Ted Haeger
Mozy Product Management
Wayward
Overlord Level 1
(648)

Re: Stash 0.14 - Better, Stronger, FASTER!

 

Ted_Haeger wrote:

Thanks, Wayward. We eagerly anticipate your reports on whether Lazy Sync and VBI improve performance for syncing Quicken files.

 

--T


Ted,

 

I've been running the new version for several days with absolutely no problem.  My primary database types are Quicken .qdf and Microsoft Access .mdb. Previous problems have been with the Quicken files. Since installing the Stash update, I've been working it pretty hard to create frequent data changes over a short period of time.  

 

It has not been long enough to be truly definitive, but so far so good ... no sync delays or failures (despite my best efforts to create a problem).  Smiley Happy

Wayward

"Five out of four people have a problem with fractions." ... unknown
0
Owyn
Emperor Level 1
(734)

Re: Stash 0.14 - Better, Stronger, FASTER!


Ted_Haeger wrote:

Owyn:

 

Should we add sq3? There may be a registry value that you can use to add it to test it out. Interested?

 

--T


Sorry for long delay, forgot about this thread and remembered a while ago...

 

I'm absolutely interested in registry value, because I and probably others could have either their own formats or not-so-well-spread file formats in some number on their computers needing this.

 

I'm not fully sure if .sq3 is a popular extension but it's used in programs which many-many people use and I'd surely like it to be added.

0
RedHead
Student
(0)

Re: Stash 0.14 - Better, Stronger, FASTER!

Hi.

 

The update seems to be great, but I still didn't get it.

Any reason for it not to be updated?

Any option to manual update?

 

Thanks, Moshe.

0
andrewsk
Employee
(133)

Re: Stash 0.14 - Better, Stronger, FASTER!

Hi Owyn:

 

Thanks for suggesting sq3, we will add that to the default list for the 0.15 client.

 

And as for the registry info:

 

The format is simple.  If you specify this registry key it will replace the default list, so its best to keep the full list and then add more entries.

 

Here is an example .reg that adds .sq3, which will become the default in the next release.  To apply it save the 3 lines between the "---" into a file with the .reg extension and double click it, OR use "regedit" to navigate to HKEY_CURRENT_USER\Software\Mozy, Inc\Stash\config and add the value manually. 

 

---

Windows Registry Editor Version 5.00

[HKEY_CURRENT_USER\Software\Mozy, Inc\Stash\config]

"msync.lazylist"=".pst,.qdb,.mdb,.ldb,.mny,.qdf,.db3,.db,.apdb,.sqlite,.dbf,.tc,.sq3"
---

 

(Its also possible to set these values on Mac.  Running this command in a Terminal should do the trick:

 

defaults write com.mozy.Stash configvars -dict-add msync.lazylist  ".pst,.qdb,.mdb,.ldb,.mny,.qdf,.db3,.db,.apdb,.sqlite,.dbf,.tc,.sq3

 

 

 

 

andrewsk
Employee
(133)

Re: Stash 0.14 - Better, Stronger, FASTER!

Hi Moshe, 

 

The Mozy service runs from multiple datacenters.  The way we are introducing Stash 0.14 (and upcoming 0.15) to the beta is one datacenter at a time.  We started with a brand new datacenter, which is where most new Stash accounts are established.  Once your datacenter is upgraded we will be able to upgrade you to the more recent Stash client.  This is a gradual process impacting many servers and accounts so it may take several months to complete as we want the service to be running very smoothly for everyone during that transition.

 

-Andrew

Mozy R+D

 

 

0