Jump to content

A few questions about your Helix backup strategy and your use of IRs


zooey
 Share

Recommended Posts

I've mentioned before that I've been working on an app to help with some aspects of IR management, presets too, a bit. It's coming along pretty well, but I thought I should check in with the community to see if its workflow fits with how you do things.

Apologies for the length of this msg, I couldn't figure out how to be much shorter and still clear.

The basic idea is that the app reads a nested directory of Helix backups and analyzes them. From there, it can show you what IRs are and aren't used and by how many presets, and what presets do and don't use IRs. Version 1 probably won't actually DO anything, only tell you what you've got.

At this time, it doesn't read setlist or bundle files, only actual IRs and presets, and because of that, it expects your backups to be laid out in a certain way.

 

Which brings us to my first question: How do you organize your backups?

What I do, and what the app currently requires is something like this:

  • firmware 2.21
    • 2017-11-05
      • IRs
      • Presets
        • Setlist Foo
        • Setlist Bar

It doesn't actually care what the directory structure above your IRs and Setlists directories looks like. Doing by firmware version then backup date makes sense to me, but it's not required. You tell it the root directory to look in, and it can scan for directories under there that contain both IRs and Setlists directories. Once you've done that, it can scan for new or changed files, so it doesn't have to process everything all over again.

It treats each directory containing both IRs and Presets directories as a backup set, whose name in the app is the path to it from the root, like "firmware 2.21/2017-11-05".

It assumes that...

  • Your IRs are directly inside the IRs dir
  • Each IR filename has a 3-digit numbered prefix, like "013 Allure 59_Tweed_P10N"
    • That prefix thing has been pretty standard in the community for a while now, since you need it to load them back into Helix after a firmware update etc.
  • The directories inside Setlists are named with setlist names
  • Your presets are directly inside those individual named setlist directories

Does that seem like it'll work for you? Alternatives I should consider?

 

Second question: How many backup sets do you have?

I'm just trying to get a ballpark idea. Of course over time there will be more and more, but still, I don't want to be completely in the dark here.

 

Third question: What sort of output would you like from this program?

The app itself is pretty good at showing you what you have and the relationships between presets and IRs. However, if you want to reorganize your IRs, like to get rid of unused ones and make room for new stuff, I thought you might want to export some usage info into Excel or something. You'd use that data to plan out your IR renames, and to know which presets should end up pointing to which new locations.

Of course we all expect Line 6 to completely obsolete this program, with much better IR management tools integrated right into the editor, but in the mean time, hopefully this will help folks out some. It helps me :)

 

Any other thoughts are welcome, of course.

Link to comment
Share on other sites

I would not, but maybe others would. I have a directory for my IR, and they are all renamed with the first 3 digits being the number location that they go in. Then when I have to re-set the unit after a firmware update it only takes a few seconds to restore.

Link to comment
Share on other sites

Thanks for the feedback Peter.

 

I think that 3-digit-prefix strategy is what most of us do. It's kind of required, given the current technology.

 

But that's not the problem this app is trying to solve, and it doesn't :)

 

What it IS for is to figure out which IRs are used, and by which presets. That's a necessary first step towards removing ones you're not using, to make room for new stuff. Similarly, once you know that the IRs in slots 17, 18, 42, 55, 63, and 87-110 aren't being used, you might want to juggle things around to make some contiguous blank areas to put the new stuff in. Otherwise, it gets spread all over, which makes auditioning them a pain, and probably isn't how you'd like them to end up.

 

The app can:

  • Show you which IRs aren't used by any presets in that backup
  • For each preset, show you which IRs it uses, and how they're hooked up (via snapshots, footswitches, directly to the IR block, etc.)

Make sense?

Link to comment
Share on other sites

The app can:

  • Show you which IRs aren't used by any presets in that backup
  • For each preset, show you which IRs it uses, and how they're hooked up (via snapshots, footswitches, directly to the IR block, etc.)

Make sense?

 

 

Yes, I could see that as very useful for a LOT of folks...

Link to comment
Share on other sites

 

Quote

 

 How do you organize your backups?

 

 

Im not too well organized. When my IRs get full i just export the whole list, date it, re-import the ones i KNOW i like, and start all over again. This usually leaves me about 100 new slots to start importing again. This also leaves a multude of problems unsolved, even with the fact that I number all of my IRs, which is a pain. Especially when the number is already there and all that REALLY needs done is for that existing number to be allowed as part of the export process....which also stilll leaves a bunch of problems unaddressed. 

 

Quote

 

 How many backup sets do you have?

 

 

3 right now. I suppose i'll start the process again right before the update which will give me 4....and if the update drags on much longer i could end up with more than that.I imagine with a dedicated tool this number can and will be reduced significantly. I could probably reduce it some just by being more organized, but being OCD organized takes away from the fun of playing guitar. Short term anyway.

Quote

 

 

What sort of output would you like from this program?

 

 

 

The most important thing for me would be knowing which IRs are and are NOT being used....or jus NOT is fine. This would preferably extend to the snapshots portion where i quite oftren change cabs. If im just nancing around in the rabbit hole, testing to see what AMPS work with which IRs, then sometimes there may be 8 different cabinets tied to a single preset.

 

The other most important thing would be for the IRs to import/export in the order they exist inside Helix, WITHOUT having to number them and/or alphabetize them.

 

The last most important thing would be to allow for more characters, so one could see the full detail of the IR file name without having to edit any text.

 

Quote

Does a tool like this seem like something you'd use? 

 

 

 

No question about it. I dont use HX cabs at all. This is one of the main things im hoping for everytime an update rolls around. 

Link to comment
Share on other sites

zooey, on 05 Nov 2017 - 6:42 PM, said:

 

I've mentioned before that I've been working on an app to help with some aspects of IR management, presets too, a bit. It's coming along pretty well, but I thought I should check in with the community to see if its workflow fits with how you do things.

Apologies for the length of this msg, I couldn't figure out how to be much shorter and still clear.

The basic idea is that the app reads a nested directory of Helix backups and analyzes them. From there, it can show you what IRs are and aren't used and by how many presets, and what presets do and don't use IRs. Version 1 probably won't actually DO anything, only tell you what you've got.

At this time, it doesn't read setlist or bundle files, only actual IRs and presets, and because of that, it expects your backups to be laid out in a certain way.

 

Which brings us to my first question: How do you organize your backups?

What I do, and what the app currently requires is something like this:

  • firmware 2.21
  • 2017-11-05
  • IRs
  • Presets
  • Setlist Foo
  • Setlist Bar

It doesn't actually care what the directory structure above your IRs and Setlists directories looks like. Doing by firmware version then backup date makes sense to me, but it's not required. You tell it the root directory to look in, and it can scan for directories under there that contain both IRs and Setlists directories. Once you've done that, it can scan for new or changed files, so it doesn't have to process everything all over again.

It treats each directory containing both IRs and Presets directories as a backup set, whose name in the app is the path to it from the root, like "firmware 2.21/2017-11-05".

It assumes that...

  • Your IRs are directly inside the IRs dir
  • Each IR filename has a 3-digit numbered prefix, like "013 Allure 59_Tweed_P10N"
    • That prefix thing has been pretty standard in the community for a while now, since you need it to load them back into Helix after a firmware update etc.
  • The directories inside Setlists are named with setlist names
  • Your presets are directly inside those individual named setlist directories
Does that seem like it'll work for you? Alternatives I should consider?

 

Second question: How many backup sets do you have?

I'm just trying to get a ballpark idea. Of course over time there will be more and more, but still, I don't want to be completely in the dark here.

 

Third question: What sort of output would you like from this program?

The app itself is pretty good at showing you what you have and the relationships between presets and IRs. However, if you want to reorganize your IRs, like to get rid of unused ones and make room for new stuff, I thought you might want to export some usage info into Excel or something. You'd use that data to plan out your IR renames, and to know which presets should end up pointing to which new locations.

Of course we all expect Line 6 to completely obsolete this program, with much better IR management tools integrated right into the editor, but in the mean time, hopefully this will help folks out some. It helps me :)

 

Any other thoughts are welcome, of course.

My backups directory structure

 

My main director is the firmware version and Editor version: (these have been different upon occasion). Under that are four directories - IRS, Bundles, Setlists, and Presets. When I make new backups I append a new version number to the file rather than doing an overwrite. The files are dated by the OS. Here is a skeletal example.

 

Firmware 2.21 - Editor 2.21

IRS

ir1

ir2

etc....

Bundles

bundle_1

bundle_2

Setlists

user1_1

user1_2

user2_1

templates_1

etc...

Presets

zep_preset_1

zep_preset_2

floyd_preset

morse_preset

etc...

 

How many backup sets do you have? I keep all my backups so the answer to this question would be a whole lot.

 

What sort of output would you like from this program? Have not put a whole lot of thought into this but I would like to be able to at least see the following

  • Each IR that is in use and its corresponding "slot" number in the Helix IR list. Not sure how you can do this by parsing the backups.
  • The name of each preset using an IR grouped by IR name and slot number
  • A list of IRs not in use would be very useful
  • A way of comparing the actual name of an IR that a preset use to point to as well as the slot number. Not sure how to do this from parsing backups either.
  • Some kind of a compare utility that enabled you to see if two IRs were exactly the same would be helpful given that you have to currently rename them due to the Helix's short namespace. Utilities that do compares are already out there but having it built into an IR tool would be helpful. Even better would be if Line6 would extend the namespace such that most renames become unnecessary.
Link to comment
Share on other sites

Thanks for taking the time to lay all that out HO.

 

Some questions etc...

  • Re how I can know the slot number of an IR, don't you use the 3-digit naming prefix strategy that's been discussed on this board (and elsewhere)? How else can you put your IRs back in their slots after an upgrade? For example: 025 OH 412 MES-ST V60 MDRN came from slot 25. That prefix is how the app "knows" what slot an IR file was in.
  • Not sure I get what you mean by "When I make new backups I append a new version number to the file rather than doing an overwrite." Are you saying that...
    • All your preset backups for a given editor and firmware version are in one 'Presets' directory? You don't keep the presets for a given backup or setlist separate? If that's the case, this app won't work well for you, because it can't tell which files are in a given backup, or within that, in a given setlist.
    • (What I think I understand to be) your organization makes sense for a library, as opposed to a backup. Are you thinking of your actual setlist and bundle files as your backups, and preset files as a library?
      Back story: It hasn't happened to me (knock silicon), but I know some people have had a corrupted preset lock up their Helix, until they figured out which one it was and didn't load it. For that reason, I've always backed up individual presets as well as setlists. I restore from setlist files when there's no reason not to, it's just simpler and quicker, but I always make individual preset backups too. I keep them in separate directories by setlist, because it's easier for me to think about them, and reduces the odds of duplicate names.
      Since my backups are laid out like I described above, it's easy for the app to know which presets and IRs belong to a given backup, and which presets are in a given setlist.
      Not that you're doing anything wrong of course. This is exactly why I asked these questions. The app fits MY workflow, but it might be a lot less useful to others who work differently, at least until it can read actual setlist and/or bundle files, which it currently can't.
  • Re the types of output you want, I meant to ask what exports you might want, as distinct from what it shows on screen. On screen, for a given backup, the app can show you the things bulleted below (among other things).
    Given those onscreen views, do you care about exports, like to get the data into Excel maybe, to plan juggling your IRs around (before the app can do that itself)?
    • The IRs that are and aren't used
    • Presets that do and don't use IRs, or that use more than one different one
    • For a given preset in a backup, it can show you all the IRs it uses, and how they're hooked up (snapshots, footswitches, directly to the IR block, etc)
  • Agree that an IR compare tool would be great.
    • Doing that is complicated by the way Helix exports IRs. It embeds the name it has in Helix, including the typical 3-digit prefix, as metadata in the actual .wav file. For that reason, the exact same IR in slot and and slot 17 won't be identical when exported. You'd have to strip the metadata out to recognize that they're the same, which I do plan to do, but not in version 1.
    • You might want to do that stripping anyway. Helix honors the Title metadata field (if I remember right) when you import IRs, so if you rename your IRs in the file system (there are tools to do that in bulk), without removing or updating that Title field, the result is super confusing. They end up in renumbered filename order, since the file system sorted them that way, but with old-numbering names based on the old titles.
    • Really what you want is a tool that can examine your whole IR library, and for a given IR in a backup, recognize the original source file, with its original too-long-to-fit-in-Helix name. I hope to get there too. Besides the title metadata thing, to compare Helix exports to your library of IRs that haven't been through the import/export process, you'd need to exactly duplicate the audio conversion Helix does -- truncation, bit reduction, etc. I've tried some likely settings using external tools, but haven't matched it. Yet. I've also asked some of the publicly visible Line 6 guys for details, no response. I do hope I'll suss it out at some point.

But all those futures aside, I really want to get this out there, even though it doesn't do everything we'd like yet. Working on it...

Thanks again for your input.

Link to comment
Share on other sites

 

Quote

 

Im not too well organized. When my IRs get full i just export the whole list, date it, re-import the ones i KNOW i like, and start all over again. This usually leaves me about 100 new slots to start importing again. This also leaves a multude of problems unsolved, even with the fact that I number all of my IRs, which is a pain. Especially when the number is already there and all that REALLY needs done is for that existing number to be allowed as part of the export process....which also stilll leaves a bunch of problems unaddressed. 

 

Quote

 

3 right now. I suppose i'll start the process again right before the update which will give me 4....and if the update drags on much longer i could end up with more than that.I imagine with a dedicated tool this number can and will be reduced significantly. I could probably reduce it some just by being more organized, but being OCD organized takes away from the fun of playing guitar. Short term anyway.

Quote

 

 

The most important thing for me would be knowing which IRs are and are NOT being used....or jus NOT is fine. This would preferably extend to the snapshots portion where i quite oftren change cabs. If im just nancing around in the rabbit hole, testing to see what cabs work with which IRs, then sometimes there may be 8 different cabinets tied to a single preset.

 

The other most important thing would be for the IRs to import/export in the order they exist inside Helix, WITHOUT having to number them and/or alphabetize them.

 

The last most important thing would be to allow for more characters, so one could see the full detail of the IR file name without having to edit any text.

 

Quote

 

 

 

No question about it. I dont use HX cabs at all. This is one of the main things im hoping for everytime an update rolls around. 

 

Thanks Will. I answered HO first because some of those answers apply to your less specific situation too; see that novel too.

 

Re keeping track of IR order without manually numbering them, unfortunately, until the app can read and manipulate bundle files I can't do that.

 

Another thing it won't do in version 1, but I hope is closer on the horizon, is to provide an interface to reorder (read - rename) IRs, delete or replace unused ones, etc, and automatically update a set of preset files to point to the new locations.

 

But I want to ship, for once, instead of waiting to build the perfect thing.

 

Ultimately, the real Line 6 Helix app should store IR hashes (kind of a digital fingerprint of the file contents) in each preset, rather than slot numbers, then all this renaming-reordering biz just goes away. There could even be a central online registry of every IR that any participating user had ever scanned, so the app could recognize the original source of IRs you don't even have, in a preset someone gave you.
 
But like I want to ship, for once, instead of waiting to build the perfect thing.
Link to comment
Share on other sites

I do this with a few scripts I wrote. I backup my presets to a directory by exporting them all at once. And I have a "staging" directory for my IRs that I drag into the Helix. The IR order in the Helix is the same as the directory order.

 

One script shows what IRs are used and by how many presets along with which IRs are not used. More verbose output options show which presets specifically use each IR.

 

The input to this script is simply a file that lists the IRs in their slot number order, and the directory that holds all my presets. Since I have a staging directory for my IRs, that list is easy to make - it's simply the directory listing saved to a file.

 

Another script I have is slightly more advanced. If I want to add a bunch of new IRs or remove some, I'll just make a new staging directory that reflects the new IR list. That script takes the old IR list, the new IR list, and the directory of presets. It then examines each preset and updates the IR slot number to reflect the new IR list order.

 

Then it's an easy matter wipe my Helix (like is done with a firmware update), and simply drag all the IRs from the new list folder into the Helix along with the updated presets.

 

Everything links up like it should and while IR slot numbers changed, all presets were updated to match, so it all works out the way it should.

 

This not only makes firmware updates easy because everything is handy in folder to simply drag back into the Helix without worrying about IR slot numbers or having to rename IRs with slot prefix numbers. It also makes adding or removing a bunch of IRs easy as well, again without concern that IR slot numbers may change.

Link to comment
Share on other sites

Sounds very cool.

 

So you don't number your IRs, you just use their natural name sort order, and let your script adjust your presets for changes?

 

Does it regen the preset file JSON from scratch, or do text replacements to update for new IR locations?

 

Are you looking for IR usage by IR blocks only, or also in snapshots and footswitches?

 

What language is it written in?

 

Regardless if any of that, very cool.

Link to comment
Share on other sites

Sounds very cool.

 

So you don't number your IRs, you just use their natural name sort order, and let your script adjust your presets for changes?

 

Does it regen the preset file JSON from scratch, or do text replacements to update for new IR locations?

 

Are you looking for IR usage by IR blocks only, or also in snapshots and footswitches?

 

What language is it written in?

 

Regardless if any of that, very cool.

 

 

1) Yes, just the natural name order. It would actually be a little easier if the Mac didn't sort directory listings kind of weirdly. If it didn't do that, I wouldn't require the directory listing of the IR directory to be in a file, I could just scan the actual IR staging directory and sort it. But if you sort the directory listing using a standard alpha-numeric sort, that's NOT the same order that a Mac will present the files in the Finder window. Even so, even it that wasn't an issue, it would only save about 10 seconds in making the input files.

 

2) It regens the json file from scratch, so I load it in using json parsing to create the data structure as represented in Python dictionaries - really no work on my part there, the json.load(file) method which does all the heavy lifting. Then I follow the data structure just enough to find the IR blocks, and specifically what slot number is assigned.

 

3) I'm looking for IR blocks and and also those assigned with snapshots. I'm not looking for footswitches, though, since I'm not using them for that, but I do use snapshots and that changes things a bit. I'd have to experiment and do a footswitch assignment to see how that changes the json and update my search method accordingly. I've had this scripting since before the snapshot capability was available in the firmware, and I remember having to update my script to account for that.

 

4) Written in Python.

 

The actual loading, updating, and writing back out the presets is very easy in a language like Python, it doesn't take much code at all, a handful or two of lines of code does all that. And while the code for finding the IR blocks in their various forms is not a lot, either, that's the key - basically just decoding the data structure L6 uses to store them. Anyone can get a feel for what that's like just by looking at the json structure they use. It's not hard, but any changes Line 6 does to the preset format in the future and I'll need to adapt. Like when snapshots were introduced. Hopefully in the future they retain the json (or other easily readable and parseable format) for the presets. If they were proprietary binary something or other what I did would not have been practical for me.

 

Thanks!

 

Unfortunately, my method is a couple of command-line scripts that use a couple of data files and the preset directory for input, and another preset directory to write out the updated presets (I use a subdir of the actual preset directory for that - I don't replace them in place just in case something goes wrong I don't destroy the originals). So it's not the most user-friendly - definitely not a nice GUI or anything that would be generally sharable. Sounds what you're working on might be, though - if so, keep going! These scripts have saved me a ton of time over the past two years, so having something like this be user friendly and available for a broad group of folks would likely be very welcome. Mine works great for me, but it's not polished or easy enough to use to be generally sharable, I suspect.

Link to comment
Share on other sites

Interesting strategy about the naming. Not sure how Windows file sorting compares to what I'd see from code. Might be worth investigating to lose the numbering biz.

 

I was thinking v1.1 of my thing might well have UI to reorder, renumber and rename a set of IRs, then adjust the presets once that was settled, but you could have longer names without the numbering. At least some of the weirdness when IR metadata doesn't match a renumbered file name should go away too. It'd be great if we could keep the names like they were the original distribution, but they're often to long anyway, and quite possibly not unique across makers.

 

Also, I like being able to control IR order as it is in Helix, like to keep a brand's stuff together.

 

Will think on it, thanks for the ideas.

Link to comment
Share on other sites

As for me I keep things pretty simple.  I've considered doing something like this myself but have been hesitant to take it on due to a lack of free time to monkey with it and the fact it would probably have a limited lifetime given that Line 6 will certainly be addressing this issue at some point or another.

 

I viewed it simply as an analysis tool which could be pointed at a directory containing individual patches and at a directory containing IRs and give me a resulting list of exceptions...or IRs that have no references in any of the patches in the directory.  Since I use the three digit naming convention, all I would need is a list of slot numbers containing unreferenced IR slots.  That would be enough to figure out how I wanted to handle things.

 

In my case this makes the most sense given that my setlists are totally dynamic as each patch is a different song, so my setlists are constantly changing.

 

I'd be afraid anything more than that would eventually be completely wasted effort.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

 Share

×
×
  • Create New...