Understanding VCS in a multi-subproject workflow (Bzr)

Discussion in 'Programming & Software Development' started by perrymitchell, Feb 12, 2013.

  1. perrymitchell

    perrymitchell Member

    Joined:
    Jan 1, 2007
    Messages:
    742
    Location:
    Helsinki, Finland
    Hi,

    I've started using Bazaar for a lot of code versioning, and I have a project I'd like to version with it. The project has one root code base (containing the base of the project), along with several "subprojects" that contain extensions and modifications of a small amount of the base code, but still greatly depend upon it.

    I want to version this with Bazaar, in what I expect to be an SVN-like manner. I can't for the life of me figure out how to get this going - I want to be able to pull down the base code and a certain subproject to work on, make changes to either separately or together (whichever), and commit them back to the server.

    I'm not particularly looking for a bazaar-how-to on this, but maybe some input from anyone that's done things like this with VCS.

    Cheers!
     
  2. cyclonite

    cyclonite Member

    Joined:
    Mar 7, 2002
    Messages:
    824
    Location:
    NSW 2009
    Maybe have a read of this page:
    http://doc.bazaar.canonical.com/bzr.dev/en/user-guide/shared_repository_layouts.html

    If I've understood your problem correctly, I don't think bzr is able to handle the granularity of sub-projects like you describe.


    This problem is generally not solved well in any version control system.
    Git has submodules (a repository which is aware of the fact that some of it's directories are actually other repositories too) which allow you to checkout all of them with one command, or ensure all are up to date etc.


    I think you have two options:
    1. Ask yourself whether you really need this level of granularity
    2. Write yourself some shell scripts which can manage the sub-repositories to reduce the pain
     
  3. OP
    OP
    perrymitchell

    perrymitchell Member

    Joined:
    Jan 1, 2007
    Messages:
    742
    Location:
    Helsinki, Finland
    Hi cyclonite, thanks for the response. I've been bashing my head against the wall on this for a while now.

    To clarify my intentions, imagine a 2 part system. Part 1 is the core, which is the major part of the system. Part 2 is more of an interface, which would need to be changed from "client to client". Say I have 2 versions of Part 2 that I want to interchange from said clients, which both require Part 1, and require it to be up to date.

    I want to, in one instance, pull down part 1 and part 2 for client A, but then be able to pull down part 1 and part 2 (different version) for client B and still be able to make changes and commit.

    I guess I should be looking at other options rather than just trying to manage the entire workflow in a VCS. Unfortunately Git doesn't suit my needs, for a number of reasons:
    - It has poor cross-platform support for clients
    - Doesn't natively support a centralised workflow
    - More complicated than I require

    I'm from an SVN background myself, though SVN has a poor track record when it comes to larger merges with multiple checkouts/branches etc. Hence why I'm leaning on bzr at the moment. Though the ideas I'm throwing around here aren't based on a particular piece of technology, I'm open to suggestions based on the strengths and weaknesses of these pieces of software.

    Cheers
     
  4. cyclonite

    cyclonite Member

    Joined:
    Mar 7, 2002
    Messages:
    824
    Location:
    NSW 2009
    Yep makes sense. I'm not trying to push Git, but it is my personal preference. Linux, OS X and Windows are all pretty well supported.

    Anyway back to the problem. My suggestion would be to:
    1) Keep your 2 separate repositories (Core1, Core2)
    2) Write yourself some wrapper scripts to
    a) Check out Core 1​
    b) Check out Core 2 for the specified client​
    3) Core 2 can be implemented as:
    a) A single Core2 repository which you branch on a per client basis: e.g. Core2/client1, Core2/client2​
    b) A repository per client: a bit more tedious, but if your customisations are large (lots of static assets) then you are probably going to have a painful time trying to keep some generic Core 2 which works for all customers anyway​
     
  5. GregDude

    GregDude Member

    Joined:
    Aug 12, 2002
    Messages:
    1,310
    Location:
    Brisbane
    A few thoughts...

    When you have a core code base and a bunch of projects using that core, store the entire project / file set in a single repository. Edit code in a local user branch, merge local changes into a shared branch if there are multiple developers. Backup your local branch by pushing it to a remote location.

    If you have variations of the core code, you will probably need to duplicate the core code for that set of related projects. What I mean is that you can't use branches to easily maintain diverging projects. So you would not make Project1, branch and make Project 2. Instead you would make Projects1 and 2 in the same branch. You can use the merge logic to share code between completely different repositories and branches if needed.

    I think Bazaar and similar DVCSs can be used in a central style workflow quite well. I switched from Bazaar to Git over a year ago for personal projects and SVN to Git for work. Using SmartGit or TortoiseGit, there is very little difference from using SVN or other VCS.

    Bazaar is file centric and simple to use from the command line. Modern Git is content centric and a little harder to use from the command line. I hate the command line and in both cases GUIs are available which will avoid the command interface almost entirely.
     
  6. OP
    OP
    perrymitchell

    perrymitchell Member

    Joined:
    Jan 1, 2007
    Messages:
    742
    Location:
    Helsinki, Finland
    Thanks guys.

    It's looking increasingly painful to try and setup :tongue:, but at least I'm starting to wrap my head around it.

    cyclonite, I like your suggestion of keeping the two separate repositories and just checking out the correct client. Wrapping that in a couple of scripts would be simple as. What about intertwining file structure? Though I guess I'd never be working with both in the same location, but only when testing. I'm developing a web system, which might put those thoughts in to context.

    Thanks GregDude. I'm having to relearn a little bit as I go, but I'm helping set this up for a few other developers here for collaboration. I need something easy and cross platform - Just a couple of reasons I've stuck with Bazaar as the solution. There's a long list of reasons why I've settled on it, but I won't go in to them. I'm also pushing for them to use the command line :p, as we do a lot of work on dedicated ubuntu web servers.

    Thanks for the input :)
     
  7. OP
    OP
    perrymitchell

    perrymitchell Member

    Joined:
    Jan 1, 2007
    Messages:
    742
    Location:
    Helsinki, Finland
    I've got another question:

    Has anyone had any experience using split or join in Bazaar? Perhaps playing with subtrees?

    I'm just thinking now, what if I want to contain only one client folder within the parent project folder and commit on both separately?
     
  8. GregDude

    GregDude Member

    Joined:
    Aug 12, 2002
    Messages:
    1,310
    Location:
    Brisbane
    Never did split and join perrymitchell, so can't help there.

    I have to warn you, one of the main reasons I switched from Bazaar, which I really did like, was instability. I'm not sure if they have fixed the issues now, but these are my reasons and worth warning a fellow developer:
    1) Internal memory bug which caused 'out of memory' error even with small commits.
    2) Complete inability to commit large files, even if they would be stored inefficiently.
    3) Corruption of local repository, only solution was to delete local and pull fresh remote copy.
    4) GUI while functional is developing very slowly, in part due to the death of the original main developer.

    In the mean time, Git commands improved and there are excellent GUIs for every platform. Git will never be as simple and 'designed for humans' as Bazaar, but there is little need for the average developer to ever touch its raw interface. I've never used Mercurial to compare, however it was a lecture on Mercurial that inspired me to use Bazaar years ago.

    Another little note is that while one of Bazzar's killer features is tracked renaming, since Git tracks content rather than files, tracking file content as it changes file name or directory is implied and automatic. If you want to investigate Git, I think this video was quite a good one: http://www.youtube.com/watch?v=ZDR433b0HJY

    I'm not trying to convince you to use one DVCs over another, but having come from Bazaar and reluctantly moved to Git, thought I should share some experience.
     
  9. OP
    OP
    perrymitchell

    perrymitchell Member

    Joined:
    Jan 1, 2007
    Messages:
    742
    Location:
    Helsinki, Finland
    That's fair enough. I ended up using 2 repositories in the end (for the contained data). So the master would be checked out, and a slave *beside* it, then I'd create a symlink in the first to the second - Getting Bazaar to ignore the symlink in that folder. The only reason I don't version the symlink is windows compatibility. This method has worked incredibly well, and is much cleaner and easier to explain :).

    In regards to Git - I understand why it has such a huge following.. And the pickup on a technology like that by the community is great for it's progression. I think Bazaar has come a long way, from reading both your comment and other stories I've found.

    I assume that either you were using an older, more unstable version of Bazaar or your problems may have been specific to your environment. I say this only because I haven't noticed any such issues with our setup here (running Ubuntu workstations with the repo on an Ubuntu 12.04 server). We've been doing both branches and checkouts at amazing speeds (for only about 20-30k files). The files range in size from a couple of kb up to several hundred meg. We haven't had any crashes or corruptions. We also back up the repo server weekly (at this stage).

    While we're still at a "teething" stage with our repo setup I feel quite safe relying on Bazaar to meet our needs. I'm able to be very flexible with Bazaar in terms of supporting our developers that come from different backgrounds. We have the flexibility to work in the checkout-style manner, whereas we wouldn't with git. Because we all work in the same location more or less, I feel the more distributed nature of Git is beyond our needs. Like I may have mentioned before, I don't like the behavior of SVN's merging... So I would have ended up using Bazaar or possibly mercurial.

    I will say that the GUI version of Bazaar seems quite good, although still lacking. It's a no-nonsense approach but I still think it could be improved. For now I'll get everyone using the command line version only.

    Thanks for all the input! It's been ages since I've done any decent code versioning... It's always addictive to get back in to it :D
     

Share This Page

Advertisement: