[OFBiz] Dev - Upgrade Proposal
Adrian Crum
adrianc at hlmksw.com
Thu Sep 23 18:56:17 EDT 2004
David E. Jones wrote:
>
> Adrian,
>
> For manual updates I'd never recommend exporting and importing if it
> can be avoided, that is a real pain (though sometimes may be
> necessary). For the most part we do changes in-place in the database,
> often with code written using the Entity Engine and such. For instance,
> right now we have services written to do various updates, as well as
> general maintenance routines for doing other things such as increasing
> column sizes (ie from 40 to 100 characters or whatever), re-creating
> foreign keys, etc.
>
> -David
Let's say I have an OFBiz installation running for a year. I don't
download any updates during that time, but the newer versions have made
successive changes to the database. Now there's a new feature in OFBiz
I'd like to have. You're saying that I can download the latest version
and it will update the database automatically? Including data migration?
>
>
> On Sep 23, 2004, at 3:11 PM, Adrian Crum wrote:
>
>> David E. Jones wrote:
>>
>>> Think about the changes that are made to the database. Each one
>>> needs some sort of upgrade/migration process. These are somewhat
>>> independent of code versions and what not, though directly relate
>>> to revisions of the entitymodel.xml files, and of course private
>>> changes to those files.
>>> I guess the main point I'm getting at is things should be managed as
>>> a list of database changes over time (with revision numbers and
>>> dates and such), and the process for an update of each one.
>>
>>
>> I like the "list over time" idea. My suggestion achieved the same
>> result, only it got there differently.
>>
>> Instead of using the database, how about using an xml file that lists
>> the database changes over time and the service to run for each one?
>> Another xml file could contain the component's upgrade history.
>>
>> As was mentioned previously, the upgrades can be chained so that any
>> version can be upgraded to any newer version. The chaining ability
>> could use the upgrade history xml file as a guide.
>>
>>> The thing about the database is, what would use it? The migrations
>>> really should all be done manually, you really don't want to risk
>>> messing up your data, especially if there is anything unique about
>>> your particular instance.
>>
>>
>> Manual migrations are an option. But what about those who would
>> prefer something more automatic?
>>
>> The "I just updated OFBiz - how do I migrate my existing data?"
>> message pops up with regularity here. Each time the question is
>> asked, the solution offered is to export the database to XML, modify
>> the XML, start the new version of OFBiz, then import the XML into the
>> new schema. I don't know about the other users of OFBiz, but I'm
>> thinking that process is a pain in the butt. Many of us are
>> intelligent, talented programmers - why do we put ourselves through
>> that? Data migration should be handled by the software - at least as
>> an option.
>>
>> User modifications to the database are easily handled by extensions
>> to the packaged upgrade service.
>>
>>> And either way, whether it a human or a process checking to see
>>> what needs to be done they should look at the actual table
>>> structures, NOT at some revision number somewhere which is error
>>> prone and could be set wrong or cleared accidently or who knows what.
>>
>>
>> That's true with any process within OFBiz. Any code that relies upon
>> a piece of data will break if that data is accidently altered.
>>
>>> Just some thoughts really....
>>> -David
>>
>>
>> I really appreciate your participation!
>>
>> Honestly, it scares me to think of deploying OFBiz here knowing that
>> I'll have to export/modify/import data every time the data structure
>> changes. I don't have time for that!
>>
>> Even if there's no interest from the community, I'll still implement
>> an upgrade system here.
>>
>>
>>> On Sep 23, 2004, at 1:35 PM, Adrian Crum wrote:
>>>
>>>> Where would you suggest we keep it?
>>>>
>>>>
>>>> David E. Jones wrote:
>>>>
>>>>> Just a quick word on this... I don't think storing version
>>>>> information in the database is a good idea at all. In general this
>>>>> is a code thing and keeping stuff like this in a database is
>>>>> error prone and difficult track history of and such.
>>>>> -David
>>>>> On Sep 22, 2004, at 9:57 AM, Adrian Crum wrote:
>>>>>
>>>>>> Daniel L. Kunkel wrote:
>>>>>>
>>>>>>> Hi Adrian...
>>>>>>> Hey, I like your ideas, just I'd rather stay with just one SVN
>>>>>>> code version for tracking everything... Although having the
>>>>>>> update code in each component makes a lot of sense.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> We can accomodate both. I'll propose some additional fields for
>>>>>> the Version entity.
>>>>>>
>>>>>> I'm not fond of the idea of using the SVN code version because
>>>>>> that ties the Upgrade feature to SVN. We can have a field
>>>>>> called REPOSITORY_CODE to accomodate it, however.
>>>>>>
>>>>>> I also see a problem with using a single update code. If the
>>>>>> component-level upgrade system is implemented as I propose, then
>>>>>> EVERY component is going to upgrade itself every time an
>>>>>> individual component changes. Maybe I'm missing something here.
>>>>>> *shrug* Having component-level versions allows only the updated
>>>>>> components to trigger the Upgrade service.
>>>>>>
>>>>>> Some component developers may want to trigger the Upgrade
>>>>>> service based on dates. I'd like to add a date/time field to the
>>>>>> Version entity - LAST_UPGRADE_DATE. It would contain the date
>>>>>> that the local copy of the component was upgraded.
>>>>>>
>>>>>> So, to summarize, I'd like to suggest this Version entity:
>>>>>>
>>>>>> COMPONENT, CHAR, 30 // Contains the component's name
>>>>>> CURRENT_VERSION, CHAR, 10 // The currently installed version
>>>>>> REPOSITORY_CODE, CHAR, 10 // The currently installed SVN code
>>>>>> LAST_UPGRADE_DATE, DATE/TIME // Date of last update/upgrade
>>>>>>
>>>>>> If the entity contains an OFBIZ record, then components can
>>>>>> upgrade based on that record - as Daniel suggested.
>>>>>> Alternatively, they can upgrade based upon their own version.
>>>>>>
>>>>>>> -----------------
>>>>>>> David, Andy
>>>>>>> Won't the POS systems need this automatic updating feature
>>>>>>> since they each have a database? Thanks
>>>>>>> Date sent: Tue, 21 Sep 2004 16:50:16 -0700
>>>>>>> From: Adrian Crum <adrianc at hlmksw.com>
>>>>>>> Organization: Hallmark Southwest Corporation
>>>>>>> To: OFBiz Development <dev at lists.ofbiz.org>
>>>>>>> Subject: Re: [OFBiz] Dev - Upgrade Proposal
>>>>>>> Send reply to: OFBiz Development <dev at lists.ofbiz.org>
>>>>>>> <mailto:dev-request at lists.ofbiz.org?subject=unsubscribe>
>>>>>>> <mailto:dev-request at lists.ofbiz.org?subject=subscribe>
>>>>>>>
>>>>>>>> Exactly! Thanks BJ!
>>>>>>>>
>>>>>>>> Trust me - I don't want to make more work for David or Andy.
>>>>>>>> I'm just thinking of adding one entity to the OFBiz core and
>>>>>>>> some agreed upon guidelines for implementing an upgrade "system."
>>>>>>>>
>>>>>>>> I'm hoping we'll only need David and Andy to monitor the ideas
>>>>>>>> and insert advice where needed.
>>>>>>>>
>>>>>>>> For the sake of illustration, let's say we agreed upon an
>>>>>>>> upgrade system and got it implemented next week. No additional
>>>>>>>> work would need to be done at this point since OFBiz is at a
>>>>>>>> "version point" - as some people put it.
>>>>>>>>
>>>>>>>> Let's say part of the eCommerce component changes in October,
>>>>>>>> and the changes require a restructuring of data. The only
>>>>>>>> additional work required would be to write code in the
>>>>>>>> eCommerce component's update service to port the existing data
>>>>>>>> over to the new format, and update the CURRENT_VERSION field
>>>>>>>> of the Version entity. (There would be one Version record per
>>>>>>>> component, by the way.)
>>>>>>>>
>>>>>>>> That's what I was trying to point out earlier - upgrades
>>>>>>>> become more feasible because they've been broken down into
>>>>>>>> smaller pieces.
>>>>>>>>
>>>>>>>> The way I see it, there are just a few things that need to be
>>>>>>>> done to get the ball rolling on this idea:
>>>>>>>>
>>>>>>>> 1. Add the Version Entity to the OFBiz core
>>>>>>>> 2. Define an ofbiz-component.xml element that describes
>>>>>>>> the component's upgrade service
>>>>>>>> 3. Define a common interface for the component upgrade
>>>>>>>> service
>>>>>>>> 4. Modify the OFBiz startup code so that it looks for
>>>>>>>> the new ofbiz-component.xml Upgrade element and calls the
>>>>>>>> component's upgrade service. If the element doesn't exist,
>>>>>>>> then nothing is done.
>>>>>>>>
>>>>>>>> Once those 4 steps are accomplished, the groundwork is
>>>>>>>> complete. Now each component is responsible for its own
>>>>>>>> upgrades and there is a system in place to call the
>>>>>>>> component's upgrade service when OFBiz starts.
>>>>>>>>
>>>>>>>> Component developers can get as slick as they want with a
>>>>>>>> component's upgrade service - even taking it to the point where
>>>>>>>> it downloads the latest files from a repository/website/local
>>>>>>>> folder.
>>>>>>>>
>>>>>>>>
>>>>>>>> bjfree at free-man.net wrote:
>>>>>>>>
>>>>>>>>> I agree David.
>>>>>>>>> I believe he was not referring to redoing the structure of ofbiz.
>>>>>>>>>
>>>>>>>>> I took it as, since different people work on different
>>>>>>>>> components, that the
>>>>>>>>> version within a component would be independent of the ofbiz
>>>>>>>>> top Version. In reality I see this now, so there is not much
>>>>>>>>> difference, except how it is
>>>>>>>>> implemented.
>>>>>>>>> sort of like that you use jre 1.4 but the version of ofbiz is
>>>>>>>>> 3.0.
>>>>>>>>>
>>>>>>>>> Also I believe he is suggestion that the people that work on
>>>>>>>>> a particular
>>>>>>>>> component, develop the testing for that component.
>>>>>>>>> This would take the load off you and Andy, and make a
>>>>>>>>> community thing, as
>>>>>>>>> you mention this project is.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> -----Original Message-----
>>>>>>>>> From: David E. Jones [mailto:jonesde at ofbiz.org]
>>>>>>>>> Sent: Tuesday, September 21, 2004 10:30 PM
>>>>>>>>> To: OFBiz Development
>>>>>>>>> Subject: Re: [OFBiz] Dev - Upgrade Proposal
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Wow, I guess some of this sounds nice, but probably not feasible.
>>>>>>>>>
>>>>>>>>> I guess one main thing you should notice about OFBiz is that
>>>>>>>>> we maintain the entire thing as a single package and we
>>>>>>>>> DON'T track versions of different components. I'm not sure
>>>>>>>>> what you had in mind along these lines, but certainly don't
>>>>>>>>> have enough resources to manage multiple packages and
>>>>>>>>> revisions like that. Heck, we have even avoided splitting
>>>>>>>>> out the framework from the applications because even two
>>>>>>>>> packages would be much much more difficult to maintain. To
>>>>>>>>> get an idea of the resources we have (or don't have)
>>>>>>>>> available to do this, check out our email response delays
>>>>>>>>> lately...
>>>>>>>>>
>>>>>>>>> IFF we get enough interest in an upgrade pack service (the one
>>>>>>>>> I mentioned would be $5000/year), we would be tracking
>>>>>>>>> specific data model changes and the SVN revision or OFBiz
>>>>>>>>> version where they changed. Now that we are using SVN it is
>>>>>>>>> a LOT easier. We can say things like "if you deployed
>>>>>>>>> before SVN rev 3192 and want to update to 3192 or later then
>>>>>>>>> you will need to run this migration service or process". So,
>>>>>>>>> we will have tested processes (that are as automated as
>>>>>>>>> possible) and tools for doing these things for all data
>>>>>>>>> model changes, and instructions for how to apply them (meant
>>>>>>>>> to be done by a DBA and/or application engineer, or someone
>>>>>>>>> with similar qualifications).
>>>>>>>>>
>>>>>>>>> I don't want to burst anyone's bubble as many of these ideas
>>>>>>>>> sound great, but I don't see them as being very feasible at
>>>>>>>>> this point...
>>>>>>>>>
>>>>>>>>> -David
>>>>>>>>> _______________________________________________
>>>>>>>>> Dev mailing list
>>>>>>>>> Dev at lists.ofbiz.org
>>>>>>>>> http://lists.ofbiz.org/mailman/listinfo/dev
>>>>>>>>> _______________________________________________
>>>>>>>>> Dev mailing list
>>>>>>>>> Dev at lists.ofbiz.org
>>>>>>>>> http://lists.ofbiz.org/mailman/listinfo/dev
>>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Dev mailing list
>>>>>>>> Dev at lists.ofbiz.org
>>>>>>>> http://lists.ofbiz.org/mailman/listinfo/dev
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Daniel
>>>>>>> *-.,,.-*"*-.,,.-*"*-.,,.-*"*-.,,.-*"*-.,,.-*"*-.,,.-*"*-.,,.-*"*-
>>>>>>> .,,.-
>>>>>>> Have a GREAT Day!
>>>>>>> Daniel Kunkel DanielKunkel at BioWaves.com
>>>>>>> BioWaves, LLC
>>>>>>> 14150 NE 20th Street, Suite 121
>>>>>>> Bellevue, WA 98007
>>>>>>> 800-734-3588 425-895-0050
>>>>>>> http://www.BioWaves.com http://www.IonToothbrush.com
>>>>>>> *-.,,.-*"*-.,,.-*"*-.,,.-*"*-.,,.-*"*-.,,.-*"*-.,,.-*"*-.,,.-*"*-
>>>>>>> .,,.-
>>>>>>> _______________________________________________
>>>>>>> Dev mailing list
>>>>>>> Dev at lists.ofbiz.org
>>>>>>> http://lists.ofbiz.org/mailman/listinfo/dev
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Dev mailing list
>>>>>> Dev at lists.ofbiz.org
>>>>>> http://lists.ofbiz.org/mailman/listinfo/dev
>>>>>
>>>>>
>>>>> --------------------------------------------------------------------
>>>>> -- --
>>>>> _______________________________________________
>>>>> Dev mailing list
>>>>> Dev at lists.ofbiz.org
>>>>> http://lists.ofbiz.org/mailman/listinfo/dev
>>>>
>>>>
>>>> _______________________________________________
>>>> Dev mailing list
>>>> Dev at lists.ofbiz.org
>>>> http://lists.ofbiz.org/mailman/listinfo/dev
>>>
>>> ----------------------------------------------------------------------
>>> --
>>> _______________________________________________
>>> Dev mailing list
>>> Dev at lists.ofbiz.org
>>> http://lists.ofbiz.org/mailman/listinfo/dev
>>
>> _______________________________________________
>> Dev mailing list
>> Dev at lists.ofbiz.org
>> http://lists.ofbiz.org/mailman/listinfo/dev
>
>
> ------------------------------------------------------------------------
>
>
> _______________________________________________
> Dev mailing list
> Dev at lists.ofbiz.org
> http://lists.ofbiz.org/mailman/listinfo/dev
More information about the Dev
mailing list