[mc1322x] Contiki tree resync

[Jon Smirl]

On Fri, Oct 30, 2009 at 10:12 AM, Mariano Alvira <mar at devl.org> wrote:
> On Thu, Oct 29, 2009 at 11:00:42PM -0400, Jon Smirl wrote:
>>
>> It would be best to sync up with Contiki CVS once every week or so.
>> It's not that we need the code from Contiki that often, it's so that
>> we'll quickly become aware of changes to Contiki that could render
>> this code unmergable.
>>
>> If you'd been syncing once a week you would have quickly noticed the
>> CVS keyword expansion issue. I was already familiar with the problems
>> caused by keyword expansion and knew how to fix it.
>
> Well unfortunately the phrase "If you'd been <...> you would have
> <...>" matches to a lot of things in this project let alone in my
> life. But certainly we both agree that more syncs are better than no
> syncs (which was the real issue with the keywords --- that and
> somebody decided that the "highly recommended option -k" for git
> cvsimport should be off by default). I'm open to suggestions on how to
> streamline this and get more people involved. I won't be stingy with
> server access.

Git is very different than CVS, with git no one else needs server
access.  Linus is the only person with access to his tree.

Everyone else makes patches in their own local git repositories. When
these patches are ready they get posted to the email list (there are
git commands for this). Everyone can comment on them, etc. When they
get into final form you ok them an apply them to the main tree, then
push the result to the server. There are git commands for getting the
patches out of the emails too. That's what patchworks does.

You should do this with your own changes. Do all of your own work in a
private git branch on your local machine. When it is ready post it to
the email list. Then apply just this posted diff to the mainline
branch.  Doing it that way avoids copying all of your intermediate
trial and error revisions into the mainline history.

For really big, messy changes a developer makes a public git branch in
a repository they own. You can then use git commands to pull the delta
into your local repo, approve it, and then push it out.  This project
is too small for that mode. An example of this is merging the x86 tree
into linus' mainline. There may be 1,000 patches in the merge.

Before you approve something, you should make a new branch, apply the
patch and check it out. If you don't like it, send some email comments
and delete the test branch. If you like it, merge the test branch into
your main branch and push the result out. You function as a
gatekeeper. It's up to you to do some basic testing before accepting
things into your mainline. Linus does that pretty much full time, but
his project is a little larger.

In our case this is complicated by the existence of the CVS
repository. When something has been in your mainline for a while and
we're pretty sure it is working well, it needs to be pushed into
Contiki CVS. You'll need a CVS account to do that.

After committing to CVS you need to resync your CVS tracking branch
and merge the CVS tracking branch into your mainline. At that point
git will find the duplicated patches and cancel them out. If they
don't cancel you get merge conflicts. That may force you into another
cycle of committing to CVS and resycning.

This sounds complicated but it really isn't. It's just a sequence of
things you need to do to keep everything in sync. The only time
everything gets complicated is when there are merge conflicts. Those
mean that two people edited the same place at the same time.  You have
to manually fix those up. Merge conflicts cause merge races where the
two developers rush to be first to commit and transfer the burden of
doing the merge onto the other party.

If you could get Contiki to switch to git you could eliminate the need
for you to do the maintainer role. Adam would end up doing it instead.
You'd become a trusted lieutenant like GregKH is for Linus.

>
> But back to the syncing and merging... the merges should be
> trivial. All our stuff should be in contiki/cpu/mc1322x and
> contiki/platform/mc1322x. For the most part, the newest version of
> files outside of these should always be taken. For the places where
> this isn't true we either need to eliminate them on our end (e.g. the
> quick hacks that I did --- see conflicts in examples) or, if they
> result from a lack of generality on Contiki's side, then we need to
> push a fix upstream. As far as I know the only change in the later
> category is to contiki/core/sys/rtimer.h, which now that I'm looking
> at it, still probably has issues with the type casting.
>
> With that said, however, there are many things in Contiki that are not
> generalized enough to work over all platforms. (e.g. coffee, sd card
> drive, energest --- I'm sure there are more). There are also missing
> things that are, in my opinion, essential and people hack around them
> in a way that only works for their platform (e.g channel changing,
> power level, CPU sleep, etc...). I feel this is the big
> merging/syncing issue that is preventing a lot of progress.

CVS makes it hard to rearchitect things. There's no easy way to push
out a repo with a new core API and then get the ARCH maintainers to do
fixes for their ARCHs and then commit everything simultaneously.

Maybe they can be talked into doing development in git and using
sourceforge CVS to simply track the git repo.

>
> As an easy example of this, take channel changing. Right now that is a
> platform hack ("If I had been <less lazy> then I wouldn't have <to
> write this example>"). The less lazy thing would be to add this to the
> contiki radio driver (a straighforward thing to do) and push that
> change upstream.
>
> For a hard example, take CPU sleeping, which is essential to make a
> low power sensor node. This is hard to generalize across
> platforms. Every processor sleeps differently and has different
> capabilities. Even the mc1322x has four major modes + plus a bucket of
> minor options + many differnet ways to wake up. (sleep vs. doze and
> CPU retention vs no retention). So what are we to do? The C64 guys
> certianly don't care. And most of the Sky guys seem to be happy with
> just turing off the radio, but that only gets you to the mA range of
> average current --- not the tens of uA you need to really make a good
> sensor.

For my application I need to get the TCPIP support working (either V4
or V6). I've been playing with V6 and 6lowpan. I keep getting bogged
down because I'm simultaneously working on a new CPU, with new
hardware, new protocol (6lowpan), etc. Right now I'm trying to figure
out why wireshark isn't fulling decoding 6lowpan packets. If I can't
decode the packets it is hard to see what is wrong with them.

>
> All that aside, and back to the merging issue --- without a set of
> tests there is nothing to say that the merge is any good. Contiki
> lacks this. Time spent on a set of contiki apps designed to test core
> Contiki functions across platforms would be a really well spent. Then
> you could generate a list of all the things that work and don't work
> for each platform.
>
>> You don't need two repositories, use branches.  CVS import should be
>> one branch, then your two versions of the code would be two other
>> branches. Change your .git/config to push all branch heads.
>>
>> [remote "digispeaker"]
>>         url = ssh://....
>>         fetch = +refs/heads/*:refs/remotes/digispeaker/*
>>         push = +refs/heads/*
>>
>> When the branches are in the server they'll automatically show in gitweb.
>
> Thanks. I'll look into this.
>
> -Mar.
>



-- 
Jon Smirl
jonsmirl at gmail.com