merge in trunk rev. 3426

copy trunk/freewrt r3264 into a new branch for making the nfo system
because upgrading the old adk branch would be a hell more work

fix dependency calculation

make dependency handling easier and fix some grave bugs in freeradius deps

add fwwif build module and a davfs2 fwwif module ☺

export a few more useful variables

fix

ever since moving the IPKG stamp files to bin/packages/ some things
were broken:
• make package=foo clean didn't clean the stamp files
• if the package version (or dashver) changed, the package did
  not get rebuilt automatically

this commit fixes that

merge base-files-arch into base-files, experimental, blame me if it breaks

new target “distclean” for packages

the treebreaker ☺

overhaul the fetch'n'checksum system to be more bsd-like
• make fetch, make checksum supported
• future expansion possibilities already prepared:
  – more distfiles than just one
  – check sizes of downloaded files (size matters!!!!!111)
  – more hashes than just MD5
  – recursive fetch/checksum targets
  – PERMIT_{SOURCES,DISTFILES}=Yes/«reason» – licence issues
• download.pl dies

quirks:
• GNU make only has a poor excuse for a .for loop
• some things (esp. toolchain and target/linux) now have
  a ${WRKDIR} where they didn't have it before
• ${WRKDIR}/.checksum_done – no comment… it was necessary
  because otherwise everything will get rebuilt every time
  (did I mention that GNU make sucks?)
• I had troubles tracking the dependency problems down
  because GNU make, unlike BSD make, doesn't have good debugging
• DISTFILES -> FULLDISTFILES
• DL_DIR -> DISTDIR
• PKG_SOURCE -> DISTFILES
• PKG_SOURCE_URL -> MASTER_SITES
• no @SF and @GNU any more, use BSD network.template syntax
  (feel free to add more master sites)
• GNU make doesn't really strip whitespaces from variables…
  (did I mention how much better BSD make is?)
• echo 'LOCAL_DISTFILES=/path/to/distfiles' >>${TOPDIR}/prefs.mk
  will save the need to download 'em, Closes: #102
• MASTER_SITES now *must* have the trailing slashes (there may be
  a few that don't and a few with double ones, please fix them)

I've tested that a normal build (unchanged .config from the default) works
on GNU/Linux. There's definitively room for improvement, but most of that
will come with the NFO system only. Fixes appreciated.

This work sponsored by AurISP, part of the NFO conversion project ☻

this tried to use the host nm

change the ipkg cookie as discussed in #FreeWRT
but move it to bin/packages/.stamps/ so that the usual and already
existing cleaning methods will get it in all cases and no manual
post-processing would be needed

THIS CHANGE REQUIRES A “make clean”!

• base-files/files/etc/init.d/rcK, base-files/files/etc/init.d/rcS:
  do correct shebang processing, like the BSD kernel and mksh: get
  first line of script to run if it begins with "#!", strip that and
  any leading space/tab off, then get shell name (terminates with
  space/tab/end of string) and shell argument quoting according to
  unix common proceedings; use /bin/sh as interpreter if none could
  be found
• mk/package.mk: ensure all init scripts are 0755 by default
• busybox: flesh out inetd init script, so that all init scripts in
  FreeWRT either can be found with -path \*/init.d/\* or -name \*.init
• other packages: add #!/bin/sh shebang line to init scripts¹, bump dashver

¹) change this to /bin/ash if you use ash-specific features that mksh
   can also do; never use ash-specific features that mksh cannot do (are
   there any?), change to /bin/mksh if you use mksh-specific features
   and add a run-time dependency on mksh in that case

autotools shortcoming (cross) – uClibc malloc(0) returns non-NULL indeed

I don't exactly know why wbx@ disabled rstrip.sh altogether in r2504,
but re-enable it, my images were getting way too big.

run some more scripts with ${MBSH}

this seems to fix libnotimpl build

by request of wbx@:

sort out the mess we had in ${TOPDIR}/staging_dir_${ARCH} and introduce
some new variables:
• ${STAGING_DIR} stays the same, but has a different location now
• ${STAGING_TOOLS} is where executables for the host system go
• ${STAGING_PARENT} is ${TOPDIR}/cross_${ARCH} and has three (at
  the moment) subdirectories: host (i.e. STAGING_TOOLS), target
  (i.e. STAGING_DIR) and pkg (used by package.mk, pkg-bottom.mk)
  but is an internal-use-only variable - please don't use it.

this builds through with a non-default config; more testing will follow

Implemented a new option to enable IPV6 support (wbx says OK). Making some compile tests, seems to work so far.

*tada*
complete rework of the kernel and kernel module build process.
more modular, very fast and simple (perl free implementation TM)

 * only choosen drivers are compiled in the ADK, significant faster
   compiles (just 10 minutes for a basic image on my X40 notebook for
   a WL500gp)
 * if the user changes something in the ADK, thats not a problem, new
   drivers are automatically compiled on next make run
 * only one simple "kernelconfig" for every board-kernel combination
 * two kind of configuration symbols:
   - FWRT_KERNEL prefix, for kernel features without packages
   - FWRT_KPACKAGE_KMOD prefix for kernel features or drivers with packages
 * very simple to add new drivers to the ADK, just add stuff to 
   target/linux/config/Config.in.* and mk/modules.mk
 * rebuild of firmware images without make clean if you change model/board
   (nearly, one bug with base-files still exist)
 * autogenerated ipkg control files  
 * first cleanup of special netfilter/iptables scripts (netfilter.mk is
  obsolete)
 * New menuconfig point ADK (for ccache, toolchain stuff, prefix,...)
 * package collection is disabled for now, it seems unmaintained. it is better
   to only have this in WIB, in my point of view
   
What is missing:
 * pcmcia support and tests on Linksys WRT54G3G
 * ip6tables 2.4/2.6
 * ebtables for 2.6
 * iptables package 
 * dependencies for shorewall/fwfirewall/iptables package
 * documentation for the developer handbook

Please test and report bugs!
 

more from the “coolz” departement:
• get rid of dependency on external ipkg and python stuff
• one distfile less too ;)
• while here, remove executable bits on more shell scripts

tested with a defaultconfig full bootstrap (cleandir+v)

provide host CFLAGS, CPPFLAGS, LDFLAGS in more than one way to the
scripts etc. – assists in fixing nfs-utils

commit part of the $PATH cleanup; this breaks in alsa (at least),
but a huge number of packages builds with it; I'll compile fixing
after I'm done with a change to the toolchain…

* Update tor to latest version (compile test only)
* Fix tor build with NDEBUG
* revert r2003, added -DNDEBUG again

Please do not remove NDEBUG, it helps very much for most packages. If problems occur without NDEBUG, please fix the specific package or report to me.

sorry dnehring@ but -DNDEBUG breaks the build of at least one
package that can't be built with it

* package.mk: pass the correct shared library runpath and the
  shared library link-time path to the linker (gcc); fixes most
  shlib dependency issues
  tested to build fine with approx. 80% of all packages; ok to
  commit now wbx@
* kpkg.mk: at least linux 2.4 modules are built with LD=ld and
  not LD=gcc, so -Wl,-foo must be replaced by -foo
  (this is the reason why it's -Wl,-rpath -Wl,/usr/lib and not
  -Wl,-rpath,/usr/lib in package.mk)

fix location of stamp files

n0-1@ says in Message-ID: <20070212103943.GB19308@nuty.aurisp.office>
that /var is a symbolic link to /tmp, so be it, I checked on my Asus,
thusly warn at package creation time if files are to be stored in any
of the ramdisk locations (/tmp /var, any more?).

Tested both with and without.

typo, my bad

* pkg-bottom.mk: use single quotes, not double quotes, for the text,
  as we don't use shell variables inside
* package.mk: it's not enough to check for /lib/* stuff in ${WRKINST},
  we must also check for it in ${IDIR_*} if we want to prevent certain
  overwrite-and-delete scenarios…

Request tg: move -DNDEBUG in CPPFLAGS. r1768 was wrong.

rework the 'which packages to build' logic according to
Message-ID: <Pine.BSM.4.64L.0701231812180.26685@herc.mirbsd.org>
and the rest of the thread

merged from branches/common-adk 1310:1685

the following packages are not converted yet:
* alsa
* asterisk
* pmacct
* zaptel
* elinks
* libgd
* lua
* mypackage
* openser
* php*
* snort*
* tmsnc

additionally, freeradius doesn't compile.

happy fixing :P

experimental: clean up stamp files
requested by eJunky, NOT to end up in 1.0 branch

move some makefiles to mk

GNU make can't do relative includes? OMFG. BSD make is so much better.

update-patches for tools/

revert r424, cf. Message-ID: <Pine.BSM.4.64L.0607270952540.24235@odem.66h.42h.de>

fix patch apply, files did not get expanded

* limit matching of patches to patch-* and *.patch
* if any patch contains an @@ -0,0 +n,m @@ chunk,
  touch ${WRKSRC}/.patched-newfiles to be aware of that

allow automatic copy of a hierarchy rooted in extra/
into ${WRKSRC} (and prevent these from being included
in patches)

agreed wbx@ on the phone

(cd package/foo; TOPDIR=../.. gmake update-patches) # should now work

IMPORTANT: gmake TOPDIR=../.. foo *does not work* (since it overrides
the correct value which otherwise gets pulled from prereq.mk after it
has been found from the environment-TOPDIR value and gets preference)

my journey to add, by request(!) of wbx@, a MirPorts-compatible
'make update-patches' target, starts here:
* add the target (disabled for now)
* copy the script

as discussed with wbx@ some 2-3 weeks ago, change
the BR2_ prefix to FWRT_

tested with a standard build (only the squashfs
image since the jffs2 didn't get built)

GNU make and templates are limiting the use of variables,
so, after HOURS of trying AND reading the texinfo documentation,
I decided to give up and move the WRKDIR spec outside of the template,
thus possibly yielding improper results, but hey it seems to work
(at least for the zlib package, will test full build now)

move some stuff out of that freaking template
I hate templates and GNU make in general

add defaults for PKG_SOURCE and fix download logic
more unbreaking to come

move the IDIR out of WRKBUILD and into WRKDIR, now that we use subdirs

* use WRKSRC for patch, not WRKBUILD
* fix clean target

XXX we should put the stamps into WRKDIR, i.e. use
XXX ${WRKDIR}/.prepared instead of ${WRKBUILD}/.prepared
XXX and get rid of the stamps in the stamps dir altogether

instead of setting WRKBUILD, set WRKSRC and let WRKBUILD?=WRKSRC like mirports
(WRKSRC isn't used anywhere though)

remove redundant code, and these mostlyclean/rebuild stuff
(the latter discussed with wbx@ last weekend)

* whitespace cleanup, while here
* move WRKDIR and DISTFILES definition into PKG_template
* define WRKDIR via parent BUILD_DIR instead of going up from WRKSRC
* use a sub-directory as WRKDIR, like BSD
* rename PKG_BUILD_DIR into WRKBUILD
* define WRKBUILD relative via WRKDIR
* rename PKG_INSTALL_DIR into WRKINST
* fix a few dependencies
* make use of common cases in the definition of WRKBUILD and
  WRKINST, removing many redundant lines
* use WRKDIR instead of BUILD_DIR in places where the latter
  was used previously
* while here, remove PKG_CAT stuff

this only affects packages, for now

almost entirely revert r296
more "cleanup after wbx" follows, after discussion in IRC

fix extracting of zip files

some files don't need our EXTRACT_CMD
empty DISTFILES will do

if PKG_CAT is unset we still want to be able to extract

small thinko regarding relative locations of packages

* new variables WRKDIR, DISTFILES (similar to MirPorts Framework)
  default to ${PKG_BUILD_DIR} and ${DL_DIR}/${PKG_SOURCE}, respectively,
  for the package/ case (more to come)
* instead of PKG_CAT and friends use an EXTRACT_CMD similar to
  the EXTRACT_CASES variable of MirPorts, only simpler

not tested yet, will do that ASAP

ok wbx@

allow CPIO distfiles

agreed wbx@

use pax to extract if it exists, instead of tar,
because it handles certain archives created by
GNU tar better (on BSD), no change on Linux

ok wbx@

add OpenWrt trunk revision 3830.