If you uninstall stuff with apt-get remove sometimes Debian does not removes config files and they also take up space.

One approach is to use apt-get purge.

An alternative is as follows:

This command lists all the packages which are removed but still have config files on your system:

dpkg --list | grep '^rc '
 
rc binutils     2.20.1-15           The GNU assembler, linker and binary utilities
rc dbus         1.2.24-3            simple interprocess messaging system
rc dpkg-dev     1.15.8.5            Debian package development tools
rc erlang-base  1:14.a-dfsg-2       Erlang/OTP virtual machine and base applications
rc fakeroot     1.14.4-1            Gives a fake root environment

Now, just to make sure check the output and then remove the config files with this command:

dpkg --list | grep '^rcb' | awk '{ print $2 }' | xargs dpkg -P