Buddhism and Being Vegan

I have been wanting to write a post about Buddhism & Being Vegan for a while but have not really known how to put it together and get across my opinions clearly. I am still not sure that I will manage it but I thought I would give it a go.
Firstly let me say that one of the most important principals in Buddhism that I see is to never knowingly do harm, which includes any living creature. This obviously lends itself to a vegan diet, however the very first Buddhists were not even vegetarian. I can see you wondering about this as I did the first time I heard it, but the first Buddhists relied on hand outs for food and would literally go from house to house begging for food. At this point it was literally eat what you are given otherwise you are wasting food and an animal had died in vain.
This is where believe that Buddhists and vegans differ and why I find it difficult to call myself vegan. Now my ideas of being vegan may have been warped by people I have come in contact with and also by misconceptions that I have seen in the media but as far as I am aware vegans believe that no animal should ever work for a human. However I believe that as a Buddhist it is not wrong for an animal to work as much as it is wrong for a human to work, all that matters is if that animal is treated with love.
I come down more on the Buddhist side of this argument I have to say. For example if an animal is well looked after and cared for and their wool is removed humanely and there is no suffering involved at all then I have absolutely no problem with using that wool to make things. The problems only arise when things are mass produced and the animals are killed and abused, then it becomes unacceptable.
Another area where I have an issue is with eggs. I know that vegans do not eat eggs at all and with the current farming techniques I completely understand that, however I have seen vegans saying that taking the egg from a chicken is cause problems for the chicken and I have a massive issue with this. The main reason for my issues is that when I was a child as a family we kept chickens, they had a little hen house and in the morning my parents let them out and they could roam in our garden and actually in the whole village. In the evening my parents would go out and they would be back in the hen house ready for bed. Every morning we had fresh eggs and we had very happy chickens that lived long lives and had no problems. Chickens completely abandon an egg that is not fertilised and they are not sitting on. If ever a hen was sitting on their eggs then they obviously got to keep it. In my view by not eating these eggs we were actually wasting perfectly good food and this is actually something I find hard to justify.
Don't get me wrong this is not an anti vegan post as I actually am eating a vegan diet at the moment and doing my best to avoid animal products in most things. I just think that sometime things are not black and white and that the most important thing should be whether we as humans are causing suffering to an animal. If the answer to the question is no then I see no problem with using animal products or with working with animals. These are just my opinions and I think that it is important that every person makes a decision they can live with and that it sits well with them. No other person can tell you what is right, it is something that everyone needs to decide for themselves. I am always interested in hearing others opinions so please do share with me.