When adding an activity to a day, can I choose which users in the trip can be added to it?

Yes – 

When creating or editing an activity for a trip, go to the “Travelers” section at the bottom of the activity. You can then add or remove specific users. You also have the option to select “All”, meaning all the users listed, or “None”, meaning none of the users are selected.