Restrict access to certain LDAP group

Is it possible to filter LDAP users based on group or organisational unit? At the moment I add my “ou=Users” to the filter which is fine but this adds all my users.

I want to be able to restrict the search to perhaps “ou=mySubUsers,ou=Users”, or search/add users that are only members of a particular group.