Building quantum network nodes based on neutral silicon vacancy centers in diamond