If you would only like to remove it from the WP Admin menu use the below code.
function wpsnipp_remove_default_taxonomies(){
global $pagenow;
register_taxonomy( 'post_tag', array() );
register_taxonomy( 'category', array() );
$tax = array('post_tag','category');
if($pagenow == 'edit-tags.php' && in_array($_GET['taxonomy'],$tax) ){
wp_die('Invalid taxonomy');
}
}
add_action('init', 'wpsnipp_remove_default_taxonomies');
If you want to completely unregister them use the below code:
add_action('init', function(){
global $wp_taxonomies;
unregister_taxonomy_for_object_type( 'category', 'post' );
unregister_taxonomy_for_object_type( 'post_tag', 'post' );
if ( taxonomy_exists( 'category'))
unset( $wp_taxonomies['category']);
if ( taxonomy_exists( 'post_tag'))
unset( $wp_taxonomies['post_tag']);
unregister_taxonomy('category');
unregister_taxonomy('post_tag');
});