Remove WordPress Default Categories And Tags Taxonomies Or Unregister

If you would only like to remove it from the WP Admin menu use the below code.

function wpsnipp_remove_default_taxonomies(){
   global $pagenow;

   register_taxonomy( 'post_tag', array() );
   register_taxonomy( 'category', array() );

   $tax = array('post_tag','category');

   if($pagenow == 'edit-tags.php' && in_array($_GET['taxonomy'],$tax) ){
      wp_die('Invalid taxonomy');
   }
}
add_action('init', 'wpsnipp_remove_default_taxonomies');

If you want to completely unregister them use the below code:

add_action('init', function(){
   global $wp_taxonomies;
   unregister_taxonomy_for_object_type( 'category', 'post' );
   unregister_taxonomy_for_object_type( 'post_tag', 'post' );
   if ( taxonomy_exists( 'category'))
      unset( $wp_taxonomies['category']);
   if ( taxonomy_exists( 'post_tag'))
      unset( $wp_taxonomies['post_tag']);
   unregister_taxonomy('category');
   unregister_taxonomy('post_tag');
});