Oct 31 07:29:44.860618 ip-10-0-130-48 systemd[1]: Starting Kubernetes Kubelet... Oct 31 07:29:45.310327 ip-10-0-130-48 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 31 07:29:45.310327 ip-10-0-130-48 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Oct 31 07:29:45.310327 ip-10-0-130-48 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 31 07:29:45.310327 ip-10-0-130-48 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 31 07:29:45.315859 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.315778 2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 31 07:29:45.315859 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.315860 2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 31 07:29:45.315930 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.315865 2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 31 07:29:45.315930 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.315868 2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 31 07:29:45.315930 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.315871 2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 31 07:29:45.315930 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.315875 2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 31 07:29:45.315930 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.315878 2577 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 31 07:29:45.315930 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.315881 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 31 07:29:45.315930 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.315884 2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 31 07:29:45.315930 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.315887 2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 31 07:29:45.315930 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.315889 2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 31 07:29:45.315930 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.315892 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 31 07:29:45.315930 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.315895 2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 31 07:29:45.315930 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.315897 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 31 07:29:45.315930 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.315900 2577 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 31 07:29:45.315930 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.315902 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 31 07:29:45.315930 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.315905 2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 31 07:29:45.315930 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.315907 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 31 07:29:45.315930 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.315910 2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 31 07:29:45.315930 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.315912 2577 feature_gate.go:386] unrecognized feature gate: EVPN Oct 31 07:29:45.315930 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.315915 2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 31 07:29:45.315930 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.315917 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 31 07:29:45.316417 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.315920 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 31 07:29:45.316417 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.315924 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 31 07:29:45.316417 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.315926 2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 31 07:29:45.316417 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.315929 2577 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 31 07:29:45.316417 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.315932 2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 31 07:29:45.316417 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.315935 2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 31 07:29:45.316417 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.315938 2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 31 07:29:45.316417 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.315941 2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 31 07:29:45.316417 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.315943 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 31 07:29:45.316417 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.315946 2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 31 07:29:45.316417 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.315949 2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 31 07:29:45.316417 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.315951 2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 31 07:29:45.316417 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.315954 2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 31 07:29:45.316417 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.315956 2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 31 07:29:45.316417 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.315958 2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 31 07:29:45.316417 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.315962 2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 31 07:29:45.316417 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.315967 2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 31 07:29:45.316417 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.315970 2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 31 07:29:45.316417 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.315972 2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 31 07:29:45.316862 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.315975 2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 31 07:29:45.316862 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.315977 2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 31 07:29:45.316862 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.315980 2577 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 31 07:29:45.316862 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.315982 2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 31 07:29:45.316862 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.315984 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 31 07:29:45.316862 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.315987 2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 31 07:29:45.316862 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.315989 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 31 07:29:45.316862 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.315991 2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 31 07:29:45.316862 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.315994 2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 31 07:29:45.316862 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.315996 2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 31 07:29:45.316862 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.315998 2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 31 07:29:45.316862 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316002 2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 31 07:29:45.316862 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316005 2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 31 07:29:45.316862 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316007 2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 31 07:29:45.316862 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316011 2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 31 07:29:45.316862 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316013 2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 31 07:29:45.316862 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316016 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 31 07:29:45.316862 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316018 2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 31 07:29:45.316862 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316020 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 31 07:29:45.317309 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316023 2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 31 07:29:45.317309 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316025 2577 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 31 07:29:45.317309 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316028 2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 31 07:29:45.317309 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316030 2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 31 07:29:45.317309 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316032 2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 31 07:29:45.317309 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316035 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 31 07:29:45.317309 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316037 2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 31 07:29:45.317309 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316040 2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 31 07:29:45.317309 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316042 2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 31 07:29:45.317309 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316045 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 31 07:29:45.317309 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316047 2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 31 07:29:45.317309 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316050 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 31 07:29:45.317309 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316053 2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 31 07:29:45.317309 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316055 2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 31 07:29:45.317309 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316058 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 31 07:29:45.317309 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316060 2577 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 31 07:29:45.317309 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316062 2577 feature_gate.go:386] unrecognized feature gate: Example2 Oct 31 07:29:45.317309 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316067 2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 31 07:29:45.317309 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316071 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 31 07:29:45.317852 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316074 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 31 07:29:45.317852 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316077 2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 31 07:29:45.317852 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316079 2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 31 07:29:45.317852 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316082 2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 31 07:29:45.317852 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316085 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 31 07:29:45.317852 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316088 2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 31 07:29:45.317852 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316090 2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 31 07:29:45.317852 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316093 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 31 07:29:45.317852 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316096 2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 31 07:29:45.317852 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316099 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 31 07:29:45.317852 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316102 2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 31 07:29:45.317852 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316104 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 31 07:29:45.317852 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316107 2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 31 07:29:45.317852 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316109 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 31 07:29:45.317852 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316112 2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 31 07:29:45.317852 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316114 2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 31 07:29:45.317852 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316117 2577 feature_gate.go:386] unrecognized feature gate: Example Oct 31 07:29:45.317852 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316119 2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 31 07:29:45.317852 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316121 2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 31 07:29:45.318644 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316124 2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 31 07:29:45.318644 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316126 2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 31 07:29:45.318644 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316129 2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 31 07:29:45.318644 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316131 2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 31 07:29:45.318644 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316133 2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 31 07:29:45.318644 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316136 2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 31 07:29:45.318644 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316141 2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 31 07:29:45.318644 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316144 2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 31 07:29:45.318644 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316936 2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 31 07:29:45.318644 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316943 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 31 07:29:45.318644 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316947 2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 31 07:29:45.318644 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316951 2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 31 07:29:45.318644 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316955 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 31 07:29:45.318644 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316958 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 31 07:29:45.318644 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316961 2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 31 07:29:45.318644 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316963 2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 31 07:29:45.318644 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316966 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 31 07:29:45.318644 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316968 2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 31 07:29:45.318644 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316972 2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 31 07:29:45.319185 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316974 2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 31 07:29:45.319185 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316977 2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 31 07:29:45.319185 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316980 2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 31 07:29:45.319185 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316982 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 31 07:29:45.319185 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316985 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 31 07:29:45.319185 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316988 2577 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 31 07:29:45.319185 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316991 2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 31 07:29:45.319185 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316993 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 31 07:29:45.319185 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316996 2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 31 07:29:45.319185 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.316998 2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 31 07:29:45.319185 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317001 2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 31 07:29:45.319185 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317003 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 31 07:29:45.319185 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317006 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 31 07:29:45.319185 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317008 2577 feature_gate.go:386] unrecognized feature gate: Example Oct 31 07:29:45.319185 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317011 2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 31 07:29:45.319185 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317014 2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 31 07:29:45.319185 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317016 2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 31 07:29:45.319185 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317018 2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 31 07:29:45.319185 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317021 2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 31 07:29:45.319185 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317023 2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 31 07:29:45.319742 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317026 2577 feature_gate.go:386] unrecognized feature gate: EVPN Oct 31 07:29:45.319742 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317029 2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 31 07:29:45.319742 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317032 2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 31 07:29:45.319742 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317034 2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 31 07:29:45.319742 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317037 2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 31 07:29:45.319742 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317039 2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 31 07:29:45.319742 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317043 2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 31 07:29:45.319742 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317045 2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 31 07:29:45.319742 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317048 2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 31 07:29:45.319742 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317050 2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 31 07:29:45.319742 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317053 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 31 07:29:45.319742 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317055 2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 31 07:29:45.319742 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317058 2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 31 07:29:45.319742 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317060 2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 31 07:29:45.319742 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317063 2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 31 07:29:45.319742 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317066 2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 31 07:29:45.319742 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317068 2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 31 07:29:45.319742 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317071 2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 31 07:29:45.319742 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317073 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 31 07:29:45.319742 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317075 2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 31 07:29:45.320401 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317078 2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 31 07:29:45.320401 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317080 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 31 07:29:45.320401 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317083 2577 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 31 07:29:45.320401 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317086 2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 31 07:29:45.320401 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317088 2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 31 07:29:45.320401 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317091 2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 31 07:29:45.320401 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317093 2577 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 31 07:29:45.320401 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317096 2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 31 07:29:45.320401 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317100 2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 31 07:29:45.320401 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317103 2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 31 07:29:45.320401 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317106 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 31 07:29:45.320401 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317109 2577 feature_gate.go:386] unrecognized feature gate: Example2 Oct 31 07:29:45.320401 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317112 2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 31 07:29:45.320401 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317115 2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 31 07:29:45.320401 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317117 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 31 07:29:45.320401 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317120 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 31 07:29:45.320401 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317123 2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 31 07:29:45.320401 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317125 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 31 07:29:45.320401 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317128 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 31 07:29:45.320868 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317130 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 31 07:29:45.320868 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317133 2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 31 07:29:45.320868 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317136 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 31 07:29:45.320868 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317138 2577 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 31 07:29:45.320868 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317141 2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 31 07:29:45.320868 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317146 2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 31 07:29:45.320868 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317150 2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 31 07:29:45.320868 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317153 2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 31 07:29:45.320868 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317156 2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 31 07:29:45.320868 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317158 2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 31 07:29:45.320868 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317161 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 31 07:29:45.320868 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317163 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 31 07:29:45.320868 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317165 2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 31 07:29:45.320868 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317168 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 31 07:29:45.320868 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317171 2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 31 07:29:45.320868 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317173 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 31 07:29:45.320868 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317175 2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 31 07:29:45.320868 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317178 2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 31 07:29:45.320868 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317181 2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 31 07:29:45.321331 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317183 2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 31 07:29:45.321331 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317185 2577 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 31 07:29:45.321331 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317188 2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 31 07:29:45.321331 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317190 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 31 07:29:45.321331 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317193 2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 31 07:29:45.321331 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317195 2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 31 07:29:45.321331 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317198 2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 31 07:29:45.321331 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317200 2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 31 07:29:45.321331 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317203 2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 31 07:29:45.321331 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317206 2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 31 07:29:45.321331 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317209 2577 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 31 07:29:45.321331 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317211 2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 31 07:29:45.321331 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317213 2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 31 07:29:45.321331 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317216 2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 31 07:29:45.321331 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317221 2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 31 07:29:45.321331 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317224 2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 31 07:29:45.321331 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.317226 2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 31 07:29:45.321331 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318139 2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 31 07:29:45.321331 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318151 2577 feature_gate.go:386] unrecognized feature gate: Example2 Oct 31 07:29:45.321331 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318156 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 31 07:29:45.321862 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318169 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 31 07:29:45.321862 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318174 2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 31 07:29:45.321862 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318178 2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 31 07:29:45.321862 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318182 2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 31 07:29:45.321862 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318188 2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 31 07:29:45.321862 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318194 2577 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 31 07:29:45.321862 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318199 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 31 07:29:45.321862 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318203 2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 31 07:29:45.321862 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318207 2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 31 07:29:45.321862 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318211 2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 31 07:29:45.321862 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318215 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 31 07:29:45.321862 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318219 2577 feature_gate.go:386] unrecognized feature gate: EVPN Oct 31 07:29:45.321862 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318228 2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 31 07:29:45.321862 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318233 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 31 07:29:45.321862 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318237 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 31 07:29:45.321862 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318241 2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 31 07:29:45.321862 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318246 2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 31 07:29:45.321862 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318250 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 31 07:29:45.321862 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318254 2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 31 07:29:45.321862 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318258 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 31 07:29:45.322327 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318263 2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 31 07:29:45.322327 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318268 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 31 07:29:45.322327 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318273 2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 31 07:29:45.322327 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318277 2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 31 07:29:45.322327 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318281 2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 31 07:29:45.322327 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318289 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 31 07:29:45.322327 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318296 2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 31 07:29:45.322327 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318301 2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 31 07:29:45.322327 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318305 2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 31 07:29:45.322327 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318309 2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 31 07:29:45.322327 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318313 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 31 07:29:45.322327 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318317 2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 31 07:29:45.322327 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318321 2577 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 31 07:29:45.322327 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318326 2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 31 07:29:45.322327 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318330 2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 31 07:29:45.322327 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318334 2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 31 07:29:45.322327 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318338 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 31 07:29:45.322327 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318342 2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 31 07:29:45.322327 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318371 2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 31 07:29:45.322327 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318376 2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 31 07:29:45.322861 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318380 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 31 07:29:45.322861 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318385 2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 31 07:29:45.322861 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318389 2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 31 07:29:45.322861 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318393 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 31 07:29:45.322861 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318397 2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 31 07:29:45.322861 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318401 2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 31 07:29:45.322861 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318405 2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 31 07:29:45.322861 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318410 2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 31 07:29:45.322861 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318416 2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 31 07:29:45.322861 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318421 2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 31 07:29:45.322861 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318430 2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 31 07:29:45.322861 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318434 2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 31 07:29:45.322861 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318438 2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 31 07:29:45.322861 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318443 2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 31 07:29:45.322861 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318448 2577 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 31 07:29:45.322861 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318452 2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 31 07:29:45.322861 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318456 2577 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 31 07:29:45.322861 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318461 2577 feature_gate.go:386] unrecognized feature gate: Example Oct 31 07:29:45.322861 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318466 2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 31 07:29:45.322861 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318470 2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 31 07:29:45.323368 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318474 2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 31 07:29:45.323368 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318479 2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 31 07:29:45.323368 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318483 2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 31 07:29:45.323368 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318491 2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 31 07:29:45.323368 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318496 2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 31 07:29:45.323368 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318502 2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 31 07:29:45.323368 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318507 2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 31 07:29:45.323368 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318511 2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 31 07:29:45.323368 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318515 2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 31 07:29:45.323368 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318521 2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 31 07:29:45.323368 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318527 2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 31 07:29:45.323368 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318532 2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 31 07:29:45.323368 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318537 2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 31 07:29:45.323368 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318541 2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 31 07:29:45.323368 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318550 2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 31 07:29:45.323368 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318554 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 31 07:29:45.323368 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318559 2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 31 07:29:45.323368 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318563 2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 31 07:29:45.323368 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318568 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 31 07:29:45.323839 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318572 2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 31 07:29:45.323839 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318578 2577 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 31 07:29:45.323839 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318582 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 31 07:29:45.323839 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318586 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 31 07:29:45.323839 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318590 2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 31 07:29:45.323839 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318593 2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 31 07:29:45.323839 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318597 2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 31 07:29:45.323839 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318602 2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 31 07:29:45.323839 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318611 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 31 07:29:45.323839 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318616 2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 31 07:29:45.323839 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318620 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 31 07:29:45.323839 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318628 2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 31 07:29:45.323839 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318632 2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 31 07:29:45.323839 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318636 2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 31 07:29:45.323839 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318639 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 31 07:29:45.323839 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318644 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 31 07:29:45.323839 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318648 2577 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 31 07:29:45.323839 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318652 2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 31 07:29:45.323839 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318657 2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 31 07:29:45.324331 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318661 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 31 07:29:45.324331 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318670 2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 31 07:29:45.324331 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318674 2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 31 07:29:45.324331 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318678 2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 31 07:29:45.324331 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.318682 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 31 07:29:45.324331 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.319936 2577 flags.go:64] FLAG: --address="0.0.0.0" Oct 31 07:29:45.324331 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320098 2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Oct 31 07:29:45.324331 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320109 2577 flags.go:64] FLAG: --anonymous-auth="true" Oct 31 07:29:45.324331 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320114 2577 flags.go:64] FLAG: --application-metrics-count-limit="100" Oct 31 07:29:45.324331 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320119 2577 flags.go:64] FLAG: --authentication-token-webhook="false" Oct 31 07:29:45.324331 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320122 2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Oct 31 07:29:45.324331 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320127 2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Oct 31 07:29:45.324331 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320131 2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Oct 31 07:29:45.324331 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320134 2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Oct 31 07:29:45.324331 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320137 2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Oct 31 07:29:45.324331 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320140 2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Oct 31 07:29:45.324331 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320144 2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Oct 31 07:29:45.324331 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320147 2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Oct 31 07:29:45.324331 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320150 2577 flags.go:64] FLAG: --cgroup-root="" Oct 31 07:29:45.324331 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320153 2577 flags.go:64] FLAG: --cgroups-per-qos="true" Oct 31 07:29:45.324331 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320155 2577 flags.go:64] FLAG: --client-ca-file="" Oct 31 07:29:45.324331 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320158 2577 flags.go:64] FLAG: --cloud-provider="external" Oct 31 07:29:45.324331 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320162 2577 flags.go:64] FLAG: --cluster-dns="[]" Oct 31 07:29:45.324935 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320166 2577 flags.go:64] FLAG: --cluster-domain="" Oct 31 07:29:45.324935 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320169 2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Oct 31 07:29:45.324935 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320172 2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Oct 31 07:29:45.324935 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320175 2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Oct 31 07:29:45.324935 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320178 2577 flags.go:64] FLAG: --container-log-max-files="5" Oct 31 07:29:45.324935 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320192 2577 flags.go:64] FLAG: --container-log-max-size="10Mi" Oct 31 07:29:45.324935 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320195 2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Oct 31 07:29:45.324935 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320198 2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Oct 31 07:29:45.324935 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320201 2577 flags.go:64] FLAG: --containerd-namespace="k8s.io" Oct 31 07:29:45.324935 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320204 2577 flags.go:64] FLAG: --contention-profiling="false" Oct 31 07:29:45.324935 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320207 2577 flags.go:64] FLAG: --cpu-cfs-quota="true" Oct 31 07:29:45.324935 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320210 2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Oct 31 07:29:45.324935 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320213 2577 flags.go:64] FLAG: --cpu-manager-policy="none" Oct 31 07:29:45.324935 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320216 2577 flags.go:64] FLAG: --cpu-manager-policy-options="" Oct 31 07:29:45.324935 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320221 2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Oct 31 07:29:45.324935 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320224 2577 flags.go:64] FLAG: --enable-controller-attach-detach="true" Oct 31 07:29:45.324935 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320227 2577 flags.go:64] FLAG: --enable-debugging-handlers="true" Oct 31 07:29:45.324935 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320230 2577 flags.go:64] FLAG: --enable-load-reader="false" Oct 31 07:29:45.324935 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320233 2577 flags.go:64] FLAG: --enable-server="true" Oct 31 07:29:45.324935 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320236 2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Oct 31 07:29:45.324935 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320241 2577 flags.go:64] FLAG: --event-burst="100" Oct 31 07:29:45.324935 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320244 2577 flags.go:64] FLAG: --event-qps="50" Oct 31 07:29:45.324935 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320247 2577 flags.go:64] FLAG: --event-storage-age-limit="default=0" Oct 31 07:29:45.324935 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320250 2577 flags.go:64] FLAG: --event-storage-event-limit="default=0" Oct 31 07:29:45.325551 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320253 2577 flags.go:64] FLAG: --eviction-hard="" Oct 31 07:29:45.325551 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320257 2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Oct 31 07:29:45.325551 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320260 2577 flags.go:64] FLAG: --eviction-minimum-reclaim="" Oct 31 07:29:45.325551 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320263 2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Oct 31 07:29:45.325551 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320266 2577 flags.go:64] FLAG: --eviction-soft="" Oct 31 07:29:45.325551 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320269 2577 flags.go:64] FLAG: --eviction-soft-grace-period="" Oct 31 07:29:45.325551 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320273 2577 flags.go:64] FLAG: --exit-on-lock-contention="false" Oct 31 07:29:45.325551 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320276 2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Oct 31 07:29:45.325551 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320279 2577 flags.go:64] FLAG: --experimental-mounter-path="" Oct 31 07:29:45.325551 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320281 2577 flags.go:64] FLAG: --fail-cgroupv1="true" Oct 31 07:29:45.325551 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320284 2577 flags.go:64] FLAG: --fail-swap-on="true" Oct 31 07:29:45.325551 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320287 2577 flags.go:64] FLAG: --feature-gates="" Oct 31 07:29:45.325551 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320291 2577 flags.go:64] FLAG: --file-check-frequency="20s" Oct 31 07:29:45.325551 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320294 2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Oct 31 07:29:45.325551 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320296 2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Oct 31 07:29:45.325551 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320300 2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Oct 31 07:29:45.325551 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320302 2577 flags.go:64] FLAG: --healthz-port="10248" Oct 31 07:29:45.325551 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320305 2577 flags.go:64] FLAG: --help="false" Oct 31 07:29:45.325551 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320308 2577 flags.go:64] FLAG: --hostname-override="ip-10-0-130-48.us-east-2.compute.internal" Oct 31 07:29:45.325551 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320312 2577 flags.go:64] FLAG: --housekeeping-interval="10s" Oct 31 07:29:45.325551 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320314 2577 flags.go:64] FLAG: --http-check-frequency="20s" Oct 31 07:29:45.325551 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320317 2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Oct 31 07:29:45.325551 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320322 2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Oct 31 07:29:45.326102 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320326 2577 flags.go:64] FLAG: --image-gc-high-threshold="85" Oct 31 07:29:45.326102 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320329 2577 flags.go:64] FLAG: --image-gc-low-threshold="80" Oct 31 07:29:45.326102 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320332 2577 flags.go:64] FLAG: --image-service-endpoint="" Oct 31 07:29:45.326102 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320335 2577 flags.go:64] FLAG: --kernel-memcg-notification="false" Oct 31 07:29:45.326102 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320338 2577 flags.go:64] FLAG: --kube-api-burst="100" Oct 31 07:29:45.326102 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320341 2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Oct 31 07:29:45.326102 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320360 2577 flags.go:64] FLAG: --kube-api-qps="50" Oct 31 07:29:45.326102 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320364 2577 flags.go:64] FLAG: --kube-reserved="" Oct 31 07:29:45.326102 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320367 2577 flags.go:64] FLAG: --kube-reserved-cgroup="" Oct 31 07:29:45.326102 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320369 2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Oct 31 07:29:45.326102 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320372 2577 flags.go:64] FLAG: --kubelet-cgroups="" Oct 31 07:29:45.326102 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320375 2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Oct 31 07:29:45.326102 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320378 2577 flags.go:64] FLAG: --lock-file="" Oct 31 07:29:45.326102 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320381 2577 flags.go:64] FLAG: --log-cadvisor-usage="false" Oct 31 07:29:45.326102 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320384 2577 flags.go:64] FLAG: --log-flush-frequency="5s" Oct 31 07:29:45.326102 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320387 2577 flags.go:64] FLAG: --log-json-info-buffer-size="0" Oct 31 07:29:45.326102 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320393 2577 flags.go:64] FLAG: --log-json-split-stream="false" Oct 31 07:29:45.326102 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320395 2577 flags.go:64] FLAG: --log-text-info-buffer-size="0" Oct 31 07:29:45.326102 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320398 2577 flags.go:64] FLAG: --log-text-split-stream="false" Oct 31 07:29:45.326102 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320401 2577 flags.go:64] FLAG: --logging-format="text" Oct 31 07:29:45.326102 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320404 2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Oct 31 07:29:45.326102 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320408 2577 flags.go:64] FLAG: --make-iptables-util-chains="true" Oct 31 07:29:45.326102 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320411 2577 flags.go:64] FLAG: --manifest-url="" Oct 31 07:29:45.326102 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320414 2577 flags.go:64] FLAG: --manifest-url-header="" Oct 31 07:29:45.326102 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320420 2577 flags.go:64] FLAG: --max-housekeeping-interval="15s" Oct 31 07:29:45.326721 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320423 2577 flags.go:64] FLAG: --max-open-files="1000000" Oct 31 07:29:45.326721 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320427 2577 flags.go:64] FLAG: --max-pods="110" Oct 31 07:29:45.326721 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320430 2577 flags.go:64] FLAG: --maximum-dead-containers="-1" Oct 31 07:29:45.326721 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320433 2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Oct 31 07:29:45.326721 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320436 2577 flags.go:64] FLAG: --memory-manager-policy="None" Oct 31 07:29:45.326721 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320438 2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Oct 31 07:29:45.326721 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320441 2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Oct 31 07:29:45.326721 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320445 2577 flags.go:64] FLAG: --node-ip="0.0.0.0" Oct 31 07:29:45.326721 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320448 2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Oct 31 07:29:45.326721 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320455 2577 flags.go:64] FLAG: --node-status-max-images="50" Oct 31 07:29:45.326721 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320458 2577 flags.go:64] FLAG: --node-status-update-frequency="10s" Oct 31 07:29:45.326721 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320461 2577 flags.go:64] FLAG: --oom-score-adj="-999" Oct 31 07:29:45.326721 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320464 2577 flags.go:64] FLAG: --pod-cidr="" Oct 31 07:29:45.326721 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320467 2577 flags.go:64] FLAG: --pod-manifest-path="" Oct 31 07:29:45.326721 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320470 2577 flags.go:64] FLAG: --pod-max-pids="-1" Oct 31 07:29:45.326721 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320473 2577 flags.go:64] FLAG: --pods-per-core="0" Oct 31 07:29:45.326721 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320476 2577 flags.go:64] FLAG: --port="10250" Oct 31 07:29:45.326721 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320479 2577 flags.go:64] FLAG: --protect-kernel-defaults="false" Oct 31 07:29:45.326721 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320481 2577 flags.go:64] FLAG: --provider-id="aws:///us-east-2a/i-0113f1ef7509cd15b" Oct 31 07:29:45.326721 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320485 2577 flags.go:64] FLAG: --qos-reserved="" Oct 31 07:29:45.326721 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320488 2577 flags.go:64] FLAG: --read-only-port="10255" Oct 31 07:29:45.326721 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320491 2577 flags.go:64] FLAG: --register-node="true" Oct 31 07:29:45.326721 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320494 2577 flags.go:64] FLAG: --register-with-taints="" Oct 31 07:29:45.326721 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320498 2577 flags.go:64] FLAG: --registry-burst="10" Oct 31 07:29:45.326721 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320501 2577 flags.go:64] FLAG: --registry-qps="5" Oct 31 07:29:45.326721 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320503 2577 flags.go:64] FLAG: --reserved-cpus="" Oct 31 07:29:45.327358 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320506 2577 flags.go:64] FLAG: --reserved-memory="" Oct 31 07:29:45.327358 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320510 2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Oct 31 07:29:45.327358 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320513 2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Oct 31 07:29:45.327358 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320516 2577 flags.go:64] FLAG: --rotate-certificates="false" Oct 31 07:29:45.327358 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320519 2577 flags.go:64] FLAG: --rotate-server-certificates="false" Oct 31 07:29:45.327358 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320522 2577 flags.go:64] FLAG: --runonce="false" Oct 31 07:29:45.327358 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320525 2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Oct 31 07:29:45.327358 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320528 2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Oct 31 07:29:45.327358 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320531 2577 flags.go:64] FLAG: --seccomp-default="false" Oct 31 07:29:45.327358 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320534 2577 flags.go:64] FLAG: --serialize-image-pulls="true" Oct 31 07:29:45.327358 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320537 2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Oct 31 07:29:45.327358 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320540 2577 flags.go:64] FLAG: --storage-driver-db="cadvisor" Oct 31 07:29:45.327358 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320543 2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Oct 31 07:29:45.327358 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320546 2577 flags.go:64] FLAG: --storage-driver-password="root" Oct 31 07:29:45.327358 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320549 2577 flags.go:64] FLAG: --storage-driver-secure="false" Oct 31 07:29:45.327358 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320552 2577 flags.go:64] FLAG: --storage-driver-table="stats" Oct 31 07:29:45.327358 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320555 2577 flags.go:64] FLAG: --storage-driver-user="root" Oct 31 07:29:45.327358 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320557 2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Oct 31 07:29:45.327358 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320560 2577 flags.go:64] FLAG: --sync-frequency="1m0s" Oct 31 07:29:45.327358 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320563 2577 flags.go:64] FLAG: --system-cgroups="" Oct 31 07:29:45.327358 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320566 2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Oct 31 07:29:45.327358 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320572 2577 flags.go:64] FLAG: --system-reserved-cgroup="" Oct 31 07:29:45.327358 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320575 2577 flags.go:64] FLAG: --tls-cert-file="" Oct 31 07:29:45.327358 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320578 2577 flags.go:64] FLAG: --tls-cipher-suites="[]" Oct 31 07:29:45.327358 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320581 2577 flags.go:64] FLAG: --tls-min-version="" Oct 31 07:29:45.327998 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320584 2577 flags.go:64] FLAG: --tls-private-key-file="" Oct 31 07:29:45.327998 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320588 2577 flags.go:64] FLAG: --topology-manager-policy="none" Oct 31 07:29:45.327998 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320591 2577 flags.go:64] FLAG: --topology-manager-policy-options="" Oct 31 07:29:45.327998 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320593 2577 flags.go:64] FLAG: --topology-manager-scope="container" Oct 31 07:29:45.327998 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320596 2577 flags.go:64] FLAG: --v="2" Oct 31 07:29:45.327998 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320601 2577 flags.go:64] FLAG: --version="false" Oct 31 07:29:45.327998 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320605 2577 flags.go:64] FLAG: --vmodule="" Oct 31 07:29:45.327998 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320608 2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Oct 31 07:29:45.327998 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.320612 2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Oct 31 07:29:45.327998 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320750 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 31 07:29:45.327998 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320754 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 31 07:29:45.327998 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320757 2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 31 07:29:45.327998 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320760 2577 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 31 07:29:45.327998 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320763 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 31 07:29:45.327998 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320766 2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 31 07:29:45.327998 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320768 2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 31 07:29:45.327998 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320775 2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 31 07:29:45.327998 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320778 2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 31 07:29:45.327998 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320781 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 31 07:29:45.327998 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320783 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 31 07:29:45.327998 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320786 2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 31 07:29:45.327998 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320789 2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 31 07:29:45.328542 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320791 2577 feature_gate.go:386] unrecognized feature gate: Example2 Oct 31 07:29:45.328542 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320794 2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 31 07:29:45.328542 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320796 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 31 07:29:45.328542 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320799 2577 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 31 07:29:45.328542 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320801 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 31 07:29:45.328542 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320804 2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 31 07:29:45.328542 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320807 2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 31 07:29:45.328542 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320809 2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 31 07:29:45.328542 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320812 2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 31 07:29:45.328542 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320816 2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 31 07:29:45.328542 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320820 2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 31 07:29:45.328542 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320823 2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 31 07:29:45.328542 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320826 2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 31 07:29:45.328542 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320829 2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 31 07:29:45.328542 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320832 2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 31 07:29:45.328542 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320834 2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 31 07:29:45.328542 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320837 2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 31 07:29:45.328542 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320839 2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 31 07:29:45.328542 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320842 2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 31 07:29:45.329000 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320844 2577 feature_gate.go:386] unrecognized feature gate: EVPN Oct 31 07:29:45.329000 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320847 2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 31 07:29:45.329000 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320850 2577 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 31 07:29:45.329000 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320852 2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 31 07:29:45.329000 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320855 2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 31 07:29:45.329000 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320857 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 31 07:29:45.329000 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320859 2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 31 07:29:45.329000 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320863 2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 31 07:29:45.329000 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320866 2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 31 07:29:45.329000 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320868 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 31 07:29:45.329000 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320871 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 31 07:29:45.329000 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320873 2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 31 07:29:45.329000 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320876 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 31 07:29:45.329000 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320879 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 31 07:29:45.329000 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320882 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 31 07:29:45.329000 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320884 2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 31 07:29:45.329000 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320887 2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 31 07:29:45.329000 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320889 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 31 07:29:45.329000 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320892 2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 31 07:29:45.329465 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320895 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 31 07:29:45.329465 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320897 2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 31 07:29:45.329465 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320900 2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 31 07:29:45.329465 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320904 2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 31 07:29:45.329465 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320908 2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 31 07:29:45.329465 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320911 2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 31 07:29:45.329465 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320913 2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 31 07:29:45.329465 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320916 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 31 07:29:45.329465 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320918 2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 31 07:29:45.329465 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320921 2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 31 07:29:45.329465 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320924 2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 31 07:29:45.329465 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320926 2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 31 07:29:45.329465 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320929 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 31 07:29:45.329465 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320931 2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 31 07:29:45.329465 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320934 2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 31 07:29:45.329465 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320936 2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 31 07:29:45.329465 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320939 2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 31 07:29:45.329465 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320941 2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 31 07:29:45.329465 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320944 2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 31 07:29:45.329912 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320946 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 31 07:29:45.329912 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320950 2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 31 07:29:45.329912 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320953 2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 31 07:29:45.329912 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320955 2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 31 07:29:45.329912 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320958 2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 31 07:29:45.329912 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320960 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 31 07:29:45.329912 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320963 2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 31 07:29:45.329912 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320965 2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 31 07:29:45.329912 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320967 2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 31 07:29:45.329912 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320970 2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 31 07:29:45.329912 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320972 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 31 07:29:45.329912 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320975 2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 31 07:29:45.329912 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320977 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 31 07:29:45.329912 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320980 2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 31 07:29:45.329912 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320982 2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 31 07:29:45.329912 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320985 2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 31 07:29:45.329912 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320987 2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 31 07:29:45.329912 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320990 2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 31 07:29:45.329912 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320992 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 31 07:29:45.330361 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320994 2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 31 07:29:45.330361 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.320997 2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 31 07:29:45.330361 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321000 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 31 07:29:45.330361 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321003 2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 31 07:29:45.330361 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321005 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 31 07:29:45.330361 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321007 2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 31 07:29:45.330361 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321010 2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 31 07:29:45.330361 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321013 2577 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 31 07:29:45.330361 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321015 2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 31 07:29:45.330361 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321017 2577 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 31 07:29:45.330361 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321021 2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 31 07:29:45.330361 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321024 2577 feature_gate.go:386] unrecognized feature gate: Example Oct 31 07:29:45.330361 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321026 2577 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 31 07:29:45.330361 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321029 2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 31 07:29:45.330361 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321032 2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 31 07:29:45.330361 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321035 2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 31 07:29:45.330361 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321037 2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 31 07:29:45.330764 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.321064 2577 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 31 07:29:45.330764 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321100 2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 31 07:29:45.330764 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321104 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 31 07:29:45.330764 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321108 2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 31 07:29:45.330764 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321110 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 31 07:29:45.330764 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321114 2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 31 07:29:45.330764 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321116 2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 31 07:29:45.330764 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321119 2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 31 07:29:45.330764 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321121 2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 31 07:29:45.330764 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321124 2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 31 07:29:45.330764 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321126 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 31 07:29:45.330764 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321129 2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 31 07:29:45.330764 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321131 2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 31 07:29:45.330764 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321134 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 31 07:29:45.330764 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321136 2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 31 07:29:45.330764 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321139 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 31 07:29:45.331160 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321141 2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 31 07:29:45.331160 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321144 2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 31 07:29:45.331160 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321147 2577 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 31 07:29:45.331160 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321150 2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 31 07:29:45.331160 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321152 2577 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 31 07:29:45.331160 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321155 2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 31 07:29:45.331160 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321157 2577 feature_gate.go:386] unrecognized feature gate: Example Oct 31 07:29:45.331160 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321160 2577 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 31 07:29:45.331160 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321162 2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 31 07:29:45.331160 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321171 2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 31 07:29:45.331160 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321173 2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 31 07:29:45.331160 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321176 2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 31 07:29:45.331160 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321178 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 31 07:29:45.331160 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321180 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 31 07:29:45.331160 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321184 2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 31 07:29:45.331160 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321187 2577 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 31 07:29:45.331160 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321189 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 31 07:29:45.331160 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321192 2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 31 07:29:45.331160 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321194 2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 31 07:29:45.331160 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321197 2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 31 07:29:45.331664 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321200 2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 31 07:29:45.331664 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321203 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 31 07:29:45.331664 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321205 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 31 07:29:45.331664 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321208 2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 31 07:29:45.331664 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321210 2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 31 07:29:45.331664 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321213 2577 feature_gate.go:386] unrecognized feature gate: Example2 Oct 31 07:29:45.331664 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321215 2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 31 07:29:45.331664 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321218 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 31 07:29:45.331664 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321220 2577 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 31 07:29:45.331664 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321223 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 31 07:29:45.331664 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321225 2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 31 07:29:45.331664 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321228 2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 31 07:29:45.331664 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321230 2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 31 07:29:45.331664 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321232 2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 31 07:29:45.331664 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321236 2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 31 07:29:45.331664 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321240 2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 31 07:29:45.331664 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321244 2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 31 07:29:45.331664 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321246 2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 31 07:29:45.331664 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321249 2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 31 07:29:45.332124 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321251 2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 31 07:29:45.332124 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321254 2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 31 07:29:45.332124 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321258 2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 31 07:29:45.332124 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321261 2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 31 07:29:45.332124 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321264 2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 31 07:29:45.332124 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321266 2577 feature_gate.go:386] unrecognized feature gate: EVPN Oct 31 07:29:45.332124 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321268 2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 31 07:29:45.332124 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321272 2577 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 31 07:29:45.332124 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321275 2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 31 07:29:45.332124 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321277 2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 31 07:29:45.332124 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321280 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 31 07:29:45.332124 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321282 2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 31 07:29:45.332124 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321284 2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 31 07:29:45.332124 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321287 2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 31 07:29:45.332124 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321289 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 31 07:29:45.332124 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321292 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 31 07:29:45.332124 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321294 2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 31 07:29:45.332124 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321296 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 31 07:29:45.332124 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321299 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 31 07:29:45.332124 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321302 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 31 07:29:45.332645 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321304 2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 31 07:29:45.332645 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321307 2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 31 07:29:45.332645 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321310 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 31 07:29:45.332645 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321313 2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 31 07:29:45.332645 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321315 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 31 07:29:45.332645 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321318 2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 31 07:29:45.332645 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321320 2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 31 07:29:45.332645 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321324 2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 31 07:29:45.332645 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321327 2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 31 07:29:45.332645 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321330 2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 31 07:29:45.332645 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321333 2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 31 07:29:45.332645 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321335 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 31 07:29:45.332645 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321338 2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 31 07:29:45.332645 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321342 2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 31 07:29:45.332645 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321361 2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 31 07:29:45.332645 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321365 2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 31 07:29:45.332645 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321367 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 31 07:29:45.332645 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321370 2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 31 07:29:45.332645 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321373 2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 31 07:29:45.333124 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321377 2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 31 07:29:45.333124 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321379 2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 31 07:29:45.333124 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321382 2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 31 07:29:45.333124 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321384 2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 31 07:29:45.333124 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321386 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 31 07:29:45.333124 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321389 2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 31 07:29:45.333124 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321391 2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 31 07:29:45.333124 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321394 2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 31 07:29:45.333124 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321396 2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 31 07:29:45.333124 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321399 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 31 07:29:45.333124 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321401 2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 31 07:29:45.333124 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321404 2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 31 07:29:45.333124 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.321406 2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 31 07:29:45.333124 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.328793 2577 server.go:525] "Kubelet version" kubeletVersion="v1.35.9" Oct 31 07:29:45.333124 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.328810 2577 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Oct 31 07:29:45.333124 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.328827 2577 watchdog_linux.go:95] "Systemd watchdog is not enabled" Oct 31 07:29:45.333124 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.328832 2577 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Oct 31 07:29:45.333124 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.328889 2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 31 07:29:45.333124 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.328893 2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 31 07:29:45.333593 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.328896 2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 31 07:29:45.333593 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.328899 2577 feature_gate.go:386] unrecognized feature gate: Example2 Oct 31 07:29:45.333593 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.328903 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 31 07:29:45.333593 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.328906 2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 31 07:29:45.333593 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.328908 2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 31 07:29:45.333593 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.328911 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 31 07:29:45.333593 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.328914 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 31 07:29:45.333593 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.328917 2577 feature_gate.go:386] unrecognized feature gate: EVPN Oct 31 07:29:45.333593 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.328920 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 31 07:29:45.333593 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.328923 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 31 07:29:45.333593 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.328925 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 31 07:29:45.333593 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.328928 2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 31 07:29:45.333593 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.328931 2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 31 07:29:45.333593 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.328934 2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 31 07:29:45.333593 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.328936 2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 31 07:29:45.333593 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.328939 2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 31 07:29:45.333593 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.328941 2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 31 07:29:45.333593 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.328944 2577 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 31 07:29:45.333593 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.328946 2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 31 07:29:45.333593 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.328949 2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 31 07:29:45.334059 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.328951 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 31 07:29:45.334059 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.328954 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 31 07:29:45.334059 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.328957 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 31 07:29:45.334059 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.328959 2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 31 07:29:45.334059 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.328961 2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 31 07:29:45.334059 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.328964 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 31 07:29:45.334059 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.328967 2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 31 07:29:45.334059 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.328971 2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 31 07:29:45.334059 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.328974 2577 feature_gate.go:386] unrecognized feature gate: Example Oct 31 07:29:45.334059 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.328976 2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 31 07:29:45.334059 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.328979 2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 31 07:29:45.334059 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.328981 2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 31 07:29:45.334059 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.328984 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 31 07:29:45.334059 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.328987 2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 31 07:29:45.334059 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.328989 2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 31 07:29:45.334059 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.328991 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 31 07:29:45.334059 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.328994 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 31 07:29:45.334059 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.328996 2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 31 07:29:45.334059 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.328999 2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 31 07:29:45.334522 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329001 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 31 07:29:45.334522 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329004 2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 31 07:29:45.334522 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329007 2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 31 07:29:45.334522 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329009 2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 31 07:29:45.334522 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329012 2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 31 07:29:45.334522 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329015 2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 31 07:29:45.334522 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329018 2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 31 07:29:45.334522 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329021 2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 31 07:29:45.334522 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329024 2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 31 07:29:45.334522 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329026 2577 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 31 07:29:45.334522 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329029 2577 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 31 07:29:45.334522 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329031 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 31 07:29:45.334522 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329033 2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 31 07:29:45.334522 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329036 2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 31 07:29:45.334522 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329038 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 31 07:29:45.334522 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329041 2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 31 07:29:45.334522 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329043 2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 31 07:29:45.334522 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329046 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 31 07:29:45.334522 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329048 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 31 07:29:45.334971 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329052 2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 31 07:29:45.334971 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329057 2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 31 07:29:45.334971 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329061 2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 31 07:29:45.334971 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329063 2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 31 07:29:45.334971 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329066 2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 31 07:29:45.334971 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329068 2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 31 07:29:45.334971 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329071 2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 31 07:29:45.334971 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329073 2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 31 07:29:45.334971 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329076 2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 31 07:29:45.334971 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329080 2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 31 07:29:45.334971 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329084 2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 31 07:29:45.334971 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329087 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 31 07:29:45.334971 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329089 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 31 07:29:45.334971 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329092 2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 31 07:29:45.334971 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329094 2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 31 07:29:45.334971 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329097 2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 31 07:29:45.334971 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329100 2577 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 31 07:29:45.334971 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329102 2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 31 07:29:45.334971 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329104 2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 31 07:29:45.335453 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329107 2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 31 07:29:45.335453 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329109 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 31 07:29:45.335453 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329112 2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 31 07:29:45.335453 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329114 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 31 07:29:45.335453 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329117 2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 31 07:29:45.335453 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329119 2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 31 07:29:45.335453 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329122 2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 31 07:29:45.335453 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329124 2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 31 07:29:45.335453 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329127 2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 31 07:29:45.335453 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329129 2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 31 07:29:45.335453 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329132 2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 31 07:29:45.335453 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329134 2577 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 31 07:29:45.335453 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329136 2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 31 07:29:45.335453 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329139 2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 31 07:29:45.335453 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329141 2577 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 31 07:29:45.335453 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329144 2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 31 07:29:45.335453 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329148 2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 31 07:29:45.335453 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329150 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 31 07:29:45.335453 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329153 2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 31 07:29:45.335453 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329155 2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 31 07:29:45.335915 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329157 2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 31 07:29:45.335915 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329160 2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 31 07:29:45.335915 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329162 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 31 07:29:45.335915 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329165 2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 31 07:29:45.335915 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329167 2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 31 07:29:45.335915 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329169 2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 31 07:29:45.335915 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329172 2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 31 07:29:45.335915 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.329198 2577 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 31 07:29:45.335915 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329337 2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 31 07:29:45.335915 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329342 2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 31 07:29:45.335915 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329363 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 31 07:29:45.335915 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329367 2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 31 07:29:45.335915 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329370 2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 31 07:29:45.335915 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329373 2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 31 07:29:45.335915 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329375 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 31 07:29:45.335915 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329378 2577 feature_gate.go:386] unrecognized feature gate: Example2 Oct 31 07:29:45.336295 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329381 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 31 07:29:45.336295 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329383 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 31 07:29:45.336295 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329386 2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 31 07:29:45.336295 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329389 2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 31 07:29:45.336295 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329391 2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 31 07:29:45.336295 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329394 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 31 07:29:45.336295 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329396 2577 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 31 07:29:45.336295 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329399 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 31 07:29:45.336295 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329402 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 31 07:29:45.336295 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329404 2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 31 07:29:45.336295 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329407 2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 31 07:29:45.336295 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329409 2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 31 07:29:45.336295 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329412 2577 feature_gate.go:386] unrecognized feature gate: Example Oct 31 07:29:45.336295 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329415 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 31 07:29:45.336295 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329418 2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 31 07:29:45.336295 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329421 2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 31 07:29:45.336295 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329423 2577 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 31 07:29:45.336295 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329425 2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 31 07:29:45.336295 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329428 2577 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 31 07:29:45.336295 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329430 2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 31 07:29:45.336794 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329433 2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 31 07:29:45.336794 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329435 2577 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 31 07:29:45.336794 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329438 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 31 07:29:45.336794 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329440 2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 31 07:29:45.336794 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329444 2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 31 07:29:45.336794 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329447 2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 31 07:29:45.336794 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329450 2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 31 07:29:45.336794 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329452 2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 31 07:29:45.336794 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329455 2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 31 07:29:45.336794 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329457 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 31 07:29:45.336794 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329459 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 31 07:29:45.336794 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329462 2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 31 07:29:45.336794 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329464 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 31 07:29:45.336794 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329467 2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 31 07:29:45.336794 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329470 2577 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 31 07:29:45.336794 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329472 2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 31 07:29:45.336794 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329475 2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 31 07:29:45.336794 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329477 2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 31 07:29:45.336794 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329480 2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 31 07:29:45.337247 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329482 2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 31 07:29:45.337247 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329485 2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 31 07:29:45.337247 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329488 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 31 07:29:45.337247 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329490 2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 31 07:29:45.337247 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329493 2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 31 07:29:45.337247 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329495 2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 31 07:29:45.337247 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329498 2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 31 07:29:45.337247 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329501 2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 31 07:29:45.337247 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329503 2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 31 07:29:45.337247 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329505 2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 31 07:29:45.337247 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329508 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 31 07:29:45.337247 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329510 2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 31 07:29:45.337247 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329513 2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 31 07:29:45.337247 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329515 2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 31 07:29:45.337247 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329517 2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 31 07:29:45.337247 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329520 2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 31 07:29:45.337247 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329522 2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 31 07:29:45.337247 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329524 2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 31 07:29:45.337247 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329527 2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 31 07:29:45.337247 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329529 2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 31 07:29:45.337718 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329531 2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 31 07:29:45.337718 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329534 2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 31 07:29:45.337718 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329536 2577 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 31 07:29:45.337718 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329538 2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 31 07:29:45.337718 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329541 2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 31 07:29:45.337718 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329543 2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 31 07:29:45.337718 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329546 2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 31 07:29:45.337718 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329549 2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 31 07:29:45.337718 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329551 2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 31 07:29:45.337718 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329553 2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 31 07:29:45.337718 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329557 2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 31 07:29:45.337718 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329561 2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 31 07:29:45.337718 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329564 2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 31 07:29:45.337718 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329567 2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 31 07:29:45.337718 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329570 2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 31 07:29:45.337718 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329572 2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 31 07:29:45.337718 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329575 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 31 07:29:45.337718 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329577 2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 31 07:29:45.337718 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329580 2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 31 07:29:45.338161 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329582 2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 31 07:29:45.338161 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329585 2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 31 07:29:45.338161 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329588 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 31 07:29:45.338161 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329590 2577 feature_gate.go:386] unrecognized feature gate: EVPN Oct 31 07:29:45.338161 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329593 2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 31 07:29:45.338161 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329595 2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 31 07:29:45.338161 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329597 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 31 07:29:45.338161 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329600 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 31 07:29:45.338161 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329602 2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 31 07:29:45.338161 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329605 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 31 07:29:45.338161 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329607 2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 31 07:29:45.338161 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329609 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 31 07:29:45.338161 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329612 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 31 07:29:45.338161 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329614 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 31 07:29:45.338161 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329616 2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 31 07:29:45.338161 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329619 2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 31 07:29:45.338161 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329621 2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 31 07:29:45.338161 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329623 2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 31 07:29:45.338161 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329626 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 31 07:29:45.338161 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329628 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 31 07:29:45.338661 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.329652 2577 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 31 07:29:45.338661 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329684 2577 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 31 07:29:45.338661 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329688 2577 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 31 07:29:45.338661 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329691 2577 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 31 07:29:45.338661 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329695 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 31 07:29:45.338661 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329698 2577 feature_gate.go:386] unrecognized feature gate: Example2 Oct 31 07:29:45.338661 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329700 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 31 07:29:45.338661 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329703 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 31 07:29:45.338661 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329706 2577 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 31 07:29:45.338661 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329708 2577 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 31 07:29:45.338661 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329711 2577 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 31 07:29:45.338661 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329714 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 31 07:29:45.338661 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329716 2577 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 31 07:29:45.338661 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329719 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 31 07:29:45.338661 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329722 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 31 07:29:45.338661 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329725 2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 31 07:29:45.339037 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329727 2577 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 31 07:29:45.339037 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329730 2577 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 31 07:29:45.339037 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329732 2577 feature_gate.go:386] unrecognized feature gate: Example Oct 31 07:29:45.339037 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329735 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 31 07:29:45.339037 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329737 2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 31 07:29:45.339037 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329740 2577 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 31 07:29:45.339037 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329742 2577 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 31 07:29:45.339037 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329744 2577 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 31 07:29:45.339037 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329747 2577 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 31 07:29:45.339037 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329749 2577 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 31 07:29:45.339037 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329751 2577 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 31 07:29:45.339037 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329754 2577 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 31 07:29:45.339037 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329756 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 31 07:29:45.339037 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329759 2577 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 31 07:29:45.339037 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329762 2577 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 31 07:29:45.339037 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329765 2577 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 31 07:29:45.339037 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329768 2577 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 31 07:29:45.339037 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329771 2577 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 31 07:29:45.339037 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329774 2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 31 07:29:45.339037 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329776 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 31 07:29:45.339525 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329779 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 31 07:29:45.339525 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329781 2577 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 31 07:29:45.339525 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329783 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 31 07:29:45.339525 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329786 2577 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 31 07:29:45.339525 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329789 2577 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 31 07:29:45.339525 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329791 2577 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 31 07:29:45.339525 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329794 2577 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 31 07:29:45.339525 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329796 2577 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 31 07:29:45.339525 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329798 2577 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 31 07:29:45.339525 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329801 2577 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 31 07:29:45.339525 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329803 2577 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 31 07:29:45.339525 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329811 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 31 07:29:45.339525 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329814 2577 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 31 07:29:45.339525 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329816 2577 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 31 07:29:45.339525 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329820 2577 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 31 07:29:45.339525 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329822 2577 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 31 07:29:45.339525 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329825 2577 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 31 07:29:45.339525 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329827 2577 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 31 07:29:45.339525 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329830 2577 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 31 07:29:45.339525 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329832 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 31 07:29:45.340030 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329835 2577 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 31 07:29:45.340030 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329837 2577 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 31 07:29:45.340030 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329839 2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 31 07:29:45.340030 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329842 2577 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 31 07:29:45.340030 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329845 2577 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 31 07:29:45.340030 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329847 2577 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 31 07:29:45.340030 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329849 2577 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 31 07:29:45.340030 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329852 2577 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 31 07:29:45.340030 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329854 2577 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 31 07:29:45.340030 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329857 2577 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 31 07:29:45.340030 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329859 2577 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 31 07:29:45.340030 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329862 2577 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 31 07:29:45.340030 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329864 2577 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 31 07:29:45.340030 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329867 2577 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 31 07:29:45.340030 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329869 2577 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 31 07:29:45.340030 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329872 2577 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 31 07:29:45.340030 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329874 2577 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 31 07:29:45.340030 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329877 2577 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 31 07:29:45.340030 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329879 2577 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 31 07:29:45.340618 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329883 2577 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 31 07:29:45.340618 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329886 2577 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 31 07:29:45.340618 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329889 2577 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 31 07:29:45.340618 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329892 2577 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 31 07:29:45.340618 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329894 2577 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 31 07:29:45.340618 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329897 2577 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 31 07:29:45.340618 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329900 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 31 07:29:45.340618 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329902 2577 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 31 07:29:45.340618 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329904 2577 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 31 07:29:45.340618 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329907 2577 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 31 07:29:45.340618 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329909 2577 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 31 07:29:45.340618 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329911 2577 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 31 07:29:45.340618 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329914 2577 feature_gate.go:386] unrecognized feature gate: EVPN Oct 31 07:29:45.340618 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329919 2577 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 31 07:29:45.340618 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329922 2577 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 31 07:29:45.340618 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329924 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 31 07:29:45.340618 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329927 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 31 07:29:45.340618 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329929 2577 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 31 07:29:45.340618 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329932 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 31 07:29:45.340618 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329934 2577 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 31 07:29:45.341120 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329936 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 31 07:29:45.341120 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329939 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 31 07:29:45.341120 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329941 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 31 07:29:45.341120 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329944 2577 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 31 07:29:45.341120 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329948 2577 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 31 07:29:45.341120 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329950 2577 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 31 07:29:45.341120 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329953 2577 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 31 07:29:45.341120 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329955 2577 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 31 07:29:45.341120 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329958 2577 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 31 07:29:45.341120 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329960 2577 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 31 07:29:45.341120 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329963 2577 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 31 07:29:45.341120 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:45.329966 2577 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 31 07:29:45.341120 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.330025 2577 server.go:957] "Client rotation is on, will bootstrap in background" Oct 31 07:29:45.341120 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.332282 2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Oct 31 07:29:45.341120 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.333398 2577 server.go:1014] "Starting client certificate rotation" Oct 31 07:29:45.341120 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.333494 2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Oct 31 07:29:45.341120 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.333528 2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Oct 31 07:29:45.341120 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.337534 2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 31 07:29:45.341584 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.340359 2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 31 07:29:45.354763 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.354746 2577 log.go:25] "Validated CRI v1 runtime API" Oct 31 07:29:45.359204 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.359187 2577 log.go:25] "Validated CRI v1 image API" Oct 31 07:29:45.360037 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.360013 2577 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Oct 31 07:29:45.362653 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.362634 2577 fs.go:135] Filesystem UUIDs: map[492eb354-d37c-4e17-9e39-73d450d41f70:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 87747e47-46e9-4bc1-9b59-cfb442c7132b:/dev/nvme0n1p4] Oct 31 07:29:45.362742 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.362652 2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}] Oct 31 07:29:45.368635 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.368513 2577 manager.go:217] Machine: {Timestamp:2026-10-31 07:29:45.367143051 +0000 UTC m=+0.375719470 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099950 MemoryCapacity:32812011520 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec269141edead14f4cc16acc2450ceec SystemUUID:ec269141-edea-d14f-4cc1-6acc2450ceec BootID:86951882-5a2d-46ec-9ec7-d012a538784f Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406003712 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562402304 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406007808 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:e9:2f:c4:23 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:e9:2f:c4:23 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:5e:5e:7c:af:96:85 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812011520 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Oct 31 07:29:45.368997 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.368980 2577 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Oct 31 07:29:45.369200 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.369189 2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Oct 31 07:29:45.369286 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.369274 2577 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Oct 31 07:29:45.370442 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.370420 2577 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Oct 31 07:29:45.370571 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.370445 2577 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-48.us-east-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Oct 31 07:29:45.372043 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.372032 2577 topology_manager.go:143] "Creating topology manager with none policy" Oct 31 07:29:45.372081 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.372045 2577 container_manager_linux.go:311] "Creating device plugin manager" Oct 31 07:29:45.372081 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.372059 2577 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 31 07:29:45.372081 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.372069 2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 31 07:29:45.373235 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.373224 2577 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager" Oct 31 07:29:45.377625 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.377613 2577 state_mem.go:41] "Initialized" logger="CPUManager state memory" Oct 31 07:29:45.377743 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.377734 2577 server.go:1235] "Using root directory" path="/var/lib/kubelet" Oct 31 07:29:45.377781 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.377771 2577 kubelet.go:493] "Attempting to sync node with API server" Oct 31 07:29:45.377808 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.377788 2577 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests" Oct 31 07:29:45.377833 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.377807 2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Oct 31 07:29:45.377833 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.377820 2577 kubelet.go:405] "Adding apiserver pod source" Oct 31 07:29:45.377833 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.377831 2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Oct 31 07:29:45.378229 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.378216 2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 31 07:29:45.380671 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.380656 2577 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1" Oct 31 07:29:45.387784 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.387765 2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 31 07:29:45.388726 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.388712 2577 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Oct 31 07:29:45.388768 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.388744 2577 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Oct 31 07:29:45.388923 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.388907 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Oct 31 07:29:45.388971 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.388939 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Oct 31 07:29:45.388971 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.388952 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Oct 31 07:29:45.388971 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.388965 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Oct 31 07:29:45.389045 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.388977 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Oct 31 07:29:45.389045 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.388989 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Oct 31 07:29:45.389638 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.389629 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Oct 31 07:29:45.389674 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.389640 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Oct 31 07:29:45.389674 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.389647 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Oct 31 07:29:45.389674 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.389653 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Oct 31 07:29:45.389674 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.389664 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Oct 31 07:29:45.389674 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.389672 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Oct 31 07:29:45.391651 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.391638 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Oct 31 07:29:45.391695 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.391657 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Oct 31 07:29:45.395859 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.395843 2577 server.go:1263] "Started kubelet" Oct 31 07:29:45.395934 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.395893 2577 server.go:182] "Starting to listen" address="0.0.0.0" port=10250 Oct 31 07:29:45.395986 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.395937 2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Oct 31 07:29:45.396035 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.396009 2577 server_v1.go:49] "podresources" method="list" useActivePods=true Oct 31 07:29:45.396616 ip-10-0-130-48 systemd[1]: Started Kubernetes Kubelet. Oct 31 07:29:45.397630 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.397529 2577 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Oct 31 07:29:45.398657 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.398640 2577 server.go:317] "Adding debug handlers to kubelet server" Oct 31 07:29:45.402056 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:45.402035 2577 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Oct 31 07:29:45.405190 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.405175 2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Oct 31 07:29:45.405825 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.405810 2577 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer" Oct 31 07:29:45.405897 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:45.405885 2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-48.us-east-2.compute.internal\" not found" Oct 31 07:29:45.406600 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.406572 2577 volume_manager.go:309] "The desired_state_of_world populator starts" Oct 31 07:29:45.406600 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.406592 2577 volume_manager.go:311] "Starting Kubelet Volume Manager" Oct 31 07:29:45.406600 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.406620 2577 factory.go:55] Registering systemd factory Oct 31 07:29:45.406600 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.406635 2577 factory.go:223] Registration of the systemd container factory successfully Oct 31 07:29:45.406600 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.406572 2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Oct 31 07:29:45.407004 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.406701 2577 reconstruct.go:97] "Volume reconstruction finished" Oct 31 07:29:45.407004 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.406710 2577 reconciler.go:29] "Reconciler: start to sync state" Oct 31 07:29:45.407004 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.406846 2577 factory.go:153] Registering CRI-O factory Oct 31 07:29:45.407004 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.406858 2577 factory.go:223] Registration of the crio container factory successfully Oct 31 07:29:45.407004 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.406910 2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Oct 31 07:29:45.407004 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.406929 2577 factory.go:103] Registering Raw factory Oct 31 07:29:45.407004 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.406944 2577 manager.go:1199] Started watching for new ooms in manager Oct 31 07:29:45.408048 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.408033 2577 manager.go:319] Starting recovery of all containers Oct 31 07:29:45.408252 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.408234 2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-48.us-east-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Oct 31 07:29:45.408359 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:45.408323 2577 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-48.us-east-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node" Oct 31 07:29:45.408641 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:45.408617 2577 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-48.us-east-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Oct 31 07:29:45.408771 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:45.408748 2577 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service" Oct 31 07:29:45.410288 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:45.408673 2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-48.us-east-2.compute.internal.18e38cb4f166f7a8 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-48.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-48.us-east-2.compute.internal,},FirstTimestamp:2026-10-31 07:29:45.395648424 +0000 UTC m=+0.404224843,LastTimestamp:2026-10-31 07:29:45.395648424 +0000 UTC m=+0.404224843,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-48.us-east-2.compute.internal,}" Oct 31 07:29:45.417390 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.417374 2577 manager.go:324] Recovery completed Oct 31 07:29:45.418592 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:45.418568 2577 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" Oct 31 07:29:45.422303 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.422282 2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-976vh" Oct 31 07:29:45.422863 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.422849 2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 31 07:29:45.426786 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.426768 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-48.us-east-2.compute.internal" event="NodeHasSufficientMemory" Oct 31 07:29:45.426856 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.426797 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-48.us-east-2.compute.internal" event="NodeHasNoDiskPressure" Oct 31 07:29:45.426856 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.426816 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-48.us-east-2.compute.internal" event="NodeHasSufficientPID" Oct 31 07:29:45.427316 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.427303 2577 cpu_manager.go:226] "Starting" policy="none" Oct 31 07:29:45.427316 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.427315 2577 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s" Oct 31 07:29:45.427434 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.427329 2577 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory" Oct 31 07:29:45.427434 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.427402 2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 31 07:29:45.428104 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.428089 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-48.us-east-2.compute.internal" event="NodeHasSufficientMemory" Oct 31 07:29:45.428173 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.428116 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-48.us-east-2.compute.internal" event="NodeHasNoDiskPressure" Oct 31 07:29:45.428173 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.428128 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-48.us-east-2.compute.internal" event="NodeHasSufficientPID" Oct 31 07:29:45.428932 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:45.428860 2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-48.us-east-2.compute.internal.18e38cb4f3420d97 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-48.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-130-48.us-east-2.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-130-48.us-east-2.compute.internal,},FirstTimestamp:2026-10-31 07:29:45.426783639 +0000 UTC m=+0.435360057,LastTimestamp:2026-10-31 07:29:45.426783639 +0000 UTC m=+0.435360057,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-48.us-east-2.compute.internal,}" Oct 31 07:29:45.429501 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.429489 2577 policy_none.go:50] "Start" Oct 31 07:29:45.429545 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.429504 2577 memory_manager.go:187] "Starting memorymanager" policy="None" Oct 31 07:29:45.429545 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.429514 2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Oct 31 07:29:45.431511 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.431499 2577 policy_none.go:44] "Start" Oct 31 07:29:45.431932 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.431918 2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-976vh" Oct 31 07:29:45.466407 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.466095 2577 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"} Oct 31 07:29:45.490654 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.474080 2577 manager.go:341] "Starting Device Plugin manager" Oct 31 07:29:45.490654 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:45.474108 2577 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Oct 31 07:29:45.490654 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.474117 2577 server.go:87] "Starting device plugin registration server" Oct 31 07:29:45.490654 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.474340 2577 eviction_manager.go:194] "Eviction manager: starting control loop" Oct 31 07:29:45.490654 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.474367 2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Oct 31 07:29:45.490654 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.474525 2577 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Oct 31 07:29:45.490654 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.474598 2577 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts" Oct 31 07:29:45.490654 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.474605 2577 plugin_manager.go:121] "Starting Kubelet Plugin Manager" Oct 31 07:29:45.490654 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:45.475012 2577 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Oct 31 07:29:45.490654 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:45.475060 2577 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-48.us-east-2.compute.internal\" not found" Oct 31 07:29:45.490654 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.484561 2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Oct 31 07:29:45.490654 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.485779 2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Oct 31 07:29:45.490654 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.485796 2577 status_manager.go:249] "Starting to sync pod status with apiserver" Oct 31 07:29:45.490654 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.485819 2577 kubelet.go:2521] "Starting kubelet main sync loop" Oct 31 07:29:45.490654 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:45.485847 2577 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Oct 31 07:29:45.490654 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.488996 2577 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Oct 31 07:29:45.575165 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.575107 2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 31 07:29:45.575938 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.575924 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-48.us-east-2.compute.internal" event="NodeHasSufficientMemory" Oct 31 07:29:45.576019 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.575951 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-48.us-east-2.compute.internal" event="NodeHasNoDiskPressure" Oct 31 07:29:45.576019 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.575961 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-48.us-east-2.compute.internal" event="NodeHasSufficientPID" Oct 31 07:29:45.576019 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.575989 2577 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-130-48.us-east-2.compute.internal" Oct 31 07:29:45.586470 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.586446 2577 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-48.us-east-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-48.us-east-2.compute.internal"] Oct 31 07:29:45.586558 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.586498 2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 31 07:29:45.587254 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.587237 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-48.us-east-2.compute.internal" event="NodeHasSufficientMemory" Oct 31 07:29:45.587333 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.587266 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-48.us-east-2.compute.internal" event="NodeHasNoDiskPressure" Oct 31 07:29:45.587333 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.587279 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-48.us-east-2.compute.internal" event="NodeHasSufficientPID" Oct 31 07:29:45.588427 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.588415 2577 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-130-48.us-east-2.compute.internal" Oct 31 07:29:45.588469 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:45.588435 2577 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-48.us-east-2.compute.internal\": node \"ip-10-0-130-48.us-east-2.compute.internal\" not found" Oct 31 07:29:45.589002 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.588991 2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 31 07:29:45.589160 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.589146 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-48.us-east-2.compute.internal" Oct 31 07:29:45.589199 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.589175 2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 31 07:29:45.589733 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.589719 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-48.us-east-2.compute.internal" event="NodeHasSufficientMemory" Oct 31 07:29:45.589779 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.589746 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-48.us-east-2.compute.internal" event="NodeHasNoDiskPressure" Oct 31 07:29:45.589779 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.589762 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-48.us-east-2.compute.internal" event="NodeHasSufficientPID" Oct 31 07:29:45.589779 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.589768 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-48.us-east-2.compute.internal" event="NodeHasSufficientMemory" Oct 31 07:29:45.589864 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.589794 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-48.us-east-2.compute.internal" event="NodeHasNoDiskPressure" Oct 31 07:29:45.589864 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.589809 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-48.us-east-2.compute.internal" event="NodeHasSufficientPID" Oct 31 07:29:45.591366 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.591337 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-48.us-east-2.compute.internal" Oct 31 07:29:45.591438 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.591378 2577 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 31 07:29:45.594508 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.594493 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-48.us-east-2.compute.internal" event="NodeHasSufficientMemory" Oct 31 07:29:45.594586 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.594519 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-48.us-east-2.compute.internal" event="NodeHasNoDiskPressure" Oct 31 07:29:45.594586 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.594534 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-48.us-east-2.compute.internal" event="NodeHasSufficientPID" Oct 31 07:29:45.615449 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:45.615428 2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-48.us-east-2.compute.internal\" not found" Oct 31 07:29:45.624135 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:45.624112 2577 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-48.us-east-2.compute.internal\" not found" node="ip-10-0-130-48.us-east-2.compute.internal" Oct 31 07:29:45.628117 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:45.628099 2577 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-48.us-east-2.compute.internal\" not found" node="ip-10-0-130-48.us-east-2.compute.internal" Oct 31 07:29:45.707778 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.707752 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/11f3ad8c54222373dd06005938b97a96-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-48.us-east-2.compute.internal\" (UID: \"11f3ad8c54222373dd06005938b97a96\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-48.us-east-2.compute.internal" Oct 31 07:29:45.707938 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.707783 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/11f3ad8c54222373dd06005938b97a96-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-48.us-east-2.compute.internal\" (UID: \"11f3ad8c54222373dd06005938b97a96\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-48.us-east-2.compute.internal" Oct 31 07:29:45.707938 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.707816 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8927b882957b5b07fb0714896c6f1a7f-config\") pod \"kube-apiserver-proxy-ip-10-0-130-48.us-east-2.compute.internal\" (UID: \"8927b882957b5b07fb0714896c6f1a7f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-48.us-east-2.compute.internal" Oct 31 07:29:45.715874 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:45.715851 2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-48.us-east-2.compute.internal\" not found" Oct 31 07:29:45.808405 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.808378 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/11f3ad8c54222373dd06005938b97a96-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-48.us-east-2.compute.internal\" (UID: \"11f3ad8c54222373dd06005938b97a96\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-48.us-east-2.compute.internal" Oct 31 07:29:45.808519 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.808410 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/11f3ad8c54222373dd06005938b97a96-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-48.us-east-2.compute.internal\" (UID: \"11f3ad8c54222373dd06005938b97a96\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-48.us-east-2.compute.internal" Oct 31 07:29:45.808519 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.808427 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8927b882957b5b07fb0714896c6f1a7f-config\") pod \"kube-apiserver-proxy-ip-10-0-130-48.us-east-2.compute.internal\" (UID: \"8927b882957b5b07fb0714896c6f1a7f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-48.us-east-2.compute.internal" Oct 31 07:29:45.808519 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.808487 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8927b882957b5b07fb0714896c6f1a7f-config\") pod \"kube-apiserver-proxy-ip-10-0-130-48.us-east-2.compute.internal\" (UID: \"8927b882957b5b07fb0714896c6f1a7f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-48.us-east-2.compute.internal" Oct 31 07:29:45.808519 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.808499 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/11f3ad8c54222373dd06005938b97a96-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-48.us-east-2.compute.internal\" (UID: \"11f3ad8c54222373dd06005938b97a96\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-48.us-east-2.compute.internal" Oct 31 07:29:45.808519 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.808494 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/11f3ad8c54222373dd06005938b97a96-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-48.us-east-2.compute.internal\" (UID: \"11f3ad8c54222373dd06005938b97a96\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-48.us-east-2.compute.internal" Oct 31 07:29:45.816469 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:45.816444 2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-48.us-east-2.compute.internal\" not found" Oct 31 07:29:45.917208 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:45.917161 2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-48.us-east-2.compute.internal\" not found" Oct 31 07:29:45.928326 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.928312 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-48.us-east-2.compute.internal" Oct 31 07:29:45.933224 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:45.933059 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-48.us-east-2.compute.internal" Oct 31 07:29:46.017965 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:46.017944 2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-48.us-east-2.compute.internal\" not found" Oct 31 07:29:46.118413 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:46.118393 2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-48.us-east-2.compute.internal\" not found" Oct 31 07:29:46.218942 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:46.218920 2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-48.us-east-2.compute.internal\" not found" Oct 31 07:29:46.227583 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:46.227557 2577 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Oct 31 07:29:46.319206 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:46.319173 2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-48.us-east-2.compute.internal\" not found" Oct 31 07:29:46.333275 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:46.333253 2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Oct 31 07:29:46.333394 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:46.333377 2577 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received" Oct 31 07:29:46.333436 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:46.333402 2577 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received" Oct 31 07:29:46.420034 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:46.420009 2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-48.us-east-2.compute.internal\" not found" Oct 31 07:29:46.438086 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:46.438054 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-30 07:24:45 +0000 UTC" deadline="2028-08-10 08:09:13.903479128 +0000 UTC" Oct 31 07:29:46.438086 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:46.438083 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15576h39m27.465398828s" Oct 31 07:29:46.520944 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:46.520881 2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-48.us-east-2.compute.internal\" not found" Oct 31 07:29:46.535602 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:46.535566 2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11f3ad8c54222373dd06005938b97a96.slice/crio-c4b192911d11550f7de0ff5884848f30c9f162951df39fa0707f7b8b1f85bc45 WatchSource:0}: Error finding container c4b192911d11550f7de0ff5884848f30c9f162951df39fa0707f7b8b1f85bc45: Status 404 returned error can't find the container with id c4b192911d11550f7de0ff5884848f30c9f162951df39fa0707f7b8b1f85bc45 Oct 31 07:29:46.536035 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:46.536016 2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8927b882957b5b07fb0714896c6f1a7f.slice/crio-c8e97bb91fa7a4579a6eb8f24f665103772028210f2a68df470a03c9f5841638 WatchSource:0}: Error finding container c8e97bb91fa7a4579a6eb8f24f665103772028210f2a68df470a03c9f5841638: Status 404 returned error can't find the container with id c8e97bb91fa7a4579a6eb8f24f665103772028210f2a68df470a03c9f5841638 Oct 31 07:29:46.540889 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:46.540865 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 31 07:29:46.547010 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:46.546996 2577 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Oct 31 07:29:46.621009 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:46.620986 2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-48.us-east-2.compute.internal\" not found" Oct 31 07:29:46.721441 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:46.721420 2577 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-48.us-east-2.compute.internal\" not found" Oct 31 07:29:46.732833 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:46.732817 2577 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161" Oct 31 07:29:46.806881 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:46.806833 2577 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-48.us-east-2.compute.internal" Oct 31 07:29:46.821187 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:46.821171 2577 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 31 07:29:46.822316 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:46.822303 2577 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-48.us-east-2.compute.internal" Oct 31 07:29:46.830869 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:46.830851 2577 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 31 07:29:47.193825 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.193798 2577 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Oct 31 07:29:47.378942 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.378686 2577 apiserver.go:52] "Watching apiserver" Oct 31 07:29:47.388696 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.388667 2577 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Oct 31 07:29:47.392065 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.389852 2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-pl5cl","kube-system/kube-apiserver-proxy-ip-10-0-130-48.us-east-2.compute.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-48.us-east-2.compute.internal","openshift-multus/multus-additional-cni-plugins-ktzg4","openshift-multus/network-metrics-daemon-7thxr","openshift-network-operator/iptables-alerter-lmsk9","openshift-cluster-node-tuning-operator/tuned-sqxmw","openshift-dns/node-resolver-scrms","openshift-image-registry/node-ca-rhg7w","openshift-network-diagnostics/network-check-target-ncnd7","kube-system/global-pull-secret-syncer-8hpcb","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r78n8","openshift-multus/multus-w6vnh","openshift-ovn-kubernetes/ovnkube-node-98whn"] Oct 31 07:29:47.394913 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.394591 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ncnd7" Oct 31 07:29:47.394913 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:47.394687 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ncnd7" podUID="d8b28b9f-37a2-4859-87fd-d27bb37d832b" Oct 31 07:29:47.396960 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.396932 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-w6vnh" Oct 31 07:29:47.401256 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.401231 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-65sf9\"" Oct 31 07:29:47.401381 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.401334 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Oct 31 07:29:47.401506 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.401486 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Oct 31 07:29:47.401568 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.401504 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.401568 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.401560 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Oct 31 07:29:47.401676 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.401563 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Oct 31 07:29:47.404994 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.404972 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Oct 31 07:29:47.405166 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.405142 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ktzg4" Oct 31 07:29:47.405166 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.405156 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-bcvgm\"" Oct 31 07:29:47.405710 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.405614 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Oct 31 07:29:47.405710 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.405689 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Oct 31 07:29:47.405710 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.405693 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Oct 31 07:29:47.405944 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.405730 2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Oct 31 07:29:47.405944 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.405751 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Oct 31 07:29:47.406940 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.406921 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Oct 31 07:29:47.407827 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.407806 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7thxr" Oct 31 07:29:47.407927 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:47.407881 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7thxr" podUID="28ec9621-7ec5-421c-a88b-51bb20d3a7bf" Oct 31 07:29:47.408980 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.408725 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Oct 31 07:29:47.409056 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.408982 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Oct 31 07:29:47.409111 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.409054 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-78wjc\"" Oct 31 07:29:47.410808 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.410789 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-scrms" Oct 31 07:29:47.412929 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.412908 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r78n8" Oct 31 07:29:47.416190 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.416170 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:29:47.416292 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.416207 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ed522b38-611a-452a-b201-3ca416428d42-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ktzg4\" (UID: \"ed522b38-611a-452a-b201-3ca416428d42\") " pod="openshift-multus/multus-additional-cni-plugins-ktzg4" Oct 31 07:29:47.416292 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.416238 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-997f5\" (UniqueName: \"kubernetes.io/projected/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-kube-api-access-997f5\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:29:47.416379 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.416263 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-host-slash\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.416379 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.416347 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-run-openvswitch\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.416467 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.416407 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-ovnkube-config\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.416467 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.416459 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28ec9621-7ec5-421c-a88b-51bb20d3a7bf-metrics-certs\") pod \"network-metrics-daemon-7thxr\" (UID: \"28ec9621-7ec5-421c-a88b-51bb20d3a7bf\") " pod="openshift-multus/network-metrics-daemon-7thxr" Oct 31 07:29:47.416543 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.416490 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ckzf\" (UniqueName: \"kubernetes.io/projected/d8b28b9f-37a2-4859-87fd-d27bb37d832b-kube-api-access-2ckzf\") pod \"network-check-target-ncnd7\" (UID: \"d8b28b9f-37a2-4859-87fd-d27bb37d832b\") " pod="openshift-network-diagnostics/network-check-target-ncnd7" Oct 31 07:29:47.416583 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.416540 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-multus-socket-dir-parent\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:29:47.416583 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.416558 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-host-run-multus-certs\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:29:47.416583 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.416575 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-run-systemd\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.416682 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.416590 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ed522b38-611a-452a-b201-3ca416428d42-cnibin\") pod \"multus-additional-cni-plugins-ktzg4\" (UID: \"ed522b38-611a-452a-b201-3ca416428d42\") " pod="openshift-multus/multus-additional-cni-plugins-ktzg4" Oct 31 07:29:47.416682 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.416606 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-multus-cni-dir\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:29:47.416682 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.416651 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-host-run-k8s-cni-cncf-io\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:29:47.416842 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.416702 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-etc-openvswitch\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.416842 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.416747 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-host-cni-bin\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.416954 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.416853 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mqks\" (UniqueName: \"kubernetes.io/projected/28ec9621-7ec5-421c-a88b-51bb20d3a7bf-kube-api-access-8mqks\") pod \"network-metrics-daemon-7thxr\" (UID: \"28ec9621-7ec5-421c-a88b-51bb20d3a7bf\") " pod="openshift-multus/network-metrics-daemon-7thxr" Oct 31 07:29:47.416954 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.416904 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-host-var-lib-cni-bin\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:29:47.417058 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.416950 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-host-var-lib-cni-multus\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:29:47.417058 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.416985 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-cni-binary-copy\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:29:47.417058 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.417013 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-host-run-netns\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:29:47.417058 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.417055 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-host-cni-netd\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.417223 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.417081 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-cnibin\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:29:47.417223 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.417104 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-host-var-lib-kubelet\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:29:47.417223 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.417126 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-hostroot\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:29:47.417223 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.417143 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-multus-daemon-config\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:29:47.417223 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.417157 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-systemd-units\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.417223 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.417175 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-log-socket\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.417223 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.417189 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-ovnkube-script-lib\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.417223 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.417205 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrvrg\" (UniqueName: \"kubernetes.io/projected/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-kube-api-access-mrvrg\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.417760 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.417264 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-system-cni-dir\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:29:47.417760 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.417287 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-host-kubelet\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.417760 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.417312 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-host-run-netns\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.417760 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.417338 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ed522b38-611a-452a-b201-3ca416428d42-system-cni-dir\") pod \"multus-additional-cni-plugins-ktzg4\" (UID: \"ed522b38-611a-452a-b201-3ca416428d42\") " pod="openshift-multus/multus-additional-cni-plugins-ktzg4" Oct 31 07:29:47.417760 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.417364 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ed522b38-611a-452a-b201-3ca416428d42-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ktzg4\" (UID: \"ed522b38-611a-452a-b201-3ca416428d42\") " pod="openshift-multus/multus-additional-cni-plugins-ktzg4" Oct 31 07:29:47.417760 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.417420 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-multus-conf-dir\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:29:47.417760 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.417503 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-etc-kubernetes\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:29:47.417760 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.417530 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-var-lib-openvswitch\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.417760 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.417556 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-node-log\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.417760 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.417581 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.417760 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.417607 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-ovn-node-metrics-cert\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.417760 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.417673 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ed522b38-611a-452a-b201-3ca416428d42-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ktzg4\" (UID: \"ed522b38-611a-452a-b201-3ca416428d42\") " pod="openshift-multus/multus-additional-cni-plugins-ktzg4" Oct 31 07:29:47.417760 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.417704 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4tdk\" (UniqueName: \"kubernetes.io/projected/ed522b38-611a-452a-b201-3ca416428d42-kube-api-access-w4tdk\") pod \"multus-additional-cni-plugins-ktzg4\" (UID: \"ed522b38-611a-452a-b201-3ca416428d42\") " pod="openshift-multus/multus-additional-cni-plugins-ktzg4" Oct 31 07:29:47.417760 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.417735 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-run-ovn\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.418500 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.417772 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-host-run-ovn-kubernetes\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.418500 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.417798 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-env-overrides\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.418500 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.417832 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ed522b38-611a-452a-b201-3ca416428d42-cni-binary-copy\") pod \"multus-additional-cni-plugins-ktzg4\" (UID: \"ed522b38-611a-452a-b201-3ca416428d42\") " pod="openshift-multus/multus-additional-cni-plugins-ktzg4" Oct 31 07:29:47.419410 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.419388 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-pl5cl" Oct 31 07:29:47.421804 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.421306 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8hpcb" Oct 31 07:29:47.421804 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:47.421402 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8hpcb" podUID="297b35d7-aa65-49c7-8460-ac3f9860248c" Oct 31 07:29:47.424116 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.424086 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rhg7w" Oct 31 07:29:47.424229 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.424207 2577 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Oct 31 07:29:47.428877 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.428487 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lmsk9" Oct 31 07:29:47.444000 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.443941 2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-7t2zf" Oct 31 07:29:47.456511 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.456485 2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-7t2zf" Oct 31 07:29:47.491235 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.491080 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-48.us-east-2.compute.internal" event={"ID":"11f3ad8c54222373dd06005938b97a96","Type":"ContainerStarted","Data":"c4b192911d11550f7de0ff5884848f30c9f162951df39fa0707f7b8b1f85bc45"} Oct 31 07:29:47.492494 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.492468 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-48.us-east-2.compute.internal" event={"ID":"8927b882957b5b07fb0714896c6f1a7f","Type":"ContainerStarted","Data":"c8e97bb91fa7a4579a6eb8f24f665103772028210f2a68df470a03c9f5841638"} Oct 31 07:29:47.508634 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.508612 2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Oct 31 07:29:47.509520 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.509496 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-jjzj5\"" Oct 31 07:29:47.509624 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.509573 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Oct 31 07:29:47.509624 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.509577 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-jshc6\"" Oct 31 07:29:47.509744 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.509675 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ssdfd\"" Oct 31 07:29:47.511105 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.511084 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Oct 31 07:29:47.511186 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.511173 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-sl77n\"" Oct 31 07:29:47.511258 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.511239 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-x8m6s\"" Oct 31 07:29:47.511330 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.511294 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Oct 31 07:29:47.511407 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.511361 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-hztdx\"" Oct 31 07:29:47.511526 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.511512 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Oct 31 07:29:47.511620 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.511606 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Oct 31 07:29:47.511684 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.511673 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Oct 31 07:29:47.511763 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.511742 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Oct 31 07:29:47.511842 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.511830 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Oct 31 07:29:47.511910 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.511897 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Oct 31 07:29:47.511991 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.511979 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Oct 31 07:29:47.512065 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.512045 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Oct 31 07:29:47.512136 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.512124 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Oct 31 07:29:47.512235 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.512221 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Oct 31 07:29:47.512314 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.512302 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Oct 31 07:29:47.512376 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.512364 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Oct 31 07:29:47.518797 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.518771 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-host-cni-bin\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.518904 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.518818 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzvgd\" (UniqueName: \"kubernetes.io/projected/53c36e19-9ef6-47a2-9bc2-387115539850-kube-api-access-zzvgd\") pod \"node-resolver-scrms\" (UID: \"53c36e19-9ef6-47a2-9bc2-387115539850\") " pod="openshift-dns/node-resolver-scrms" Oct 31 07:29:47.518904 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.518848 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ea9beb38-1491-4cf0-83ca-fd79935fa6a9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r78n8\" (UID: \"ea9beb38-1491-4cf0-83ca-fd79935fa6a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r78n8" Oct 31 07:29:47.518904 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.518875 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-lib-modules\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:29:47.518904 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.518900 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/236238a7-255d-40f7-a99b-d65903b3bcb7-serviceca\") pod \"node-ca-rhg7w\" (UID: \"236238a7-255d-40f7-a99b-d65903b3bcb7\") " pod="openshift-image-registry/node-ca-rhg7w" Oct 31 07:29:47.519114 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.518927 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-host-var-lib-cni-bin\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:29:47.519114 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.518959 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-cni-binary-copy\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:29:47.519114 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.518985 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-host-run-netns\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:29:47.519114 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.519009 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53c36e19-9ef6-47a2-9bc2-387115539850-tmp-dir\") pod \"node-resolver-scrms\" (UID: \"53c36e19-9ef6-47a2-9bc2-387115539850\") " pod="openshift-dns/node-resolver-scrms" Oct 31 07:29:47.519114 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.519034 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-etc-kubernetes\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:29:47.519114 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.519059 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xv5b5\" (UniqueName: \"kubernetes.io/projected/ac4bf75a-377c-49b2-818a-7d2436c21e9c-kube-api-access-xv5b5\") pod \"iptables-alerter-lmsk9\" (UID: \"ac4bf75a-377c-49b2-818a-7d2436c21e9c\") " pod="openshift-network-operator/iptables-alerter-lmsk9" Oct 31 07:29:47.519114 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.519084 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-cnibin\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:29:47.519492 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.519117 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-hostroot\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:29:47.519492 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.519171 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-multus-daemon-config\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:29:47.519492 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.519197 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-ovnkube-script-lib\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.519492 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.519223 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrvrg\" (UniqueName: \"kubernetes.io/projected/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-kube-api-access-mrvrg\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.519492 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.519258 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-etc-systemd\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:29:47.519492 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.519289 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-system-cni-dir\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:29:47.519492 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.519320 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-host-kubelet\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.519492 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.519347 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ea9beb38-1491-4cf0-83ca-fd79935fa6a9-sys-fs\") pod \"aws-ebs-csi-driver-node-r78n8\" (UID: \"ea9beb38-1491-4cf0-83ca-fd79935fa6a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r78n8" Oct 31 07:29:47.519492 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.519384 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-etc-modprobe-d\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:29:47.519492 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.519433 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d25f2854-fd38-44e7-ad7e-da744d57e671-konnectivity-ca\") pod \"konnectivity-agent-pl5cl\" (UID: \"d25f2854-fd38-44e7-ad7e-da744d57e671\") " pod="kube-system/konnectivity-agent-pl5cl" Oct 31 07:29:47.519492 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.519458 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/297b35d7-aa65-49c7-8460-ac3f9860248c-dbus\") pod \"global-pull-secret-syncer-8hpcb\" (UID: \"297b35d7-aa65-49c7-8460-ac3f9860248c\") " pod="kube-system/global-pull-secret-syncer-8hpcb" Oct 31 07:29:47.519492 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.519485 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/297b35d7-aa65-49c7-8460-ac3f9860248c-original-pull-secret\") pod \"global-pull-secret-syncer-8hpcb\" (UID: \"297b35d7-aa65-49c7-8460-ac3f9860248c\") " pod="kube-system/global-pull-secret-syncer-8hpcb" Oct 31 07:29:47.520097 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.519510 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-node-log\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.520097 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.519542 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.520097 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.519581 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-ovn-node-metrics-cert\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.520097 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.519611 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ed522b38-611a-452a-b201-3ca416428d42-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ktzg4\" (UID: \"ed522b38-611a-452a-b201-3ca416428d42\") " pod="openshift-multus/multus-additional-cni-plugins-ktzg4" Oct 31 07:29:47.520097 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.519644 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ea9beb38-1491-4cf0-83ca-fd79935fa6a9-registration-dir\") pod \"aws-ebs-csi-driver-node-r78n8\" (UID: \"ea9beb38-1491-4cf0-83ca-fd79935fa6a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r78n8" Oct 31 07:29:47.520097 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.519694 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrj54\" (UniqueName: \"kubernetes.io/projected/ea9beb38-1491-4cf0-83ca-fd79935fa6a9-kube-api-access-wrj54\") pod \"aws-ebs-csi-driver-node-r78n8\" (UID: \"ea9beb38-1491-4cf0-83ca-fd79935fa6a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r78n8" Oct 31 07:29:47.520097 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.519719 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-etc-sysconfig\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:29:47.520097 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.519746 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76x7d\" (UniqueName: \"kubernetes.io/projected/046b1de8-d709-4729-9aec-391a2b3294d5-kube-api-access-76x7d\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:29:47.520097 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.519772 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-run-ovn\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.520097 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.519798 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-host-run-ovn-kubernetes\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.520097 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.519827 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-env-overrides\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.520097 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.519854 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ed522b38-611a-452a-b201-3ca416428d42-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ktzg4\" (UID: \"ed522b38-611a-452a-b201-3ca416428d42\") " pod="openshift-multus/multus-additional-cni-plugins-ktzg4" Oct 31 07:29:47.520097 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.519880 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-997f5\" (UniqueName: \"kubernetes.io/projected/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-kube-api-access-997f5\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:29:47.520097 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.519907 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-run-openvswitch\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.520097 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.519932 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28ec9621-7ec5-421c-a88b-51bb20d3a7bf-metrics-certs\") pod \"network-metrics-daemon-7thxr\" (UID: \"28ec9621-7ec5-421c-a88b-51bb20d3a7bf\") " pod="openshift-multus/network-metrics-daemon-7thxr" Oct 31 07:29:47.520097 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.519956 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-host\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:29:47.520920 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.519981 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/046b1de8-d709-4729-9aec-391a2b3294d5-etc-tuned\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:29:47.520920 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.520012 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-multus-socket-dir-parent\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:29:47.520920 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.520037 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-run-systemd\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.520920 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.520066 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ed522b38-611a-452a-b201-3ca416428d42-cnibin\") pod \"multus-additional-cni-plugins-ktzg4\" (UID: \"ed522b38-611a-452a-b201-3ca416428d42\") " pod="openshift-multus/multus-additional-cni-plugins-ktzg4" Oct 31 07:29:47.520920 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.520095 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ea9beb38-1491-4cf0-83ca-fd79935fa6a9-socket-dir\") pod \"aws-ebs-csi-driver-node-r78n8\" (UID: \"ea9beb38-1491-4cf0-83ca-fd79935fa6a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r78n8" Oct 31 07:29:47.520920 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.520131 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ac4bf75a-377c-49b2-818a-7d2436c21e9c-host-slash\") pod \"iptables-alerter-lmsk9\" (UID: \"ac4bf75a-377c-49b2-818a-7d2436c21e9c\") " pod="openshift-network-operator/iptables-alerter-lmsk9" Oct 31 07:29:47.520920 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.520166 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-multus-cni-dir\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:29:47.520920 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.520194 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-host-run-k8s-cni-cncf-io\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:29:47.520920 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.520222 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mqks\" (UniqueName: \"kubernetes.io/projected/28ec9621-7ec5-421c-a88b-51bb20d3a7bf-kube-api-access-8mqks\") pod \"network-metrics-daemon-7thxr\" (UID: \"28ec9621-7ec5-421c-a88b-51bb20d3a7bf\") " pod="openshift-multus/network-metrics-daemon-7thxr" Oct 31 07:29:47.520920 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.520254 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-etc-sysctl-conf\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:29:47.520920 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.520282 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-host-var-lib-cni-multus\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:29:47.520920 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.520307 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-host-cni-netd\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.520920 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.520334 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-host-var-lib-kubelet\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:29:47.520920 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.520360 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-systemd-units\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.520920 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.520385 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-log-socket\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.520920 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.520413 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ac4bf75a-377c-49b2-818a-7d2436c21e9c-iptables-alerter-script\") pod \"iptables-alerter-lmsk9\" (UID: \"ac4bf75a-377c-49b2-818a-7d2436c21e9c\") " pod="openshift-network-operator/iptables-alerter-lmsk9" Oct 31 07:29:47.520920 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.520456 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-host-run-netns\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.521655 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.520482 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ed522b38-611a-452a-b201-3ca416428d42-system-cni-dir\") pod \"multus-additional-cni-plugins-ktzg4\" (UID: \"ed522b38-611a-452a-b201-3ca416428d42\") " pod="openshift-multus/multus-additional-cni-plugins-ktzg4" Oct 31 07:29:47.521655 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.520510 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ed522b38-611a-452a-b201-3ca416428d42-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ktzg4\" (UID: \"ed522b38-611a-452a-b201-3ca416428d42\") " pod="openshift-multus/multus-additional-cni-plugins-ktzg4" Oct 31 07:29:47.521655 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.520537 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-multus-conf-dir\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:29:47.521655 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.520567 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-etc-kubernetes\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:29:47.521655 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.520593 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-var-lib-openvswitch\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.521655 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.520621 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4tdk\" (UniqueName: \"kubernetes.io/projected/ed522b38-611a-452a-b201-3ca416428d42-kube-api-access-w4tdk\") pod \"multus-additional-cni-plugins-ktzg4\" (UID: \"ed522b38-611a-452a-b201-3ca416428d42\") " pod="openshift-multus/multus-additional-cni-plugins-ktzg4" Oct 31 07:29:47.521655 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.520654 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/53c36e19-9ef6-47a2-9bc2-387115539850-hosts-file\") pod \"node-resolver-scrms\" (UID: \"53c36e19-9ef6-47a2-9bc2-387115539850\") " pod="openshift-dns/node-resolver-scrms" Oct 31 07:29:47.521655 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.520679 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-run\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:29:47.521655 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.520711 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ed522b38-611a-452a-b201-3ca416428d42-cni-binary-copy\") pod \"multus-additional-cni-plugins-ktzg4\" (UID: \"ed522b38-611a-452a-b201-3ca416428d42\") " pod="openshift-multus/multus-additional-cni-plugins-ktzg4" Oct 31 07:29:47.521655 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.520746 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-etc-sysctl-d\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:29:47.521655 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.520774 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-var-lib-kubelet\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:29:47.521655 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.520808 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d25f2854-fd38-44e7-ad7e-da744d57e671-agent-certs\") pod \"konnectivity-agent-pl5cl\" (UID: \"d25f2854-fd38-44e7-ad7e-da744d57e671\") " pod="kube-system/konnectivity-agent-pl5cl" Oct 31 07:29:47.521655 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.520845 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-host-slash\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.521655 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.520875 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-ovnkube-config\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.521655 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.520905 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ea9beb38-1491-4cf0-83ca-fd79935fa6a9-etc-selinux\") pod \"aws-ebs-csi-driver-node-r78n8\" (UID: \"ea9beb38-1491-4cf0-83ca-fd79935fa6a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r78n8" Oct 31 07:29:47.521655 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.520932 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-sys\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:29:47.521655 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.520957 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/046b1de8-d709-4729-9aec-391a2b3294d5-tmp\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:29:47.522293 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.520985 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftf9v\" (UniqueName: \"kubernetes.io/projected/236238a7-255d-40f7-a99b-d65903b3bcb7-kube-api-access-ftf9v\") pod \"node-ca-rhg7w\" (UID: \"236238a7-255d-40f7-a99b-d65903b3bcb7\") " pod="openshift-image-registry/node-ca-rhg7w" Oct 31 07:29:47.522293 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.521015 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ckzf\" (UniqueName: \"kubernetes.io/projected/d8b28b9f-37a2-4859-87fd-d27bb37d832b-kube-api-access-2ckzf\") pod \"network-check-target-ncnd7\" (UID: \"d8b28b9f-37a2-4859-87fd-d27bb37d832b\") " pod="openshift-network-diagnostics/network-check-target-ncnd7" Oct 31 07:29:47.522293 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.521045 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-host-run-multus-certs\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:29:47.522293 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.521074 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ea9beb38-1491-4cf0-83ca-fd79935fa6a9-device-dir\") pod \"aws-ebs-csi-driver-node-r78n8\" (UID: \"ea9beb38-1491-4cf0-83ca-fd79935fa6a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r78n8" Oct 31 07:29:47.522293 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.521102 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/297b35d7-aa65-49c7-8460-ac3f9860248c-kubelet-config\") pod \"global-pull-secret-syncer-8hpcb\" (UID: \"297b35d7-aa65-49c7-8460-ac3f9860248c\") " pod="kube-system/global-pull-secret-syncer-8hpcb" Oct 31 07:29:47.522293 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.521127 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/236238a7-255d-40f7-a99b-d65903b3bcb7-host\") pod \"node-ca-rhg7w\" (UID: \"236238a7-255d-40f7-a99b-d65903b3bcb7\") " pod="openshift-image-registry/node-ca-rhg7w" Oct 31 07:29:47.522293 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.521161 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-etc-openvswitch\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.522293 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.521252 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-etc-openvswitch\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.522293 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.521301 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-host-cni-bin\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.522293 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.521439 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-host-var-lib-cni-bin\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:29:47.522293 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.522026 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-cni-binary-copy\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:29:47.522293 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.522120 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-host-run-netns\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:29:47.522293 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.522253 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-cnibin\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:29:47.522799 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.522300 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-hostroot\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:29:47.522799 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.522754 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-multus-daemon-config\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:29:47.523298 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.523278 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-ovnkube-script-lib\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.523682 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.523655 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-system-cni-dir\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:29:47.523787 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.523712 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-host-kubelet\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.523915 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.523835 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-node-log\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.523961 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.523915 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.524252 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.524230 2577 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Oct 31 07:29:47.524633 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.524602 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-systemd-units\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.525174 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.525149 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ed522b38-611a-452a-b201-3ca416428d42-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ktzg4\" (UID: \"ed522b38-611a-452a-b201-3ca416428d42\") " pod="openshift-multus/multus-additional-cni-plugins-ktzg4" Oct 31 07:29:47.525315 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.525292 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-run-ovn\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.525458 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.525348 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-host-run-ovn-kubernetes\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.525759 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.525737 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-env-overrides\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.526193 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.526171 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ed522b38-611a-452a-b201-3ca416428d42-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ktzg4\" (UID: \"ed522b38-611a-452a-b201-3ca416428d42\") " pod="openshift-multus/multus-additional-cni-plugins-ktzg4" Oct 31 07:29:47.526478 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.526458 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-run-openvswitch\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.526570 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:47.526555 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 31 07:29:47.526628 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:47.526620 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28ec9621-7ec5-421c-a88b-51bb20d3a7bf-metrics-certs podName:28ec9621-7ec5-421c-a88b-51bb20d3a7bf nodeName:}" failed. No retries permitted until 2026-10-31 07:29:48.026600136 +0000 UTC m=+3.035176571 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/28ec9621-7ec5-421c-a88b-51bb20d3a7bf-metrics-certs") pod "network-metrics-daemon-7thxr" (UID: "28ec9621-7ec5-421c-a88b-51bb20d3a7bf") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 31 07:29:47.526963 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.526945 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-multus-socket-dir-parent\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:29:47.527027 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.527002 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-run-systemd\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.527074 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.527051 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ed522b38-611a-452a-b201-3ca416428d42-cnibin\") pod \"multus-additional-cni-plugins-ktzg4\" (UID: \"ed522b38-611a-452a-b201-3ca416428d42\") " pod="openshift-multus/multus-additional-cni-plugins-ktzg4" Oct 31 07:29:47.527214 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.527197 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-multus-cni-dir\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:29:47.527265 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.527250 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-host-run-k8s-cni-cncf-io\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:29:47.527551 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.527520 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-host-var-lib-cni-multus\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:29:47.527627 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.527581 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-host-cni-netd\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.527664 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.527624 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-host-var-lib-kubelet\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:29:47.528584 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.528483 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-log-socket\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.528584 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.528554 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ed522b38-611a-452a-b201-3ca416428d42-system-cni-dir\") pod \"multus-additional-cni-plugins-ktzg4\" (UID: \"ed522b38-611a-452a-b201-3ca416428d42\") " pod="openshift-multus/multus-additional-cni-plugins-ktzg4" Oct 31 07:29:47.528735 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.528628 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-host-slash\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.528735 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.528655 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ed522b38-611a-452a-b201-3ca416428d42-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ktzg4\" (UID: \"ed522b38-611a-452a-b201-3ca416428d42\") " pod="openshift-multus/multus-additional-cni-plugins-ktzg4" Oct 31 07:29:47.528735 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.528677 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-host-run-multus-certs\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:29:47.528735 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.528677 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-var-lib-openvswitch\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.528735 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.528718 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-multus-conf-dir\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:29:47.528735 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.528709 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-host-run-netns\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.528735 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.528732 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-etc-kubernetes\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:29:47.529346 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.529325 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ed522b38-611a-452a-b201-3ca416428d42-cni-binary-copy\") pod \"multus-additional-cni-plugins-ktzg4\" (UID: \"ed522b38-611a-452a-b201-3ca416428d42\") " pod="openshift-multus/multus-additional-cni-plugins-ktzg4" Oct 31 07:29:47.529474 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.529455 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-ovnkube-config\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.529911 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.529891 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-ovn-node-metrics-cert\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.537415 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.537392 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrvrg\" (UniqueName: \"kubernetes.io/projected/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-kube-api-access-mrvrg\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.539161 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:47.538852 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 31 07:29:47.539161 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:47.538874 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 31 07:29:47.539161 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.538878 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-997f5\" (UniqueName: \"kubernetes.io/projected/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-kube-api-access-997f5\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:29:47.539161 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:47.538888 2577 projected.go:196] Error preparing data for projected volume kube-api-access-2ckzf for pod openshift-network-diagnostics/network-check-target-ncnd7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 31 07:29:47.539161 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:47.538970 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d8b28b9f-37a2-4859-87fd-d27bb37d832b-kube-api-access-2ckzf podName:d8b28b9f-37a2-4859-87fd-d27bb37d832b nodeName:}" failed. No retries permitted until 2026-10-31 07:29:48.038945427 +0000 UTC m=+3.047521841 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-2ckzf" (UniqueName: "kubernetes.io/projected/d8b28b9f-37a2-4859-87fd-d27bb37d832b-kube-api-access-2ckzf") pod "network-check-target-ncnd7" (UID: "d8b28b9f-37a2-4859-87fd-d27bb37d832b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 31 07:29:47.541442 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.541399 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mqks\" (UniqueName: \"kubernetes.io/projected/28ec9621-7ec5-421c-a88b-51bb20d3a7bf-kube-api-access-8mqks\") pod \"network-metrics-daemon-7thxr\" (UID: \"28ec9621-7ec5-421c-a88b-51bb20d3a7bf\") " pod="openshift-multus/network-metrics-daemon-7thxr" Oct 31 07:29:47.543098 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.543075 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4tdk\" (UniqueName: \"kubernetes.io/projected/ed522b38-611a-452a-b201-3ca416428d42-kube-api-access-w4tdk\") pod \"multus-additional-cni-plugins-ktzg4\" (UID: \"ed522b38-611a-452a-b201-3ca416428d42\") " pod="openshift-multus/multus-additional-cni-plugins-ktzg4" Oct 31 07:29:47.621832 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.621795 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzvgd\" (UniqueName: \"kubernetes.io/projected/53c36e19-9ef6-47a2-9bc2-387115539850-kube-api-access-zzvgd\") pod \"node-resolver-scrms\" (UID: \"53c36e19-9ef6-47a2-9bc2-387115539850\") " pod="openshift-dns/node-resolver-scrms" Oct 31 07:29:47.621832 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.621828 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ea9beb38-1491-4cf0-83ca-fd79935fa6a9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r78n8\" (UID: \"ea9beb38-1491-4cf0-83ca-fd79935fa6a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r78n8" Oct 31 07:29:47.621832 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.621844 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-lib-modules\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:29:47.622156 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.621860 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/236238a7-255d-40f7-a99b-d65903b3bcb7-serviceca\") pod \"node-ca-rhg7w\" (UID: \"236238a7-255d-40f7-a99b-d65903b3bcb7\") " pod="openshift-image-registry/node-ca-rhg7w" Oct 31 07:29:47.622156 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.621924 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53c36e19-9ef6-47a2-9bc2-387115539850-tmp-dir\") pod \"node-resolver-scrms\" (UID: \"53c36e19-9ef6-47a2-9bc2-387115539850\") " pod="openshift-dns/node-resolver-scrms" Oct 31 07:29:47.622156 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.621941 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ea9beb38-1491-4cf0-83ca-fd79935fa6a9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r78n8\" (UID: \"ea9beb38-1491-4cf0-83ca-fd79935fa6a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r78n8" Oct 31 07:29:47.622156 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.621973 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-etc-kubernetes\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:29:47.622156 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.622007 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xv5b5\" (UniqueName: \"kubernetes.io/projected/ac4bf75a-377c-49b2-818a-7d2436c21e9c-kube-api-access-xv5b5\") pod \"iptables-alerter-lmsk9\" (UID: \"ac4bf75a-377c-49b2-818a-7d2436c21e9c\") " pod="openshift-network-operator/iptables-alerter-lmsk9" Oct 31 07:29:47.622156 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.622023 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-lib-modules\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:29:47.622156 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.622038 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-etc-systemd\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:29:47.622156 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.622065 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ea9beb38-1491-4cf0-83ca-fd79935fa6a9-sys-fs\") pod \"aws-ebs-csi-driver-node-r78n8\" (UID: \"ea9beb38-1491-4cf0-83ca-fd79935fa6a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r78n8" Oct 31 07:29:47.622156 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.622084 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-etc-kubernetes\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:29:47.622156 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.622105 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-etc-modprobe-d\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:29:47.622156 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.622131 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d25f2854-fd38-44e7-ad7e-da744d57e671-konnectivity-ca\") pod \"konnectivity-agent-pl5cl\" (UID: \"d25f2854-fd38-44e7-ad7e-da744d57e671\") " pod="kube-system/konnectivity-agent-pl5cl" Oct 31 07:29:47.622725 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.622154 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/297b35d7-aa65-49c7-8460-ac3f9860248c-dbus\") pod \"global-pull-secret-syncer-8hpcb\" (UID: \"297b35d7-aa65-49c7-8460-ac3f9860248c\") " pod="kube-system/global-pull-secret-syncer-8hpcb" Oct 31 07:29:47.622725 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.622192 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/297b35d7-aa65-49c7-8460-ac3f9860248c-original-pull-secret\") pod \"global-pull-secret-syncer-8hpcb\" (UID: \"297b35d7-aa65-49c7-8460-ac3f9860248c\") " pod="kube-system/global-pull-secret-syncer-8hpcb" Oct 31 07:29:47.622725 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.622337 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-etc-modprobe-d\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:29:47.622725 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.622569 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/236238a7-255d-40f7-a99b-d65903b3bcb7-serviceca\") pod \"node-ca-rhg7w\" (UID: \"236238a7-255d-40f7-a99b-d65903b3bcb7\") " pod="openshift-image-registry/node-ca-rhg7w" Oct 31 07:29:47.622725 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.622629 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53c36e19-9ef6-47a2-9bc2-387115539850-tmp-dir\") pod \"node-resolver-scrms\" (UID: \"53c36e19-9ef6-47a2-9bc2-387115539850\") " pod="openshift-dns/node-resolver-scrms" Oct 31 07:29:47.622725 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.622663 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ea9beb38-1491-4cf0-83ca-fd79935fa6a9-registration-dir\") pod \"aws-ebs-csi-driver-node-r78n8\" (UID: \"ea9beb38-1491-4cf0-83ca-fd79935fa6a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r78n8" Oct 31 07:29:47.622725 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.622729 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-etc-systemd\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:29:47.623111 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.622744 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/297b35d7-aa65-49c7-8460-ac3f9860248c-dbus\") pod \"global-pull-secret-syncer-8hpcb\" (UID: \"297b35d7-aa65-49c7-8460-ac3f9860248c\") " pod="kube-system/global-pull-secret-syncer-8hpcb" Oct 31 07:29:47.623111 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.622785 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ea9beb38-1491-4cf0-83ca-fd79935fa6a9-sys-fs\") pod \"aws-ebs-csi-driver-node-r78n8\" (UID: \"ea9beb38-1491-4cf0-83ca-fd79935fa6a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r78n8" Oct 31 07:29:47.623111 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.622788 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrj54\" (UniqueName: \"kubernetes.io/projected/ea9beb38-1491-4cf0-83ca-fd79935fa6a9-kube-api-access-wrj54\") pod \"aws-ebs-csi-driver-node-r78n8\" (UID: \"ea9beb38-1491-4cf0-83ca-fd79935fa6a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r78n8" Oct 31 07:29:47.623111 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.622838 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d25f2854-fd38-44e7-ad7e-da744d57e671-konnectivity-ca\") pod \"konnectivity-agent-pl5cl\" (UID: \"d25f2854-fd38-44e7-ad7e-da744d57e671\") " pod="kube-system/konnectivity-agent-pl5cl" Oct 31 07:29:47.623111 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:47.622880 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 31 07:29:47.623111 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:47.622938 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/297b35d7-aa65-49c7-8460-ac3f9860248c-original-pull-secret podName:297b35d7-aa65-49c7-8460-ac3f9860248c nodeName:}" failed. No retries permitted until 2026-10-31 07:29:48.122920119 +0000 UTC m=+3.131496542 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/297b35d7-aa65-49c7-8460-ac3f9860248c-original-pull-secret") pod "global-pull-secret-syncer-8hpcb" (UID: "297b35d7-aa65-49c7-8460-ac3f9860248c") : object "kube-system"/"original-pull-secret" not registered Oct 31 07:29:47.623111 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.623042 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-etc-sysconfig\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:29:47.623111 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.623080 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ea9beb38-1491-4cf0-83ca-fd79935fa6a9-registration-dir\") pod \"aws-ebs-csi-driver-node-r78n8\" (UID: \"ea9beb38-1491-4cf0-83ca-fd79935fa6a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r78n8" Oct 31 07:29:47.623547 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.623130 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76x7d\" (UniqueName: \"kubernetes.io/projected/046b1de8-d709-4729-9aec-391a2b3294d5-kube-api-access-76x7d\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:29:47.623547 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.623162 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-etc-sysconfig\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:29:47.623547 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.623325 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-host\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:29:47.623547 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.623379 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/046b1de8-d709-4729-9aec-391a2b3294d5-etc-tuned\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:29:47.623547 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.623406 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ea9beb38-1491-4cf0-83ca-fd79935fa6a9-socket-dir\") pod \"aws-ebs-csi-driver-node-r78n8\" (UID: \"ea9beb38-1491-4cf0-83ca-fd79935fa6a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r78n8" Oct 31 07:29:47.623547 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.623409 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-host\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:29:47.623547 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.623424 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ac4bf75a-377c-49b2-818a-7d2436c21e9c-host-slash\") pod \"iptables-alerter-lmsk9\" (UID: \"ac4bf75a-377c-49b2-818a-7d2436c21e9c\") " pod="openshift-network-operator/iptables-alerter-lmsk9" Oct 31 07:29:47.623547 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.623478 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ac4bf75a-377c-49b2-818a-7d2436c21e9c-host-slash\") pod \"iptables-alerter-lmsk9\" (UID: \"ac4bf75a-377c-49b2-818a-7d2436c21e9c\") " pod="openshift-network-operator/iptables-alerter-lmsk9" Oct 31 07:29:47.623547 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.623481 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-etc-sysctl-conf\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:29:47.623547 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.623524 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ac4bf75a-377c-49b2-818a-7d2436c21e9c-iptables-alerter-script\") pod \"iptables-alerter-lmsk9\" (UID: \"ac4bf75a-377c-49b2-818a-7d2436c21e9c\") " pod="openshift-network-operator/iptables-alerter-lmsk9" Oct 31 07:29:47.624039 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.623561 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/53c36e19-9ef6-47a2-9bc2-387115539850-hosts-file\") pod \"node-resolver-scrms\" (UID: \"53c36e19-9ef6-47a2-9bc2-387115539850\") " pod="openshift-dns/node-resolver-scrms" Oct 31 07:29:47.624039 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.623590 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-run\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:29:47.624039 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.623622 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-etc-sysctl-d\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:29:47.624039 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.623653 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-var-lib-kubelet\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:29:47.624039 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.623667 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-run\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:29:47.624039 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.623683 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d25f2854-fd38-44e7-ad7e-da744d57e671-agent-certs\") pod \"konnectivity-agent-pl5cl\" (UID: \"d25f2854-fd38-44e7-ad7e-da744d57e671\") " pod="kube-system/konnectivity-agent-pl5cl" Oct 31 07:29:47.624039 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.623717 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ea9beb38-1491-4cf0-83ca-fd79935fa6a9-etc-selinux\") pod \"aws-ebs-csi-driver-node-r78n8\" (UID: \"ea9beb38-1491-4cf0-83ca-fd79935fa6a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r78n8" Oct 31 07:29:47.624039 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.623727 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/53c36e19-9ef6-47a2-9bc2-387115539850-hosts-file\") pod \"node-resolver-scrms\" (UID: \"53c36e19-9ef6-47a2-9bc2-387115539850\") " pod="openshift-dns/node-resolver-scrms" Oct 31 07:29:47.624039 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.623748 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-sys\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:29:47.624039 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.623622 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-etc-sysctl-conf\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:29:47.624039 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.623577 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ea9beb38-1491-4cf0-83ca-fd79935fa6a9-socket-dir\") pod \"aws-ebs-csi-driver-node-r78n8\" (UID: \"ea9beb38-1491-4cf0-83ca-fd79935fa6a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r78n8" Oct 31 07:29:47.624039 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.623802 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-sys\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:29:47.624039 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.623795 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/046b1de8-d709-4729-9aec-391a2b3294d5-tmp\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:29:47.624039 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.623844 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftf9v\" (UniqueName: \"kubernetes.io/projected/236238a7-255d-40f7-a99b-d65903b3bcb7-kube-api-access-ftf9v\") pod \"node-ca-rhg7w\" (UID: \"236238a7-255d-40f7-a99b-d65903b3bcb7\") " pod="openshift-image-registry/node-ca-rhg7w" Oct 31 07:29:47.624039 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.623889 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ea9beb38-1491-4cf0-83ca-fd79935fa6a9-device-dir\") pod \"aws-ebs-csi-driver-node-r78n8\" (UID: \"ea9beb38-1491-4cf0-83ca-fd79935fa6a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r78n8" Oct 31 07:29:47.624039 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.623896 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-etc-sysctl-d\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:29:47.624039 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.623918 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/297b35d7-aa65-49c7-8460-ac3f9860248c-kubelet-config\") pod \"global-pull-secret-syncer-8hpcb\" (UID: \"297b35d7-aa65-49c7-8460-ac3f9860248c\") " pod="kube-system/global-pull-secret-syncer-8hpcb" Oct 31 07:29:47.624039 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.623945 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/236238a7-255d-40f7-a99b-d65903b3bcb7-host\") pod \"node-ca-rhg7w\" (UID: \"236238a7-255d-40f7-a99b-d65903b3bcb7\") " pod="openshift-image-registry/node-ca-rhg7w" Oct 31 07:29:47.624982 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.623994 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ea9beb38-1491-4cf0-83ca-fd79935fa6a9-etc-selinux\") pod \"aws-ebs-csi-driver-node-r78n8\" (UID: \"ea9beb38-1491-4cf0-83ca-fd79935fa6a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r78n8" Oct 31 07:29:47.624982 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.624024 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/236238a7-255d-40f7-a99b-d65903b3bcb7-host\") pod \"node-ca-rhg7w\" (UID: \"236238a7-255d-40f7-a99b-d65903b3bcb7\") " pod="openshift-image-registry/node-ca-rhg7w" Oct 31 07:29:47.624982 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.624051 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-var-lib-kubelet\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:29:47.624982 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.624248 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ea9beb38-1491-4cf0-83ca-fd79935fa6a9-device-dir\") pod \"aws-ebs-csi-driver-node-r78n8\" (UID: \"ea9beb38-1491-4cf0-83ca-fd79935fa6a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r78n8" Oct 31 07:29:47.624982 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.624282 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/297b35d7-aa65-49c7-8460-ac3f9860248c-kubelet-config\") pod \"global-pull-secret-syncer-8hpcb\" (UID: \"297b35d7-aa65-49c7-8460-ac3f9860248c\") " pod="kube-system/global-pull-secret-syncer-8hpcb" Oct 31 07:29:47.624982 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.624347 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ac4bf75a-377c-49b2-818a-7d2436c21e9c-iptables-alerter-script\") pod \"iptables-alerter-lmsk9\" (UID: \"ac4bf75a-377c-49b2-818a-7d2436c21e9c\") " pod="openshift-network-operator/iptables-alerter-lmsk9" Oct 31 07:29:47.627020 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.626993 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d25f2854-fd38-44e7-ad7e-da744d57e671-agent-certs\") pod \"konnectivity-agent-pl5cl\" (UID: \"d25f2854-fd38-44e7-ad7e-da744d57e671\") " pod="kube-system/konnectivity-agent-pl5cl" Oct 31 07:29:47.627869 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.627843 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/046b1de8-d709-4729-9aec-391a2b3294d5-etc-tuned\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:29:47.628571 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.628550 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/046b1de8-d709-4729-9aec-391a2b3294d5-tmp\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:29:47.635340 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.635319 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzvgd\" (UniqueName: \"kubernetes.io/projected/53c36e19-9ef6-47a2-9bc2-387115539850-kube-api-access-zzvgd\") pod \"node-resolver-scrms\" (UID: \"53c36e19-9ef6-47a2-9bc2-387115539850\") " pod="openshift-dns/node-resolver-scrms" Oct 31 07:29:47.636513 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.636489 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrj54\" (UniqueName: \"kubernetes.io/projected/ea9beb38-1491-4cf0-83ca-fd79935fa6a9-kube-api-access-wrj54\") pod \"aws-ebs-csi-driver-node-r78n8\" (UID: \"ea9beb38-1491-4cf0-83ca-fd79935fa6a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r78n8" Oct 31 07:29:47.636991 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.636968 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xv5b5\" (UniqueName: \"kubernetes.io/projected/ac4bf75a-377c-49b2-818a-7d2436c21e9c-kube-api-access-xv5b5\") pod \"iptables-alerter-lmsk9\" (UID: \"ac4bf75a-377c-49b2-818a-7d2436c21e9c\") " pod="openshift-network-operator/iptables-alerter-lmsk9" Oct 31 07:29:47.642122 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.637646 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftf9v\" (UniqueName: \"kubernetes.io/projected/236238a7-255d-40f7-a99b-d65903b3bcb7-kube-api-access-ftf9v\") pod \"node-ca-rhg7w\" (UID: \"236238a7-255d-40f7-a99b-d65903b3bcb7\") " pod="openshift-image-registry/node-ca-rhg7w" Oct 31 07:29:47.642122 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.639630 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-76x7d\" (UniqueName: \"kubernetes.io/projected/046b1de8-d709-4729-9aec-391a2b3294d5-kube-api-access-76x7d\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:29:47.714546 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.714474 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-w6vnh" Oct 31 07:29:47.727421 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:47.727394 2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ae10413_8f49_4dfe_8aad_00dcb9d390c8.slice/crio-9a2eb5642a9bcd6ecb79d58106e89b4b13ee221824aba714bee38dfaf9f57a8a WatchSource:0}: Error finding container 9a2eb5642a9bcd6ecb79d58106e89b4b13ee221824aba714bee38dfaf9f57a8a: Status 404 returned error can't find the container with id 9a2eb5642a9bcd6ecb79d58106e89b4b13ee221824aba714bee38dfaf9f57a8a Oct 31 07:29:47.746719 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.746687 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:29:47.751195 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.751170 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-scrms" Oct 31 07:29:47.758065 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:47.758037 2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode00eb9e3_0d3c_4541_b238_4130fe1de2f1.slice/crio-b4a16403faa883c4260e5c78aead776d888c71eb79b7d356dd8b397d49d3b4da WatchSource:0}: Error finding container b4a16403faa883c4260e5c78aead776d888c71eb79b7d356dd8b397d49d3b4da: Status 404 returned error can't find the container with id b4a16403faa883c4260e5c78aead776d888c71eb79b7d356dd8b397d49d3b4da Oct 31 07:29:47.762541 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:47.762512 2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53c36e19_9ef6_47a2_9bc2_387115539850.slice/crio-49788a1d09e2bb192e4b46ecea1b481cb3df28ad0cbb7feb731e2765844db5e2 WatchSource:0}: Error finding container 49788a1d09e2bb192e4b46ecea1b481cb3df28ad0cbb7feb731e2765844db5e2: Status 404 returned error can't find the container with id 49788a1d09e2bb192e4b46ecea1b481cb3df28ad0cbb7feb731e2765844db5e2 Oct 31 07:29:47.764740 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.764717 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:29:47.769284 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.769264 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r78n8" Oct 31 07:29:47.774534 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.774489 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ktzg4" Oct 31 07:29:47.776528 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.776499 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-pl5cl" Oct 31 07:29:47.776952 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:47.776925 2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod046b1de8_d709_4729_9aec_391a2b3294d5.slice/crio-b8f9976ff45a9a3b9246775990072df0586490e32925959c028535b18b777bad WatchSource:0}: Error finding container b8f9976ff45a9a3b9246775990072df0586490e32925959c028535b18b777bad: Status 404 returned error can't find the container with id b8f9976ff45a9a3b9246775990072df0586490e32925959c028535b18b777bad Oct 31 07:29:47.779529 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.779513 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rhg7w" Oct 31 07:29:47.781475 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:47.781399 2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea9beb38_1491_4cf0_83ca_fd79935fa6a9.slice/crio-a6992e8cfd7f4c537e4ca2c5d75ff78aaf7e9d831eb4dbaeec9dec17e03ec726 WatchSource:0}: Error finding container a6992e8cfd7f4c537e4ca2c5d75ff78aaf7e9d831eb4dbaeec9dec17e03ec726: Status 404 returned error can't find the container with id a6992e8cfd7f4c537e4ca2c5d75ff78aaf7e9d831eb4dbaeec9dec17e03ec726 Oct 31 07:29:47.787264 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:47.787234 2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded522b38_611a_452a_b201_3ca416428d42.slice/crio-7d73c3717618a3632d06b92aba13225d3694b19f95fb058748881de13dbcfd6e WatchSource:0}: Error finding container 7d73c3717618a3632d06b92aba13225d3694b19f95fb058748881de13dbcfd6e: Status 404 returned error can't find the container with id 7d73c3717618a3632d06b92aba13225d3694b19f95fb058748881de13dbcfd6e Oct 31 07:29:47.788055 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:47.787915 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lmsk9" Oct 31 07:29:47.799557 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:29:47.799529 2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac4bf75a_377c_49b2_818a_7d2436c21e9c.slice/crio-a32ed9582736eace8657e4c5e9e1521b6a93be7e5fb2ff212d0a71e3a22fc402 WatchSource:0}: Error finding container a32ed9582736eace8657e4c5e9e1521b6a93be7e5fb2ff212d0a71e3a22fc402: Status 404 returned error can't find the container with id a32ed9582736eace8657e4c5e9e1521b6a93be7e5fb2ff212d0a71e3a22fc402 Oct 31 07:29:48.027247 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:48.027181 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28ec9621-7ec5-421c-a88b-51bb20d3a7bf-metrics-certs\") pod \"network-metrics-daemon-7thxr\" (UID: \"28ec9621-7ec5-421c-a88b-51bb20d3a7bf\") " pod="openshift-multus/network-metrics-daemon-7thxr" Oct 31 07:29:48.027391 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:48.027305 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 31 07:29:48.027391 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:48.027372 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28ec9621-7ec5-421c-a88b-51bb20d3a7bf-metrics-certs podName:28ec9621-7ec5-421c-a88b-51bb20d3a7bf nodeName:}" failed. No retries permitted until 2026-10-31 07:29:49.02735043 +0000 UTC m=+4.035926860 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/28ec9621-7ec5-421c-a88b-51bb20d3a7bf-metrics-certs") pod "network-metrics-daemon-7thxr" (UID: "28ec9621-7ec5-421c-a88b-51bb20d3a7bf") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 31 07:29:48.128259 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:48.128223 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ckzf\" (UniqueName: \"kubernetes.io/projected/d8b28b9f-37a2-4859-87fd-d27bb37d832b-kube-api-access-2ckzf\") pod \"network-check-target-ncnd7\" (UID: \"d8b28b9f-37a2-4859-87fd-d27bb37d832b\") " pod="openshift-network-diagnostics/network-check-target-ncnd7" Oct 31 07:29:48.128398 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:48.128280 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/297b35d7-aa65-49c7-8460-ac3f9860248c-original-pull-secret\") pod \"global-pull-secret-syncer-8hpcb\" (UID: \"297b35d7-aa65-49c7-8460-ac3f9860248c\") " pod="kube-system/global-pull-secret-syncer-8hpcb" Oct 31 07:29:48.128476 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:48.128445 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 31 07:29:48.128533 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:48.128507 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/297b35d7-aa65-49c7-8460-ac3f9860248c-original-pull-secret podName:297b35d7-aa65-49c7-8460-ac3f9860248c nodeName:}" failed. No retries permitted until 2026-10-31 07:29:49.128489034 +0000 UTC m=+4.137065463 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/297b35d7-aa65-49c7-8460-ac3f9860248c-original-pull-secret") pod "global-pull-secret-syncer-8hpcb" (UID: "297b35d7-aa65-49c7-8460-ac3f9860248c") : object "kube-system"/"original-pull-secret" not registered Oct 31 07:29:48.128864 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:48.128844 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 31 07:29:48.128960 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:48.128868 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 31 07:29:48.128960 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:48.128880 2577 projected.go:196] Error preparing data for projected volume kube-api-access-2ckzf for pod openshift-network-diagnostics/network-check-target-ncnd7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 31 07:29:48.128960 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:48.128923 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d8b28b9f-37a2-4859-87fd-d27bb37d832b-kube-api-access-2ckzf podName:d8b28b9f-37a2-4859-87fd-d27bb37d832b nodeName:}" failed. No retries permitted until 2026-10-31 07:29:49.128907995 +0000 UTC m=+4.137484418 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-2ckzf" (UniqueName: "kubernetes.io/projected/d8b28b9f-37a2-4859-87fd-d27bb37d832b-kube-api-access-2ckzf") pod "network-check-target-ncnd7" (UID: "d8b28b9f-37a2-4859-87fd-d27bb37d832b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 31 07:29:48.460790 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:48.460618 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-30 07:24:47 +0000 UTC" deadline="2028-06-15 21:06:05.023752591 +0000 UTC" Oct 31 07:29:48.460790 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:48.460700 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14245h36m16.563057011s" Oct 31 07:29:48.500607 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:48.499768 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-48.us-east-2.compute.internal" event={"ID":"8927b882957b5b07fb0714896c6f1a7f","Type":"ContainerStarted","Data":"067c9d6db4729beb6980f4858378e6c88f4a10834ca4d7804f322dceae6f7f0f"} Oct 31 07:29:48.502847 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:48.502812 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rhg7w" event={"ID":"236238a7-255d-40f7-a99b-d65903b3bcb7","Type":"ContainerStarted","Data":"1040e401a45072871ed858c2c92c752e820ba1e6887613456bd5083ef8a83877"} Oct 31 07:29:48.505463 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:48.505419 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-pl5cl" event={"ID":"d25f2854-fd38-44e7-ad7e-da744d57e671","Type":"ContainerStarted","Data":"e4c449b22aa9ecd2802282cc3b7a684d789c2edc99dbea854c45798757d6012f"} Oct 31 07:29:48.512991 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:48.512965 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ktzg4" event={"ID":"ed522b38-611a-452a-b201-3ca416428d42","Type":"ContainerStarted","Data":"7d73c3717618a3632d06b92aba13225d3694b19f95fb058748881de13dbcfd6e"} Oct 31 07:29:48.515377 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:48.515341 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" event={"ID":"046b1de8-d709-4729-9aec-391a2b3294d5","Type":"ContainerStarted","Data":"b8f9976ff45a9a3b9246775990072df0586490e32925959c028535b18b777bad"} Oct 31 07:29:48.521523 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:48.521494 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-scrms" event={"ID":"53c36e19-9ef6-47a2-9bc2-387115539850","Type":"ContainerStarted","Data":"49788a1d09e2bb192e4b46ecea1b481cb3df28ad0cbb7feb731e2765844db5e2"} Oct 31 07:29:48.526891 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:48.526864 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-98whn" event={"ID":"e00eb9e3-0d3c-4541-b238-4130fe1de2f1","Type":"ContainerStarted","Data":"b4a16403faa883c4260e5c78aead776d888c71eb79b7d356dd8b397d49d3b4da"} Oct 31 07:29:48.529249 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:48.529225 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-w6vnh" event={"ID":"6ae10413-8f49-4dfe-8aad-00dcb9d390c8","Type":"ContainerStarted","Data":"9a2eb5642a9bcd6ecb79d58106e89b4b13ee221824aba714bee38dfaf9f57a8a"} Oct 31 07:29:48.537218 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:48.536416 2577 generic.go:356] "Generic (PLEG): container finished" podID="11f3ad8c54222373dd06005938b97a96" containerID="f1629f237dd7db0cfceeb4f1731b6ccb4f8c325b5187830e81989b689220cb36" exitCode=0 Oct 31 07:29:48.537218 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:48.537172 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-48.us-east-2.compute.internal" event={"ID":"11f3ad8c54222373dd06005938b97a96","Type":"ContainerDied","Data":"f1629f237dd7db0cfceeb4f1731b6ccb4f8c325b5187830e81989b689220cb36"} Oct 31 07:29:48.541266 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:48.541087 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lmsk9" event={"ID":"ac4bf75a-377c-49b2-818a-7d2436c21e9c","Type":"ContainerStarted","Data":"a32ed9582736eace8657e4c5e9e1521b6a93be7e5fb2ff212d0a71e3a22fc402"} Oct 31 07:29:48.552529 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:48.552432 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r78n8" event={"ID":"ea9beb38-1491-4cf0-83ca-fd79935fa6a9","Type":"ContainerStarted","Data":"a6992e8cfd7f4c537e4ca2c5d75ff78aaf7e9d831eb4dbaeec9dec17e03ec726"} Oct 31 07:29:48.557389 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:48.556931 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-48.us-east-2.compute.internal" podStartSLOduration=2.556918578 podStartE2EDuration="2.556918578s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 07:29:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:29:48.521016274 +0000 UTC m=+3.529592708" watchObservedRunningTime="2026-10-31 07:29:48.556918578 +0000 UTC m=+3.565495009" Oct 31 07:29:49.034450 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:49.034341 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28ec9621-7ec5-421c-a88b-51bb20d3a7bf-metrics-certs\") pod \"network-metrics-daemon-7thxr\" (UID: \"28ec9621-7ec5-421c-a88b-51bb20d3a7bf\") " pod="openshift-multus/network-metrics-daemon-7thxr" Oct 31 07:29:49.034720 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:49.034591 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 31 07:29:49.034720 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:49.034661 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28ec9621-7ec5-421c-a88b-51bb20d3a7bf-metrics-certs podName:28ec9621-7ec5-421c-a88b-51bb20d3a7bf nodeName:}" failed. No retries permitted until 2026-10-31 07:29:51.034638903 +0000 UTC m=+6.043215339 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/28ec9621-7ec5-421c-a88b-51bb20d3a7bf-metrics-certs") pod "network-metrics-daemon-7thxr" (UID: "28ec9621-7ec5-421c-a88b-51bb20d3a7bf") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 31 07:29:49.135369 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:49.135336 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ckzf\" (UniqueName: \"kubernetes.io/projected/d8b28b9f-37a2-4859-87fd-d27bb37d832b-kube-api-access-2ckzf\") pod \"network-check-target-ncnd7\" (UID: \"d8b28b9f-37a2-4859-87fd-d27bb37d832b\") " pod="openshift-network-diagnostics/network-check-target-ncnd7" Oct 31 07:29:49.135567 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:49.135431 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/297b35d7-aa65-49c7-8460-ac3f9860248c-original-pull-secret\") pod \"global-pull-secret-syncer-8hpcb\" (UID: \"297b35d7-aa65-49c7-8460-ac3f9860248c\") " pod="kube-system/global-pull-secret-syncer-8hpcb" Oct 31 07:29:49.135632 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:49.135578 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 31 07:29:49.135689 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:49.135638 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/297b35d7-aa65-49c7-8460-ac3f9860248c-original-pull-secret podName:297b35d7-aa65-49c7-8460-ac3f9860248c nodeName:}" failed. No retries permitted until 2026-10-31 07:29:51.135620099 +0000 UTC m=+6.144196510 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/297b35d7-aa65-49c7-8460-ac3f9860248c-original-pull-secret") pod "global-pull-secret-syncer-8hpcb" (UID: "297b35d7-aa65-49c7-8460-ac3f9860248c") : object "kube-system"/"original-pull-secret" not registered Oct 31 07:29:49.136102 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:49.136080 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 31 07:29:49.136196 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:49.136108 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 31 07:29:49.136196 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:49.136122 2577 projected.go:196] Error preparing data for projected volume kube-api-access-2ckzf for pod openshift-network-diagnostics/network-check-target-ncnd7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 31 07:29:49.136196 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:49.136168 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d8b28b9f-37a2-4859-87fd-d27bb37d832b-kube-api-access-2ckzf podName:d8b28b9f-37a2-4859-87fd-d27bb37d832b nodeName:}" failed. No retries permitted until 2026-10-31 07:29:51.136151857 +0000 UTC m=+6.144728267 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-2ckzf" (UniqueName: "kubernetes.io/projected/d8b28b9f-37a2-4859-87fd-d27bb37d832b-kube-api-access-2ckzf") pod "network-check-target-ncnd7" (UID: "d8b28b9f-37a2-4859-87fd-d27bb37d832b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 31 07:29:49.164667 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:49.164641 2577 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Oct 31 07:29:49.461874 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:49.461832 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-30 07:24:47 +0000 UTC" deadline="2028-04-15 08:04:58.15300733 +0000 UTC" Oct 31 07:29:49.461874 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:49.461871 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12768h35m8.691141048s" Oct 31 07:29:49.490861 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:49.490829 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7thxr" Oct 31 07:29:49.491040 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:49.490969 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7thxr" podUID="28ec9621-7ec5-421c-a88b-51bb20d3a7bf" Oct 31 07:29:49.491401 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:49.491367 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ncnd7" Oct 31 07:29:49.491499 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:49.491465 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ncnd7" podUID="d8b28b9f-37a2-4859-87fd-d27bb37d832b" Oct 31 07:29:49.491547 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:49.491529 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8hpcb" Oct 31 07:29:49.491817 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:49.491599 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8hpcb" podUID="297b35d7-aa65-49c7-8460-ac3f9860248c" Oct 31 07:29:49.574204 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:49.574101 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-48.us-east-2.compute.internal" event={"ID":"11f3ad8c54222373dd06005938b97a96","Type":"ContainerStarted","Data":"73675a98041d2bd87e79f8e8424f260d98e196b1a934e51b37625e26c91fdad8"} Oct 31 07:29:51.055146 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:51.055108 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28ec9621-7ec5-421c-a88b-51bb20d3a7bf-metrics-certs\") pod \"network-metrics-daemon-7thxr\" (UID: \"28ec9621-7ec5-421c-a88b-51bb20d3a7bf\") " pod="openshift-multus/network-metrics-daemon-7thxr" Oct 31 07:29:51.055654 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:51.055275 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 31 07:29:51.055654 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:51.055364 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28ec9621-7ec5-421c-a88b-51bb20d3a7bf-metrics-certs podName:28ec9621-7ec5-421c-a88b-51bb20d3a7bf nodeName:}" failed. No retries permitted until 2026-10-31 07:29:55.055323843 +0000 UTC m=+10.063900263 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/28ec9621-7ec5-421c-a88b-51bb20d3a7bf-metrics-certs") pod "network-metrics-daemon-7thxr" (UID: "28ec9621-7ec5-421c-a88b-51bb20d3a7bf") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 31 07:29:51.156125 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:51.156049 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/297b35d7-aa65-49c7-8460-ac3f9860248c-original-pull-secret\") pod \"global-pull-secret-syncer-8hpcb\" (UID: \"297b35d7-aa65-49c7-8460-ac3f9860248c\") " pod="kube-system/global-pull-secret-syncer-8hpcb" Oct 31 07:29:51.156282 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:51.156136 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ckzf\" (UniqueName: \"kubernetes.io/projected/d8b28b9f-37a2-4859-87fd-d27bb37d832b-kube-api-access-2ckzf\") pod \"network-check-target-ncnd7\" (UID: \"d8b28b9f-37a2-4859-87fd-d27bb37d832b\") " pod="openshift-network-diagnostics/network-check-target-ncnd7" Oct 31 07:29:51.156282 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:51.156268 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 31 07:29:51.156416 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:51.156289 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 31 07:29:51.156416 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:51.156301 2577 projected.go:196] Error preparing data for projected volume kube-api-access-2ckzf for pod openshift-network-diagnostics/network-check-target-ncnd7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 31 07:29:51.156416 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:51.156378 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d8b28b9f-37a2-4859-87fd-d27bb37d832b-kube-api-access-2ckzf podName:d8b28b9f-37a2-4859-87fd-d27bb37d832b nodeName:}" failed. No retries permitted until 2026-10-31 07:29:55.156340475 +0000 UTC m=+10.164916894 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-2ckzf" (UniqueName: "kubernetes.io/projected/d8b28b9f-37a2-4859-87fd-d27bb37d832b-kube-api-access-2ckzf") pod "network-check-target-ncnd7" (UID: "d8b28b9f-37a2-4859-87fd-d27bb37d832b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 31 07:29:51.156788 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:51.156769 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 31 07:29:51.156890 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:51.156823 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/297b35d7-aa65-49c7-8460-ac3f9860248c-original-pull-secret podName:297b35d7-aa65-49c7-8460-ac3f9860248c nodeName:}" failed. No retries permitted until 2026-10-31 07:29:55.156809241 +0000 UTC m=+10.165385662 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/297b35d7-aa65-49c7-8460-ac3f9860248c-original-pull-secret") pod "global-pull-secret-syncer-8hpcb" (UID: "297b35d7-aa65-49c7-8460-ac3f9860248c") : object "kube-system"/"original-pull-secret" not registered Oct 31 07:29:51.489172 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:51.489136 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7thxr" Oct 31 07:29:51.489338 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:51.489292 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7thxr" podUID="28ec9621-7ec5-421c-a88b-51bb20d3a7bf" Oct 31 07:29:51.489719 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:51.489696 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ncnd7" Oct 31 07:29:51.489821 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:51.489787 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ncnd7" podUID="d8b28b9f-37a2-4859-87fd-d27bb37d832b" Oct 31 07:29:51.489873 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:51.489849 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8hpcb" Oct 31 07:29:51.489941 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:51.489922 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8hpcb" podUID="297b35d7-aa65-49c7-8460-ac3f9860248c" Oct 31 07:29:51.584579 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:51.584533 2577 generic.go:356] "Generic (PLEG): container finished" podID="ed522b38-611a-452a-b201-3ca416428d42" containerID="81b3078f11cd9615e6f1318d1b692b1f4c5ae4de91dc4671bd11c0374748a756" exitCode=0 Oct 31 07:29:51.584748 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:51.584594 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ktzg4" event={"ID":"ed522b38-611a-452a-b201-3ca416428d42","Type":"ContainerDied","Data":"81b3078f11cd9615e6f1318d1b692b1f4c5ae4de91dc4671bd11c0374748a756"} Oct 31 07:29:51.611616 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:51.611565 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-48.us-east-2.compute.internal" podStartSLOduration=5.611546921 podStartE2EDuration="5.611546921s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 07:29:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:29:49.592517427 +0000 UTC m=+4.601093876" watchObservedRunningTime="2026-10-31 07:29:51.611546921 +0000 UTC m=+6.620123338" Oct 31 07:29:52.590811 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:52.590773 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-pl5cl" event={"ID":"d25f2854-fd38-44e7-ad7e-da744d57e671","Type":"ContainerStarted","Data":"ce0402f91d4015740a9eaa6059c0bdabea0a2bbd1c567b42ba89113ad8090208"} Oct 31 07:29:53.492533 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:53.492502 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ncnd7" Oct 31 07:29:53.492689 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:53.492667 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ncnd7" podUID="d8b28b9f-37a2-4859-87fd-d27bb37d832b" Oct 31 07:29:53.492780 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:53.492766 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7thxr" Oct 31 07:29:53.492918 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:53.492900 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7thxr" podUID="28ec9621-7ec5-421c-a88b-51bb20d3a7bf" Oct 31 07:29:53.493241 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:53.493226 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8hpcb" Oct 31 07:29:53.493322 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:53.493308 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8hpcb" podUID="297b35d7-aa65-49c7-8460-ac3f9860248c" Oct 31 07:29:53.512073 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:53.511923 2577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-pl5cl" Oct 31 07:29:53.512577 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:53.512417 2577 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-pl5cl" Oct 31 07:29:53.529984 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:53.529058 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-pl5cl" podStartSLOduration=3.5658926539999998 podStartE2EDuration="7.529043313s" totalImagesPullingTime="3.963150659s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:29:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:29:52.60883201 +0000 UTC m=+7.617408443" watchObservedRunningTime="2026-10-31 07:29:53.529043313 +0000 UTC m=+8.537619744" Oct 31 07:29:53.596370 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:53.596284 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rhg7w" event={"ID":"236238a7-255d-40f7-a99b-d65903b3bcb7","Type":"ContainerStarted","Data":"322e3bb89a52bf2952c594841426cee14f89a114970973425f1a9a5082a952bc"} Oct 31 07:29:54.601920 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:54.601602 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r78n8" event={"ID":"ea9beb38-1491-4cf0-83ca-fd79935fa6a9","Type":"ContainerStarted","Data":"7f13900adfd1dc4cb7d9d7e77bbec7235874f58430ff7a7da242a52974485ab8"} Oct 31 07:29:54.601920 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:54.601678 2577 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 31 07:29:55.091461 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:55.091427 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28ec9621-7ec5-421c-a88b-51bb20d3a7bf-metrics-certs\") pod \"network-metrics-daemon-7thxr\" (UID: \"28ec9621-7ec5-421c-a88b-51bb20d3a7bf\") " pod="openshift-multus/network-metrics-daemon-7thxr" Oct 31 07:29:55.091631 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:55.091586 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 31 07:29:55.091715 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:55.091650 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28ec9621-7ec5-421c-a88b-51bb20d3a7bf-metrics-certs podName:28ec9621-7ec5-421c-a88b-51bb20d3a7bf nodeName:}" failed. No retries permitted until 2026-10-31 07:30:03.091632923 +0000 UTC m=+18.100209338 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/28ec9621-7ec5-421c-a88b-51bb20d3a7bf-metrics-certs") pod "network-metrics-daemon-7thxr" (UID: "28ec9621-7ec5-421c-a88b-51bb20d3a7bf") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 31 07:29:55.182647 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:55.182606 2577 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Oct 31 07:29:55.191837 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:55.191808 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ckzf\" (UniqueName: \"kubernetes.io/projected/d8b28b9f-37a2-4859-87fd-d27bb37d832b-kube-api-access-2ckzf\") pod \"network-check-target-ncnd7\" (UID: \"d8b28b9f-37a2-4859-87fd-d27bb37d832b\") " pod="openshift-network-diagnostics/network-check-target-ncnd7" Oct 31 07:29:55.191944 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:55.191856 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/297b35d7-aa65-49c7-8460-ac3f9860248c-original-pull-secret\") pod \"global-pull-secret-syncer-8hpcb\" (UID: \"297b35d7-aa65-49c7-8460-ac3f9860248c\") " pod="kube-system/global-pull-secret-syncer-8hpcb" Oct 31 07:29:55.192021 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:55.191982 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 31 07:29:55.192076 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:55.192038 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/297b35d7-aa65-49c7-8460-ac3f9860248c-original-pull-secret podName:297b35d7-aa65-49c7-8460-ac3f9860248c nodeName:}" failed. No retries permitted until 2026-10-31 07:30:03.192018956 +0000 UTC m=+18.200595375 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/297b35d7-aa65-49c7-8460-ac3f9860248c-original-pull-secret") pod "global-pull-secret-syncer-8hpcb" (UID: "297b35d7-aa65-49c7-8460-ac3f9860248c") : object "kube-system"/"original-pull-secret" not registered Oct 31 07:29:55.192139 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:55.192129 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 31 07:29:55.192189 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:55.192143 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 31 07:29:55.192189 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:55.192159 2577 projected.go:196] Error preparing data for projected volume kube-api-access-2ckzf for pod openshift-network-diagnostics/network-check-target-ncnd7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 31 07:29:55.192289 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:55.192194 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d8b28b9f-37a2-4859-87fd-d27bb37d832b-kube-api-access-2ckzf podName:d8b28b9f-37a2-4859-87fd-d27bb37d832b nodeName:}" failed. No retries permitted until 2026-10-31 07:30:03.192181289 +0000 UTC m=+18.200757708 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-2ckzf" (UniqueName: "kubernetes.io/projected/d8b28b9f-37a2-4859-87fd-d27bb37d832b-kube-api-access-2ckzf") pod "network-check-target-ncnd7" (UID: "d8b28b9f-37a2-4859-87fd-d27bb37d832b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 31 07:29:55.478844 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:55.478085 2577 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-31T07:29:55.182626273Z","UUID":"80bbb010-c3b6-47d6-a2dd-ba7add09aa19","Handler":null,"Name":"","Endpoint":""} Oct 31 07:29:55.479789 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:55.479768 2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Oct 31 07:29:55.479908 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:55.479801 2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Oct 31 07:29:55.488819 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:55.488798 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ncnd7" Oct 31 07:29:55.488938 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:55.488909 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ncnd7" podUID="d8b28b9f-37a2-4859-87fd-d27bb37d832b" Oct 31 07:29:55.489286 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:55.489064 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8hpcb" Oct 31 07:29:55.489286 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:55.489180 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8hpcb" podUID="297b35d7-aa65-49c7-8460-ac3f9860248c" Oct 31 07:29:55.489286 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:55.489215 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7thxr" Oct 31 07:29:55.489491 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:55.489310 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7thxr" podUID="28ec9621-7ec5-421c-a88b-51bb20d3a7bf" Oct 31 07:29:55.606406 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:55.606169 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r78n8" event={"ID":"ea9beb38-1491-4cf0-83ca-fd79935fa6a9","Type":"ContainerStarted","Data":"75495009e86844ef0618e5377ef251f8078236f5a0befda0822337439eadbbed"} Oct 31 07:29:56.611886 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:56.611644 2577 generic.go:356] "Generic (PLEG): container finished" podID="ed522b38-611a-452a-b201-3ca416428d42" containerID="d255068b661bc489dc9facfded815b3b2fdcbc435885f24ac5fa7c86bfe68dce" exitCode=0 Oct 31 07:29:56.612274 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:56.611773 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ktzg4" event={"ID":"ed522b38-611a-452a-b201-3ca416428d42","Type":"ContainerDied","Data":"d255068b661bc489dc9facfded815b3b2fdcbc435885f24ac5fa7c86bfe68dce"} Oct 31 07:29:56.614806 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:56.614776 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-scrms" event={"ID":"53c36e19-9ef6-47a2-9bc2-387115539850","Type":"ContainerStarted","Data":"2228871bd7d45178b96adbe27f293a7c4e18f8e41562d7a35aca81570f3074a5"} Oct 31 07:29:56.618900 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:56.618850 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-w6vnh" event={"ID":"6ae10413-8f49-4dfe-8aad-00dcb9d390c8","Type":"ContainerStarted","Data":"ec31b161a4c6f9bfc0e66423bcd5593d298bfc1f4b517f29e652e4e5c42bc782"} Oct 31 07:29:56.621205 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:56.621185 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r78n8" event={"ID":"ea9beb38-1491-4cf0-83ca-fd79935fa6a9","Type":"ContainerStarted","Data":"6b3b05d2cf110dec572658ffdd0639e71089efac4f2cd47d040c0423f7151c2f"} Oct 31 07:29:56.637724 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:56.637679 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-rhg7w" podStartSLOduration=5.6971825240000005 podStartE2EDuration="10.637662521s" totalImagesPullingTime="4.940479997s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 07:29:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:29:53.613372846 +0000 UTC m=+8.621949272" watchObservedRunningTime="2026-10-31 07:29:56.637662521 +0000 UTC m=+11.646238950" Oct 31 07:29:56.652085 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:56.651996 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-scrms" podStartSLOduration=3.756916842 podStartE2EDuration="11.651956667s" totalImagesPullingTime="7.895039825s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 07:29:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:29:56.651920538 +0000 UTC m=+11.660496968" watchObservedRunningTime="2026-10-31 07:29:56.651956667 +0000 UTC m=+11.660533095" Oct 31 07:29:56.672739 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:56.672700 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r78n8" podStartSLOduration=3.456404982 podStartE2EDuration="11.672689343s" totalImagesPullingTime="8.216284361s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 07:29:45 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:29:56.672183366 +0000 UTC m=+11.680759795" watchObservedRunningTime="2026-10-31 07:29:56.672689343 +0000 UTC m=+11.681265770" Oct 31 07:29:56.691990 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:56.691950 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-w6vnh" podStartSLOduration=3.893345344 podStartE2EDuration="11.691935135s" totalImagesPullingTime="7.798589791s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 07:29:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:29:56.69179932 +0000 UTC m=+11.700375752" watchObservedRunningTime="2026-10-31 07:29:56.691935135 +0000 UTC m=+11.700511564" Oct 31 07:29:57.486430 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:57.486368 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ncnd7" Oct 31 07:29:57.486430 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:57.486382 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7thxr" Oct 31 07:29:57.486606 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:57.486467 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ncnd7" podUID="d8b28b9f-37a2-4859-87fd-d27bb37d832b" Oct 31 07:29:57.486810 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:57.486778 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7thxr" podUID="28ec9621-7ec5-421c-a88b-51bb20d3a7bf" Oct 31 07:29:57.486890 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:57.486864 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8hpcb" Oct 31 07:29:57.486960 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:57.486941 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8hpcb" podUID="297b35d7-aa65-49c7-8460-ac3f9860248c" Oct 31 07:29:57.626102 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:57.626023 2577 generic.go:356] "Generic (PLEG): container finished" podID="ed522b38-611a-452a-b201-3ca416428d42" containerID="786e5a0a40cb4f1ba47f7ed2e5bb820c8bfdac16ac667cd7f63d40dfb804f043" exitCode=0 Oct 31 07:29:57.626102 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:57.626064 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ktzg4" event={"ID":"ed522b38-611a-452a-b201-3ca416428d42","Type":"ContainerDied","Data":"786e5a0a40cb4f1ba47f7ed2e5bb820c8bfdac16ac667cd7f63d40dfb804f043"} Oct 31 07:29:57.628365 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:57.628324 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lmsk9" event={"ID":"ac4bf75a-377c-49b2-818a-7d2436c21e9c","Type":"ContainerStarted","Data":"154605c061e72051f9d7047ecd02dbac2651f7704a5a7f7893d6fcca2c6b1f31"} Oct 31 07:29:57.672902 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:57.672860 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-lmsk9" podStartSLOduration=3.494138141 podStartE2EDuration="11.672847561s" totalImagesPullingTime="8.17870942s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 07:29:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:29:57.67256229 +0000 UTC m=+12.681138719" watchObservedRunningTime="2026-10-31 07:29:57.672847561 +0000 UTC m=+12.681423989" Oct 31 07:29:58.633630 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:58.633594 2577 generic.go:356] "Generic (PLEG): container finished" podID="ed522b38-611a-452a-b201-3ca416428d42" containerID="050e01d8523ffaf42d5f2babfd83abee1dc9af7a5dabef0ace9cbce09bbf1f82" exitCode=0 Oct 31 07:29:58.634277 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:58.633664 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ktzg4" event={"ID":"ed522b38-611a-452a-b201-3ca416428d42","Type":"ContainerDied","Data":"050e01d8523ffaf42d5f2babfd83abee1dc9af7a5dabef0ace9cbce09bbf1f82"} Oct 31 07:29:58.635583 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:58.635561 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" event={"ID":"046b1de8-d709-4729-9aec-391a2b3294d5","Type":"ContainerStarted","Data":"70e48ec4540917b12913a2108cdbc088601d854ebf370bc64e93a8b2fecfee5a"} Oct 31 07:29:58.680625 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:58.680568 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" podStartSLOduration=2.430390033 podStartE2EDuration="12.680554822s" totalImagesPullingTime="10.250164789s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 07:29:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:29:58.680094143 +0000 UTC m=+13.688670573" watchObservedRunningTime="2026-10-31 07:29:58.680554822 +0000 UTC m=+13.689131250" Oct 31 07:29:59.487694 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:59.487477 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8hpcb" Oct 31 07:29:59.487694 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:59.487516 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ncnd7" Oct 31 07:29:59.487694 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:29:59.487487 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7thxr" Oct 31 07:29:59.487694 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:59.487634 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8hpcb" podUID="297b35d7-aa65-49c7-8460-ac3f9860248c" Oct 31 07:29:59.488002 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:59.487709 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7thxr" podUID="28ec9621-7ec5-421c-a88b-51bb20d3a7bf" Oct 31 07:29:59.488002 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:29:59.487766 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ncnd7" podUID="d8b28b9f-37a2-4859-87fd-d27bb37d832b" Oct 31 07:30:01.487227 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:01.486994 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7thxr" Oct 31 07:30:01.487817 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:01.487017 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ncnd7" Oct 31 07:30:01.487817 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:01.487052 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8hpcb" Oct 31 07:30:01.487817 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:01.487311 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7thxr" podUID="28ec9621-7ec5-421c-a88b-51bb20d3a7bf" Oct 31 07:30:01.487817 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:01.487416 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8hpcb" podUID="297b35d7-aa65-49c7-8460-ac3f9860248c" Oct 31 07:30:01.487817 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:01.487505 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ncnd7" podUID="d8b28b9f-37a2-4859-87fd-d27bb37d832b" Oct 31 07:30:01.644880 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:01.644840 2577 generic.go:356] "Generic (PLEG): container finished" podID="ed522b38-611a-452a-b201-3ca416428d42" containerID="0857706c3a24d3436bc55abbea6cc7e47995262128b08cec8803612caac184d0" exitCode=0 Oct 31 07:30:01.645047 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:01.644896 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ktzg4" event={"ID":"ed522b38-611a-452a-b201-3ca416428d42","Type":"ContainerDied","Data":"0857706c3a24d3436bc55abbea6cc7e47995262128b08cec8803612caac184d0"} Oct 31 07:30:02.650018 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:02.649991 2577 generic.go:356] "Generic (PLEG): container finished" podID="ed522b38-611a-452a-b201-3ca416428d42" containerID="ec55eab7889a3e6a63fcd29bdbc907e6fae3e3d265b5fb4b4f6ca295ebd8023e" exitCode=0 Oct 31 07:30:02.650462 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:02.650074 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ktzg4" event={"ID":"ed522b38-611a-452a-b201-3ca416428d42","Type":"ContainerDied","Data":"ec55eab7889a3e6a63fcd29bdbc907e6fae3e3d265b5fb4b4f6ca295ebd8023e"} Oct 31 07:30:02.659296 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:02.659272 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-98whn" event={"ID":"e00eb9e3-0d3c-4541-b238-4130fe1de2f1","Type":"ContainerStarted","Data":"453c85d26162bf5d2f3321352635a46dff4705eac4085bd1b08ae8998b59e47f"} Oct 31 07:30:02.659419 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:02.659376 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-98whn" event={"ID":"e00eb9e3-0d3c-4541-b238-4130fe1de2f1","Type":"ContainerStarted","Data":"7625fc16075fc6178679ca5cc2afe663b27989beb7879679c7dec15be9cc2fce"} Oct 31 07:30:02.659419 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:02.659406 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-98whn" event={"ID":"e00eb9e3-0d3c-4541-b238-4130fe1de2f1","Type":"ContainerStarted","Data":"cc8eaada4b6c7b659974a2650994131fb0fe898199c31a278049540280e2e3d3"} Oct 31 07:30:02.659527 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:02.659432 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-98whn" event={"ID":"e00eb9e3-0d3c-4541-b238-4130fe1de2f1","Type":"ContainerStarted","Data":"7001a577de47f8f8767c6890325c00e2c035ad6f58e085decad7c6b27bf31609"} Oct 31 07:30:02.659527 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:02.659458 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-98whn" event={"ID":"e00eb9e3-0d3c-4541-b238-4130fe1de2f1","Type":"ContainerStarted","Data":"c84ae0b933da7ef74d7e3d711b39f0046f20ad5b5a16144b1a3e8e87de77ede5"} Oct 31 07:30:03.148245 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:03.148044 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28ec9621-7ec5-421c-a88b-51bb20d3a7bf-metrics-certs\") pod \"network-metrics-daemon-7thxr\" (UID: \"28ec9621-7ec5-421c-a88b-51bb20d3a7bf\") " pod="openshift-multus/network-metrics-daemon-7thxr" Oct 31 07:30:03.148245 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:03.148191 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 31 07:30:03.148454 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:03.148308 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28ec9621-7ec5-421c-a88b-51bb20d3a7bf-metrics-certs podName:28ec9621-7ec5-421c-a88b-51bb20d3a7bf nodeName:}" failed. No retries permitted until 2026-10-31 07:30:19.148292732 +0000 UTC m=+34.156869151 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/28ec9621-7ec5-421c-a88b-51bb20d3a7bf-metrics-certs") pod "network-metrics-daemon-7thxr" (UID: "28ec9621-7ec5-421c-a88b-51bb20d3a7bf") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 31 07:30:03.249227 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:03.249196 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ckzf\" (UniqueName: \"kubernetes.io/projected/d8b28b9f-37a2-4859-87fd-d27bb37d832b-kube-api-access-2ckzf\") pod \"network-check-target-ncnd7\" (UID: \"d8b28b9f-37a2-4859-87fd-d27bb37d832b\") " pod="openshift-network-diagnostics/network-check-target-ncnd7" Oct 31 07:30:03.249400 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:03.249235 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/297b35d7-aa65-49c7-8460-ac3f9860248c-original-pull-secret\") pod \"global-pull-secret-syncer-8hpcb\" (UID: \"297b35d7-aa65-49c7-8460-ac3f9860248c\") " pod="kube-system/global-pull-secret-syncer-8hpcb" Oct 31 07:30:03.249400 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:03.249325 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 31 07:30:03.249400 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:03.249340 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 31 07:30:03.249400 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:03.249373 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 31 07:30:03.249400 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:03.249382 2577 projected.go:196] Error preparing data for projected volume kube-api-access-2ckzf for pod openshift-network-diagnostics/network-check-target-ncnd7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 31 07:30:03.249400 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:03.249386 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/297b35d7-aa65-49c7-8460-ac3f9860248c-original-pull-secret podName:297b35d7-aa65-49c7-8460-ac3f9860248c nodeName:}" failed. No retries permitted until 2026-10-31 07:30:19.249373948 +0000 UTC m=+34.257950354 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/297b35d7-aa65-49c7-8460-ac3f9860248c-original-pull-secret") pod "global-pull-secret-syncer-8hpcb" (UID: "297b35d7-aa65-49c7-8460-ac3f9860248c") : object "kube-system"/"original-pull-secret" not registered Oct 31 07:30:03.249573 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:03.249415 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d8b28b9f-37a2-4859-87fd-d27bb37d832b-kube-api-access-2ckzf podName:d8b28b9f-37a2-4859-87fd-d27bb37d832b nodeName:}" failed. No retries permitted until 2026-10-31 07:30:19.249404869 +0000 UTC m=+34.257981285 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-2ckzf" (UniqueName: "kubernetes.io/projected/d8b28b9f-37a2-4859-87fd-d27bb37d832b-kube-api-access-2ckzf") pod "network-check-target-ncnd7" (UID: "d8b28b9f-37a2-4859-87fd-d27bb37d832b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 31 07:30:03.486330 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:03.486300 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ncnd7" Oct 31 07:30:03.486475 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:03.486343 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8hpcb" Oct 31 07:30:03.486475 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:03.486424 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7thxr" Oct 31 07:30:03.486475 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:03.486440 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ncnd7" podUID="d8b28b9f-37a2-4859-87fd-d27bb37d832b" Oct 31 07:30:03.486634 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:03.486507 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7thxr" podUID="28ec9621-7ec5-421c-a88b-51bb20d3a7bf" Oct 31 07:30:03.486634 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:03.486589 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8hpcb" podUID="297b35d7-aa65-49c7-8460-ac3f9860248c" Oct 31 07:30:03.664700 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:03.664666 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ktzg4" event={"ID":"ed522b38-611a-452a-b201-3ca416428d42","Type":"ContainerStarted","Data":"2b0468be17a61aedbf41268742298c2ae22e5bf293f0773e1b46d864211830b1"} Oct 31 07:30:03.667273 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:03.667253 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-98whn" event={"ID":"e00eb9e3-0d3c-4541-b238-4130fe1de2f1","Type":"ContainerStarted","Data":"6018b3e4b91b73e0bbff79fdf0e9b9e1616e829bd897a767a1255db4794f3e58"} Oct 31 07:30:03.691221 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:03.691180 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-ktzg4" podStartSLOduration=7.796091435 podStartE2EDuration="18.691169108s" totalImagesPullingTime="10.895077673s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 07:29:45 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:30:03.691115393 +0000 UTC m=+18.699691824" watchObservedRunningTime="2026-10-31 07:30:03.691169108 +0000 UTC m=+18.699745535" Oct 31 07:30:05.487243 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:05.487209 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ncnd7" Oct 31 07:30:05.487746 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:05.487293 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ncnd7" podUID="d8b28b9f-37a2-4859-87fd-d27bb37d832b" Oct 31 07:30:05.487746 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:05.487401 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7thxr" Oct 31 07:30:05.487874 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:05.487831 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7thxr" podUID="28ec9621-7ec5-421c-a88b-51bb20d3a7bf" Oct 31 07:30:05.487936 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:05.487884 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8hpcb" Oct 31 07:30:05.488018 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:05.488000 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8hpcb" podUID="297b35d7-aa65-49c7-8460-ac3f9860248c" Oct 31 07:30:05.673372 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:05.673326 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-98whn" event={"ID":"e00eb9e3-0d3c-4541-b238-4130fe1de2f1","Type":"ContainerStarted","Data":"5f0d0c7dbba5b8ce1c0e3820af418d72acdc9b1e8dcd41e53bae071534fd2dcc"} Oct 31 07:30:06.048834 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:06.048804 2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-pl5cl" Oct 31 07:30:06.048990 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:06.048899 2577 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 31 07:30:06.049418 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:06.049398 2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-pl5cl" Oct 31 07:30:07.486589 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:07.486545 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7thxr" Oct 31 07:30:07.486589 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:07.486545 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8hpcb" Oct 31 07:30:07.487295 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:07.486673 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7thxr" podUID="28ec9621-7ec5-421c-a88b-51bb20d3a7bf" Oct 31 07:30:07.487295 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:07.486717 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ncnd7" Oct 31 07:30:07.487295 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:07.486785 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8hpcb" podUID="297b35d7-aa65-49c7-8460-ac3f9860248c" Oct 31 07:30:07.487295 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:07.486869 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ncnd7" podUID="d8b28b9f-37a2-4859-87fd-d27bb37d832b" Oct 31 07:30:07.681111 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:07.681074 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-98whn" event={"ID":"e00eb9e3-0d3c-4541-b238-4130fe1de2f1","Type":"ContainerStarted","Data":"d6d4293455c8bbd0f22807eb01e84fb79b527f91bd14f290a7b853c5b004ec63"} Oct 31 07:30:07.681343 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:07.681320 2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:07.681455 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:07.681415 2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:07.697293 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:07.697272 2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:07.697418 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:07.697384 2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:07.714836 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:07.714795 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-98whn" podStartSLOduration=8.112788936 podStartE2EDuration="22.714783795s" totalImagesPullingTime="14.601994859s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 07:29:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:30:07.71306189 +0000 UTC m=+22.721638335" watchObservedRunningTime="2026-10-31 07:30:07.714783795 +0000 UTC m=+22.723360597" Oct 31 07:30:08.556249 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:08.556057 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8hpcb"] Oct 31 07:30:08.556790 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:08.556385 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8hpcb" Oct 31 07:30:08.556790 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:08.556494 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8hpcb" podUID="297b35d7-aa65-49c7-8460-ac3f9860248c" Oct 31 07:30:08.557469 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:08.557446 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-ncnd7"] Oct 31 07:30:08.557575 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:08.557535 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ncnd7" Oct 31 07:30:08.557627 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:08.557610 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ncnd7" podUID="d8b28b9f-37a2-4859-87fd-d27bb37d832b" Oct 31 07:30:08.557953 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:08.557936 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7thxr"] Oct 31 07:30:08.558028 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:08.558019 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7thxr" Oct 31 07:30:08.558102 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:08.558088 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7thxr" podUID="28ec9621-7ec5-421c-a88b-51bb20d3a7bf" Oct 31 07:30:08.683476 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:08.683447 2577 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 31 07:30:09.685597 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:09.685569 2577 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 31 07:30:10.164592 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:10.164558 2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:10.486472 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:10.486448 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ncnd7" Oct 31 07:30:10.486622 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:10.486535 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ncnd7" podUID="d8b28b9f-37a2-4859-87fd-d27bb37d832b" Oct 31 07:30:10.486622 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:10.486614 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8hpcb" Oct 31 07:30:10.486725 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:10.486709 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8hpcb" podUID="297b35d7-aa65-49c7-8460-ac3f9860248c" Oct 31 07:30:10.486756 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:10.486748 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7thxr" Oct 31 07:30:10.486817 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:10.486806 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7thxr" podUID="28ec9621-7ec5-421c-a88b-51bb20d3a7bf" Oct 31 07:30:12.486925 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:12.486885 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ncnd7" Oct 31 07:30:12.487410 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:12.486967 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7thxr" Oct 31 07:30:12.487410 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:12.487044 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8hpcb" Oct 31 07:30:12.487410 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:12.487063 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ncnd7" podUID="d8b28b9f-37a2-4859-87fd-d27bb37d832b" Oct 31 07:30:12.487410 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:12.487110 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8hpcb" podUID="297b35d7-aa65-49c7-8460-ac3f9860248c" Oct 31 07:30:12.487410 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:12.487170 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7thxr" podUID="28ec9621-7ec5-421c-a88b-51bb20d3a7bf" Oct 31 07:30:14.486240 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:14.486209 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ncnd7" Oct 31 07:30:14.486761 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:14.486209 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8hpcb" Oct 31 07:30:14.486761 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:14.486333 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ncnd7" podUID="d8b28b9f-37a2-4859-87fd-d27bb37d832b" Oct 31 07:30:14.486761 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:14.486225 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7thxr" Oct 31 07:30:14.486761 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:14.486404 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8hpcb" podUID="297b35d7-aa65-49c7-8460-ac3f9860248c" Oct 31 07:30:14.486761 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:14.486494 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7thxr" podUID="28ec9621-7ec5-421c-a88b-51bb20d3a7bf" Oct 31 07:30:15.231669 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.231647 2577 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-48.us-east-2.compute.internal" event="NodeReady" Oct 31 07:30:15.231806 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.231754 2577 kubelet_node_status.go:477] "Fast updating node status as it just became ready" Oct 31 07:30:15.266384 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.266321 2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-vhcq9"] Oct 31 07:30:15.269173 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.269157 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-vhcq9" Oct 31 07:30:15.270259 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.270228 2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-cpkfr"] Oct 31 07:30:15.274985 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.274967 2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-xkmbn"] Oct 31 07:30:15.276374 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.275424 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-cpkfr" Oct 31 07:30:15.279588 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.279537 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\"" Oct 31 07:30:15.279588 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.279549 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\"" Oct 31 07:30:15.285343 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.281621 2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-54854c7897-pc7km"] Oct 31 07:30:15.285343 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.281774 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-xkmbn" Oct 31 07:30:15.285343 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.284519 2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-69fbc98d44-fn7fh"] Oct 31 07:30:15.285343 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.284611 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-54854c7897-pc7km" Oct 31 07:30:15.287305 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.287285 2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-m6sns"] Oct 31 07:30:15.287690 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.287665 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" Oct 31 07:30:15.289966 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.289947 2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-46qvv"] Oct 31 07:30:15.290090 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.290066 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-m6sns" Oct 31 07:30:15.292771 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.292754 2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-62tn2"] Oct 31 07:30:15.292932 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.292915 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-46qvv" Oct 31 07:30:15.295253 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.295237 2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-86n8d"] Oct 31 07:30:15.295382 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.295367 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-62tn2" Oct 31 07:30:15.299581 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.299475 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Oct 31 07:30:15.299925 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.299907 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Oct 31 07:30:15.300232 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.300180 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Oct 31 07:30:15.300448 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.300432 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-25rsd\"" Oct 31 07:30:15.300862 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.300844 2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-s75pm"] Oct 31 07:30:15.301637 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.301622 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-86n8d" Oct 31 07:30:15.304135 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.304090 2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-7c4986d449-hz58r"] Oct 31 07:30:15.304453 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.304436 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Oct 31 07:30:15.304611 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.304595 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-s75pm" Oct 31 07:30:15.307576 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.307560 2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-282kr"] Oct 31 07:30:15.307687 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.307667 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-hz58r" Oct 31 07:30:15.311091 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.311073 2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-jztcz"] Oct 31 07:30:15.313180 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.312783 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-282kr" Oct 31 07:30:15.316065 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.316046 2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-5549647ff-cxcdt"] Oct 31 07:30:15.316730 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.316711 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-jztcz" Oct 31 07:30:15.319738 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.319717 2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-4xtdb"] Oct 31 07:30:15.321418 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.321401 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5549647ff-cxcdt" Oct 31 07:30:15.324853 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.323947 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-vhcq9"] Oct 31 07:30:15.324853 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.324041 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-xkmbn"] Oct 31 07:30:15.324853 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.324066 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-m6sns"] Oct 31 07:30:15.324853 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.324096 2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-z5sfp"] Oct 31 07:30:15.324853 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.323998 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-4xtdb" Oct 31 07:30:15.328154 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.328136 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-cpkfr"] Oct 31 07:30:15.328249 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.328173 2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-22cmd"] Oct 31 07:30:15.328311 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.328245 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-z5sfp" Oct 31 07:30:15.331197 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.331178 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-hz58r"] Oct 31 07:30:15.331304 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.331218 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-s75pm"] Oct 31 07:30:15.331304 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.331242 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-54854c7897-pc7km"] Oct 31 07:30:15.331304 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.331263 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5549647ff-cxcdt"] Oct 31 07:30:15.331304 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.331287 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-jztcz"] Oct 31 07:30:15.331304 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.331305 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-69fbc98d44-fn7fh"] Oct 31 07:30:15.331573 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.331328 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-86n8d"] Oct 31 07:30:15.331573 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.331389 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-4xtdb"] Oct 31 07:30:15.331573 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.331411 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-62tn2"] Oct 31 07:30:15.331573 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.331429 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-46qvv"] Oct 31 07:30:15.331573 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.331450 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-z5sfp"] Oct 31 07:30:15.331573 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.331476 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-282kr"] Oct 31 07:30:15.331573 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.331493 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-22cmd"] Oct 31 07:30:15.331573 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.331571 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-22cmd" Oct 31 07:30:15.340540 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.340519 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4zgz\" (UniqueName: \"kubernetes.io/projected/3cbf4695-0536-45d5-bc48-be56861b87b8-kube-api-access-v4zgz\") pod \"kas-connection-checker-685895c7c9-vhcq9\" (UID: \"3cbf4695-0536-45d5-bc48-be56861b87b8\") " pod="kube-system/kas-connection-checker-685895c7c9-vhcq9" Oct 31 07:30:15.378760 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.378737 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Oct 31 07:30:15.378859 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.378796 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Oct 31 07:30:15.378859 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.378832 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Oct 31 07:30:15.378859 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.378837 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Oct 31 07:30:15.379012 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.378905 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Oct 31 07:30:15.379012 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.378921 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Oct 31 07:30:15.379012 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.378982 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Oct 31 07:30:15.379012 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.378986 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Oct 31 07:30:15.379186 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.379029 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Oct 31 07:30:15.379186 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.379039 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Oct 31 07:30:15.379186 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.379061 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Oct 31 07:30:15.379186 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.379088 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Oct 31 07:30:15.379186 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.379090 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Oct 31 07:30:15.379186 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.379111 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Oct 31 07:30:15.379186 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.379068 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Oct 31 07:30:15.379186 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.379102 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Oct 31 07:30:15.379186 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.379113 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Oct 31 07:30:15.379186 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.379144 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Oct 31 07:30:15.379186 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.379153 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Oct 31 07:30:15.379186 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.379164 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Oct 31 07:30:15.379572 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.379554 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Oct 31 07:30:15.380589 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.380573 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Oct 31 07:30:15.380779 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.380762 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\"" Oct 31 07:30:15.380937 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.380918 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Oct 31 07:30:15.381006 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.380938 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Oct 31 07:30:15.381006 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.380929 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Oct 31 07:30:15.381006 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.380965 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Oct 31 07:30:15.382073 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.382057 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Oct 31 07:30:15.387295 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.387280 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Oct 31 07:30:15.397628 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.397607 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Oct 31 07:30:15.397924 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.397903 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Oct 31 07:30:15.398146 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.398126 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Oct 31 07:30:15.398509 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.398185 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Oct 31 07:30:15.398898 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.398877 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Oct 31 07:30:15.399373 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.399334 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Oct 31 07:30:15.399700 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.399687 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Oct 31 07:30:15.399874 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.399863 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-lhm78\"" Oct 31 07:30:15.399930 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.399912 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Oct 31 07:30:15.400191 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.400179 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-2schs\"" Oct 31 07:30:15.400225 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.400213 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5t598\"" Oct 31 07:30:15.400304 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.400295 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Oct 31 07:30:15.400439 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.400428 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Oct 31 07:30:15.400494 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.400479 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Oct 31 07:30:15.400542 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.400533 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Oct 31 07:30:15.400642 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.400633 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Oct 31 07:30:15.400875 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.400861 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-tghnm\"" Oct 31 07:30:15.400980 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.400970 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Oct 31 07:30:15.441017 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.440998 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lx427\" (UniqueName: \"kubernetes.io/projected/54296c9d-1c43-44cd-bfb6-d81fe892484c-kube-api-access-lx427\") pod \"volume-data-source-validator-58c9f5f46d-m6sns\" (UID: \"54296c9d-1c43-44cd-bfb6-d81fe892484c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-m6sns" Oct 31 07:30:15.441126 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.441023 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-image-registry-private-configuration\") pod \"image-registry-54854c7897-pc7km\" (UID: \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\") " pod="openshift-image-registry/image-registry-54854c7897-pc7km" Oct 31 07:30:15.441126 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.441041 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-ca-trust-extracted\") pod \"image-registry-54854c7897-pc7km\" (UID: \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\") " pod="openshift-image-registry/image-registry-54854c7897-pc7km" Oct 31 07:30:15.441126 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.441058 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p76jj\" (UniqueName: \"kubernetes.io/projected/b6997c76-e103-4c3f-b39d-a6a17fb98270-kube-api-access-p76jj\") pod \"service-ca-operator-544d749c8c-62tn2\" (UID: \"b6997c76-e103-4c3f-b39d-a6a17fb98270\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-62tn2" Oct 31 07:30:15.441126 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.441118 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-registry-certificates\") pod \"image-registry-54854c7897-pc7km\" (UID: \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\") " pod="openshift-image-registry/image-registry-54854c7897-pc7km" Oct 31 07:30:15.441254 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.441144 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-trusted-ca\") pod \"image-registry-54854c7897-pc7km\" (UID: \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\") " pod="openshift-image-registry/image-registry-54854c7897-pc7km" Oct 31 07:30:15.441254 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.441168 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4dd8884-1d6e-4abe-ad6c-3c947623bb60-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-hz58r\" (UID: \"b4dd8884-1d6e-4abe-ad6c-3c947623bb60\") " pod="openshift-insights/insights-operator-7c4986d449-hz58r" Oct 31 07:30:15.441254 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.441196 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hldt6\" (UniqueName: \"kubernetes.io/projected/4104ba06-fba8-4c3c-b876-cca5087a7d0a-kube-api-access-hldt6\") pod \"image-registry-69fbc98d44-fn7fh\" (UID: \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\") " pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" Oct 31 07:30:15.441337 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.441322 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/58bc3ea1-f58c-4891-a049-1289a99be055-metrics-tls\") pod \"dns-default-z5sfp\" (UID: \"58bc3ea1-f58c-4891-a049-1289a99be055\") " pod="openshift-dns/dns-default-z5sfp" Oct 31 07:30:15.441389 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.441365 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/28be803a-a932-4009-ad61-b356366821ef-default-certificate\") pod \"router-default-5549647ff-cxcdt\" (UID: \"28be803a-a932-4009-ad61-b356366821ef\") " pod="openshift-ingress/router-default-5549647ff-cxcdt" Oct 31 07:30:15.441425 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.441388 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/5e743234-2b5f-4c91-9b06-189f546cbce4-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-282kr\" (UID: \"5e743234-2b5f-4c91-9b06-189f546cbce4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-282kr" Oct 31 07:30:15.441425 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.441403 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28be803a-a932-4009-ad61-b356366821ef-service-ca-bundle\") pod \"router-default-5549647ff-cxcdt\" (UID: \"28be803a-a932-4009-ad61-b356366821ef\") " pod="openshift-ingress/router-default-5549647ff-cxcdt" Oct 31 07:30:15.441425 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.441418 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82jqk\" (UniqueName: \"kubernetes.io/projected/43c59587-4ff8-4546-9c0a-6161ece201d0-kube-api-access-82jqk\") pod \"network-check-source-6f78d59f99-4xtdb\" (UID: \"43c59587-4ff8-4546-9c0a-6161ece201d0\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-4xtdb" Oct 31 07:30:15.441511 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.441439 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-installation-pull-secrets\") pod \"image-registry-54854c7897-pc7km\" (UID: \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\") " pod="openshift-image-registry/image-registry-54854c7897-pc7km" Oct 31 07:30:15.441511 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.441464 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58bc3ea1-f58c-4891-a049-1289a99be055-tmp-dir\") pod \"dns-default-z5sfp\" (UID: \"58bc3ea1-f58c-4891-a049-1289a99be055\") " pod="openshift-dns/dns-default-z5sfp" Oct 31 07:30:15.441511 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.441486 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4104ba06-fba8-4c3c-b876-cca5087a7d0a-trusted-ca\") pod \"image-registry-69fbc98d44-fn7fh\" (UID: \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\") " pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" Oct 31 07:30:15.441592 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.441524 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9k9q8\" (UniqueName: \"kubernetes.io/projected/b4dd8884-1d6e-4abe-ad6c-3c947623bb60-kube-api-access-9k9q8\") pod \"insights-operator-7c4986d449-hz58r\" (UID: \"b4dd8884-1d6e-4abe-ad6c-3c947623bb60\") " pod="openshift-insights/insights-operator-7c4986d449-hz58r" Oct 31 07:30:15.441592 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.441546 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b6997c76-e103-4c3f-b39d-a6a17fb98270-config\") pod \"service-ca-operator-544d749c8c-62tn2\" (UID: \"b6997c76-e103-4c3f-b39d-a6a17fb98270\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-62tn2" Oct 31 07:30:15.441592 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.441579 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4q9t4\" (UniqueName: \"kubernetes.io/projected/58bc3ea1-f58c-4891-a049-1289a99be055-kube-api-access-4q9t4\") pod \"dns-default-z5sfp\" (UID: \"58bc3ea1-f58c-4891-a049-1289a99be055\") " pod="openshift-dns/dns-default-z5sfp" Oct 31 07:30:15.441677 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.441601 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/58bc3ea1-f58c-4891-a049-1289a99be055-config-volume\") pod \"dns-default-z5sfp\" (UID: \"58bc3ea1-f58c-4891-a049-1289a99be055\") " pod="openshift-dns/dns-default-z5sfp" Oct 31 07:30:15.441677 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.441619 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/00a0d978-8020-46df-932d-b0dd22f946f9-cert\") pod \"ingress-canary-22cmd\" (UID: \"00a0d978-8020-46df-932d-b0dd22f946f9\") " pod="openshift-ingress-canary/ingress-canary-22cmd" Oct 31 07:30:15.441677 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.441633 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-86n8d\" (UID: \"d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-86n8d" Oct 31 07:30:15.441677 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.441650 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cf9zh\" (UniqueName: \"kubernetes.io/projected/28be803a-a932-4009-ad61-b356366821ef-kube-api-access-cf9zh\") pod \"router-default-5549647ff-cxcdt\" (UID: \"28be803a-a932-4009-ad61-b356366821ef\") " pod="openshift-ingress/router-default-5549647ff-cxcdt" Oct 31 07:30:15.441790 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.441727 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f3bf19fd-bc58-4dff-a1a8-4f61381e1fde-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-46qvv\" (UID: \"f3bf19fd-bc58-4dff-a1a8-4f61381e1fde\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-46qvv" Oct 31 07:30:15.441790 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.441759 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cb5dx\" (UniqueName: \"kubernetes.io/projected/f3bf19fd-bc58-4dff-a1a8-4f61381e1fde-kube-api-access-cb5dx\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-46qvv\" (UID: \"f3bf19fd-bc58-4dff-a1a8-4f61381e1fde\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-46qvv" Oct 31 07:30:15.441849 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.441794 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-registry-tls\") pod \"image-registry-54854c7897-pc7km\" (UID: \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\") " pod="openshift-image-registry/image-registry-54854c7897-pc7km" Oct 31 07:30:15.441849 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.441826 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/28be803a-a932-4009-ad61-b356366821ef-stats-auth\") pod \"router-default-5549647ff-cxcdt\" (UID: \"28be803a-a932-4009-ad61-b356366821ef\") " pod="openshift-ingress/router-default-5549647ff-cxcdt" Oct 31 07:30:15.441908 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.441869 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xksf5\" (UniqueName: \"kubernetes.io/projected/e84bef13-a137-49f7-931d-9f94952dacfc-kube-api-access-xksf5\") pod \"kas-connection-checker-685895c7c9-cpkfr\" (UID: \"e84bef13-a137-49f7-931d-9f94952dacfc\") " pod="kube-system/kas-connection-checker-685895c7c9-cpkfr" Oct 31 07:30:15.441908 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.441893 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f3bf19fd-bc58-4dff-a1a8-4f61381e1fde-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-46qvv\" (UID: \"f3bf19fd-bc58-4dff-a1a8-4f61381e1fde\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-46qvv" Oct 31 07:30:15.441963 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.441908 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/396e1c5e-3700-4fe6-b367-54ad2a244225-serving-cert\") pod \"console-operator-7cd56b8dbc-jztcz\" (UID: \"396e1c5e-3700-4fe6-b367-54ad2a244225\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jztcz" Oct 31 07:30:15.441963 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.441925 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/396e1c5e-3700-4fe6-b367-54ad2a244225-trusted-ca\") pod \"console-operator-7cd56b8dbc-jztcz\" (UID: \"396e1c5e-3700-4fe6-b367-54ad2a244225\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jztcz" Oct 31 07:30:15.442021 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.441962 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/563d3b8b-0040-439e-b0b2-e858eed5abf6-nginx-conf\") pod \"networking-console-plugin-86c478bb88-s75pm\" (UID: \"563d3b8b-0040-439e-b0b2-e858eed5abf6\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-s75pm" Oct 31 07:30:15.442051 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.442037 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/b4dd8884-1d6e-4abe-ad6c-3c947623bb60-snapshots\") pod \"insights-operator-7c4986d449-hz58r\" (UID: \"b4dd8884-1d6e-4abe-ad6c-3c947623bb60\") " pod="openshift-insights/insights-operator-7c4986d449-hz58r" Oct 31 07:30:15.442081 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.442062 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsvzt\" (UniqueName: \"kubernetes.io/projected/6fae9634-7abe-40c6-a892-9eb58166e9ad-kube-api-access-wsvzt\") pod \"kas-connection-checker-685895c7c9-xkmbn\" (UID: \"6fae9634-7abe-40c6-a892-9eb58166e9ad\") " pod="kube-system/kas-connection-checker-685895c7c9-xkmbn" Oct 31 07:30:15.442112 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.442095 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5e743234-2b5f-4c91-9b06-189f546cbce4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-282kr\" (UID: \"5e743234-2b5f-4c91-9b06-189f546cbce4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-282kr" Oct 31 07:30:15.442145 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.442115 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4104ba06-fba8-4c3c-b876-cca5087a7d0a-image-registry-private-configuration\") pod \"image-registry-69fbc98d44-fn7fh\" (UID: \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\") " pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" Oct 31 07:30:15.442175 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.442142 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-bound-sa-token\") pod \"image-registry-54854c7897-pc7km\" (UID: \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\") " pod="openshift-image-registry/image-registry-54854c7897-pc7km" Oct 31 07:30:15.442203 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.442179 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5s89\" (UniqueName: \"kubernetes.io/projected/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-kube-api-access-z5s89\") pod \"image-registry-54854c7897-pc7km\" (UID: \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\") " pod="openshift-image-registry/image-registry-54854c7897-pc7km" Oct 31 07:30:15.442233 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.442209 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b4dd8884-1d6e-4abe-ad6c-3c947623bb60-serving-cert\") pod \"insights-operator-7c4986d449-hz58r\" (UID: \"b4dd8884-1d6e-4abe-ad6c-3c947623bb60\") " pod="openshift-insights/insights-operator-7c4986d449-hz58r" Oct 31 07:30:15.442233 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.442230 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4104ba06-fba8-4c3c-b876-cca5087a7d0a-bound-sa-token\") pod \"image-registry-69fbc98d44-fn7fh\" (UID: \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\") " pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" Oct 31 07:30:15.442292 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.442277 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/396e1c5e-3700-4fe6-b367-54ad2a244225-config\") pod \"console-operator-7cd56b8dbc-jztcz\" (UID: \"396e1c5e-3700-4fe6-b367-54ad2a244225\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jztcz" Oct 31 07:30:15.442322 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.442299 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g76kv\" (UniqueName: \"kubernetes.io/projected/396e1c5e-3700-4fe6-b367-54ad2a244225-kube-api-access-g76kv\") pod \"console-operator-7cd56b8dbc-jztcz\" (UID: \"396e1c5e-3700-4fe6-b367-54ad2a244225\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jztcz" Oct 31 07:30:15.442322 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.442314 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlm6c\" (UniqueName: \"kubernetes.io/projected/d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e-kube-api-access-nlm6c\") pod \"cluster-samples-operator-6ddb6c8b94-86n8d\" (UID: \"d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-86n8d" Oct 31 07:30:15.442393 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.442343 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4zgz\" (UniqueName: \"kubernetes.io/projected/3cbf4695-0536-45d5-bc48-be56861b87b8-kube-api-access-v4zgz\") pod \"kas-connection-checker-685895c7c9-vhcq9\" (UID: \"3cbf4695-0536-45d5-bc48-be56861b87b8\") " pod="kube-system/kas-connection-checker-685895c7c9-vhcq9" Oct 31 07:30:15.442424 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.442414 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/563d3b8b-0040-439e-b0b2-e858eed5abf6-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-s75pm\" (UID: \"563d3b8b-0040-439e-b0b2-e858eed5abf6\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-s75pm" Oct 31 07:30:15.442456 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.442438 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4104ba06-fba8-4c3c-b876-cca5087a7d0a-ca-trust-extracted\") pod \"image-registry-69fbc98d44-fn7fh\" (UID: \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\") " pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" Oct 31 07:30:15.442484 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.442466 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b4dd8884-1d6e-4abe-ad6c-3c947623bb60-tmp\") pod \"insights-operator-7c4986d449-hz58r\" (UID: \"b4dd8884-1d6e-4abe-ad6c-3c947623bb60\") " pod="openshift-insights/insights-operator-7c4986d449-hz58r" Oct 31 07:30:15.442546 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.442534 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28be803a-a932-4009-ad61-b356366821ef-metrics-certs\") pod \"router-default-5549647ff-cxcdt\" (UID: \"28be803a-a932-4009-ad61-b356366821ef\") " pod="openshift-ingress/router-default-5549647ff-cxcdt" Oct 31 07:30:15.442578 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.442553 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4104ba06-fba8-4c3c-b876-cca5087a7d0a-registry-tls\") pod \"image-registry-69fbc98d44-fn7fh\" (UID: \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\") " pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" Oct 31 07:30:15.442578 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.442568 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4104ba06-fba8-4c3c-b876-cca5087a7d0a-registry-certificates\") pod \"image-registry-69fbc98d44-fn7fh\" (UID: \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\") " pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" Oct 31 07:30:15.442637 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.442586 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b6997c76-e103-4c3f-b39d-a6a17fb98270-serving-cert\") pod \"service-ca-operator-544d749c8c-62tn2\" (UID: \"b6997c76-e103-4c3f-b39d-a6a17fb98270\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-62tn2" Oct 31 07:30:15.442668 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.442636 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djqrt\" (UniqueName: \"kubernetes.io/projected/5e743234-2b5f-4c91-9b06-189f546cbce4-kube-api-access-djqrt\") pod \"cluster-monitoring-operator-884576c57-282kr\" (UID: \"5e743234-2b5f-4c91-9b06-189f546cbce4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-282kr" Oct 31 07:30:15.442668 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.442652 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jhpf\" (UniqueName: \"kubernetes.io/projected/00a0d978-8020-46df-932d-b0dd22f946f9-kube-api-access-2jhpf\") pod \"ingress-canary-22cmd\" (UID: \"00a0d978-8020-46df-932d-b0dd22f946f9\") " pod="openshift-ingress-canary/ingress-canary-22cmd" Oct 31 07:30:15.442724 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.442668 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4dd8884-1d6e-4abe-ad6c-3c947623bb60-service-ca-bundle\") pod \"insights-operator-7c4986d449-hz58r\" (UID: \"b4dd8884-1d6e-4abe-ad6c-3c947623bb60\") " pod="openshift-insights/insights-operator-7c4986d449-hz58r" Oct 31 07:30:15.442724 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.442697 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4104ba06-fba8-4c3c-b876-cca5087a7d0a-installation-pull-secrets\") pod \"image-registry-69fbc98d44-fn7fh\" (UID: \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\") " pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" Oct 31 07:30:15.442821 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.442722 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-86n8d\" (UID: \"d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-86n8d" Oct 31 07:30:15.460201 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.460172 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4zgz\" (UniqueName: \"kubernetes.io/projected/3cbf4695-0536-45d5-bc48-be56861b87b8-kube-api-access-v4zgz\") pod \"kas-connection-checker-685895c7c9-vhcq9\" (UID: \"3cbf4695-0536-45d5-bc48-be56861b87b8\") " pod="kube-system/kas-connection-checker-685895c7c9-vhcq9" Oct 31 07:30:15.543290 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.543222 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9k9q8\" (UniqueName: \"kubernetes.io/projected/b4dd8884-1d6e-4abe-ad6c-3c947623bb60-kube-api-access-9k9q8\") pod \"insights-operator-7c4986d449-hz58r\" (UID: \"b4dd8884-1d6e-4abe-ad6c-3c947623bb60\") " pod="openshift-insights/insights-operator-7c4986d449-hz58r" Oct 31 07:30:15.543290 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.543254 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b6997c76-e103-4c3f-b39d-a6a17fb98270-config\") pod \"service-ca-operator-544d749c8c-62tn2\" (UID: \"b6997c76-e103-4c3f-b39d-a6a17fb98270\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-62tn2" Oct 31 07:30:15.543290 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.543277 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4q9t4\" (UniqueName: \"kubernetes.io/projected/58bc3ea1-f58c-4891-a049-1289a99be055-kube-api-access-4q9t4\") pod \"dns-default-z5sfp\" (UID: \"58bc3ea1-f58c-4891-a049-1289a99be055\") " pod="openshift-dns/dns-default-z5sfp" Oct 31 07:30:15.543832 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.543302 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/58bc3ea1-f58c-4891-a049-1289a99be055-config-volume\") pod \"dns-default-z5sfp\" (UID: \"58bc3ea1-f58c-4891-a049-1289a99be055\") " pod="openshift-dns/dns-default-z5sfp" Oct 31 07:30:15.543832 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.543324 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/00a0d978-8020-46df-932d-b0dd22f946f9-cert\") pod \"ingress-canary-22cmd\" (UID: \"00a0d978-8020-46df-932d-b0dd22f946f9\") " pod="openshift-ingress-canary/ingress-canary-22cmd" Oct 31 07:30:15.543832 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.543464 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-86n8d\" (UID: \"d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-86n8d" Oct 31 07:30:15.543832 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.543502 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cf9zh\" (UniqueName: \"kubernetes.io/projected/28be803a-a932-4009-ad61-b356366821ef-kube-api-access-cf9zh\") pod \"router-default-5549647ff-cxcdt\" (UID: \"28be803a-a932-4009-ad61-b356366821ef\") " pod="openshift-ingress/router-default-5549647ff-cxcdt" Oct 31 07:30:15.543832 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.543552 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f3bf19fd-bc58-4dff-a1a8-4f61381e1fde-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-46qvv\" (UID: \"f3bf19fd-bc58-4dff-a1a8-4f61381e1fde\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-46qvv" Oct 31 07:30:15.543832 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:15.543571 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Oct 31 07:30:15.543832 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.543577 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cb5dx\" (UniqueName: \"kubernetes.io/projected/f3bf19fd-bc58-4dff-a1a8-4f61381e1fde-kube-api-access-cb5dx\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-46qvv\" (UID: \"f3bf19fd-bc58-4dff-a1a8-4f61381e1fde\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-46qvv" Oct 31 07:30:15.543832 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.543604 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-registry-tls\") pod \"image-registry-54854c7897-pc7km\" (UID: \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\") " pod="openshift-image-registry/image-registry-54854c7897-pc7km" Oct 31 07:30:15.543832 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:15.543628 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e-samples-operator-tls podName:d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e nodeName:}" failed. No retries permitted until 2026-10-31 07:30:16.043610509 +0000 UTC m=+31.052186935 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-86n8d" (UID: "d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e") : secret "samples-operator-tls" not found Oct 31 07:30:15.543832 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.543658 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/28be803a-a932-4009-ad61-b356366821ef-stats-auth\") pod \"router-default-5549647ff-cxcdt\" (UID: \"28be803a-a932-4009-ad61-b356366821ef\") " pod="openshift-ingress/router-default-5549647ff-cxcdt" Oct 31 07:30:15.543832 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:15.543685 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 31 07:30:15.543832 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.543691 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xksf5\" (UniqueName: \"kubernetes.io/projected/e84bef13-a137-49f7-931d-9f94952dacfc-kube-api-access-xksf5\") pod \"kas-connection-checker-685895c7c9-cpkfr\" (UID: \"e84bef13-a137-49f7-931d-9f94952dacfc\") " pod="kube-system/kas-connection-checker-685895c7c9-cpkfr" Oct 31 07:30:15.543832 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.543720 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f3bf19fd-bc58-4dff-a1a8-4f61381e1fde-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-46qvv\" (UID: \"f3bf19fd-bc58-4dff-a1a8-4f61381e1fde\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-46qvv" Oct 31 07:30:15.543832 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.543746 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/396e1c5e-3700-4fe6-b367-54ad2a244225-serving-cert\") pod \"console-operator-7cd56b8dbc-jztcz\" (UID: \"396e1c5e-3700-4fe6-b367-54ad2a244225\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jztcz" Oct 31 07:30:15.543832 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.543775 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/396e1c5e-3700-4fe6-b367-54ad2a244225-trusted-ca\") pod \"console-operator-7cd56b8dbc-jztcz\" (UID: \"396e1c5e-3700-4fe6-b367-54ad2a244225\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jztcz" Oct 31 07:30:15.543832 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.543800 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/563d3b8b-0040-439e-b0b2-e858eed5abf6-nginx-conf\") pod \"networking-console-plugin-86c478bb88-s75pm\" (UID: \"563d3b8b-0040-439e-b0b2-e858eed5abf6\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-s75pm" Oct 31 07:30:15.544599 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.543861 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/b4dd8884-1d6e-4abe-ad6c-3c947623bb60-snapshots\") pod \"insights-operator-7c4986d449-hz58r\" (UID: \"b4dd8884-1d6e-4abe-ad6c-3c947623bb60\") " pod="openshift-insights/insights-operator-7c4986d449-hz58r" Oct 31 07:30:15.544599 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.543892 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsvzt\" (UniqueName: \"kubernetes.io/projected/6fae9634-7abe-40c6-a892-9eb58166e9ad-kube-api-access-wsvzt\") pod \"kas-connection-checker-685895c7c9-xkmbn\" (UID: \"6fae9634-7abe-40c6-a892-9eb58166e9ad\") " pod="kube-system/kas-connection-checker-685895c7c9-xkmbn" Oct 31 07:30:15.544599 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.543932 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5e743234-2b5f-4c91-9b06-189f546cbce4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-282kr\" (UID: \"5e743234-2b5f-4c91-9b06-189f546cbce4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-282kr" Oct 31 07:30:15.544599 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.543949 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/58bc3ea1-f58c-4891-a049-1289a99be055-config-volume\") pod \"dns-default-z5sfp\" (UID: \"58bc3ea1-f58c-4891-a049-1289a99be055\") " pod="openshift-dns/dns-default-z5sfp" Oct 31 07:30:15.544599 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.543958 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4104ba06-fba8-4c3c-b876-cca5087a7d0a-image-registry-private-configuration\") pod \"image-registry-69fbc98d44-fn7fh\" (UID: \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\") " pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" Oct 31 07:30:15.544599 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.543997 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-bound-sa-token\") pod \"image-registry-54854c7897-pc7km\" (UID: \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\") " pod="openshift-image-registry/image-registry-54854c7897-pc7km" Oct 31 07:30:15.544599 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.544014 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b6997c76-e103-4c3f-b39d-a6a17fb98270-config\") pod \"service-ca-operator-544d749c8c-62tn2\" (UID: \"b6997c76-e103-4c3f-b39d-a6a17fb98270\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-62tn2" Oct 31 07:30:15.544599 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.544022 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5s89\" (UniqueName: \"kubernetes.io/projected/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-kube-api-access-z5s89\") pod \"image-registry-54854c7897-pc7km\" (UID: \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\") " pod="openshift-image-registry/image-registry-54854c7897-pc7km" Oct 31 07:30:15.544599 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.544046 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b4dd8884-1d6e-4abe-ad6c-3c947623bb60-serving-cert\") pod \"insights-operator-7c4986d449-hz58r\" (UID: \"b4dd8884-1d6e-4abe-ad6c-3c947623bb60\") " pod="openshift-insights/insights-operator-7c4986d449-hz58r" Oct 31 07:30:15.544599 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.544069 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4104ba06-fba8-4c3c-b876-cca5087a7d0a-bound-sa-token\") pod \"image-registry-69fbc98d44-fn7fh\" (UID: \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\") " pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" Oct 31 07:30:15.544599 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.544095 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/396e1c5e-3700-4fe6-b367-54ad2a244225-config\") pod \"console-operator-7cd56b8dbc-jztcz\" (UID: \"396e1c5e-3700-4fe6-b367-54ad2a244225\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jztcz" Oct 31 07:30:15.544599 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.544118 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g76kv\" (UniqueName: \"kubernetes.io/projected/396e1c5e-3700-4fe6-b367-54ad2a244225-kube-api-access-g76kv\") pod \"console-operator-7cd56b8dbc-jztcz\" (UID: \"396e1c5e-3700-4fe6-b367-54ad2a244225\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jztcz" Oct 31 07:30:15.544599 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.544142 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlm6c\" (UniqueName: \"kubernetes.io/projected/d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e-kube-api-access-nlm6c\") pod \"cluster-samples-operator-6ddb6c8b94-86n8d\" (UID: \"d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-86n8d" Oct 31 07:30:15.544599 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.544171 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/563d3b8b-0040-439e-b0b2-e858eed5abf6-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-s75pm\" (UID: \"563d3b8b-0040-439e-b0b2-e858eed5abf6\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-s75pm" Oct 31 07:30:15.544599 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.544195 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4104ba06-fba8-4c3c-b876-cca5087a7d0a-ca-trust-extracted\") pod \"image-registry-69fbc98d44-fn7fh\" (UID: \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\") " pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" Oct 31 07:30:15.544599 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.544219 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b4dd8884-1d6e-4abe-ad6c-3c947623bb60-tmp\") pod \"insights-operator-7c4986d449-hz58r\" (UID: \"b4dd8884-1d6e-4abe-ad6c-3c947623bb60\") " pod="openshift-insights/insights-operator-7c4986d449-hz58r" Oct 31 07:30:15.545323 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.544245 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28be803a-a932-4009-ad61-b356366821ef-metrics-certs\") pod \"router-default-5549647ff-cxcdt\" (UID: \"28be803a-a932-4009-ad61-b356366821ef\") " pod="openshift-ingress/router-default-5549647ff-cxcdt" Oct 31 07:30:15.545323 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.544267 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4104ba06-fba8-4c3c-b876-cca5087a7d0a-registry-tls\") pod \"image-registry-69fbc98d44-fn7fh\" (UID: \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\") " pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" Oct 31 07:30:15.545323 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.544292 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4104ba06-fba8-4c3c-b876-cca5087a7d0a-registry-certificates\") pod \"image-registry-69fbc98d44-fn7fh\" (UID: \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\") " pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" Oct 31 07:30:15.545323 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.544313 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b6997c76-e103-4c3f-b39d-a6a17fb98270-serving-cert\") pod \"service-ca-operator-544d749c8c-62tn2\" (UID: \"b6997c76-e103-4c3f-b39d-a6a17fb98270\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-62tn2" Oct 31 07:30:15.545323 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.544337 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djqrt\" (UniqueName: \"kubernetes.io/projected/5e743234-2b5f-4c91-9b06-189f546cbce4-kube-api-access-djqrt\") pod \"cluster-monitoring-operator-884576c57-282kr\" (UID: \"5e743234-2b5f-4c91-9b06-189f546cbce4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-282kr" Oct 31 07:30:15.545323 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.544386 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jhpf\" (UniqueName: \"kubernetes.io/projected/00a0d978-8020-46df-932d-b0dd22f946f9-kube-api-access-2jhpf\") pod \"ingress-canary-22cmd\" (UID: \"00a0d978-8020-46df-932d-b0dd22f946f9\") " pod="openshift-ingress-canary/ingress-canary-22cmd" Oct 31 07:30:15.545323 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.544392 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f3bf19fd-bc58-4dff-a1a8-4f61381e1fde-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-46qvv\" (UID: \"f3bf19fd-bc58-4dff-a1a8-4f61381e1fde\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-46qvv" Oct 31 07:30:15.545323 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.544408 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4dd8884-1d6e-4abe-ad6c-3c947623bb60-service-ca-bundle\") pod \"insights-operator-7c4986d449-hz58r\" (UID: \"b4dd8884-1d6e-4abe-ad6c-3c947623bb60\") " pod="openshift-insights/insights-operator-7c4986d449-hz58r" Oct 31 07:30:15.545323 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.544436 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4104ba06-fba8-4c3c-b876-cca5087a7d0a-installation-pull-secrets\") pod \"image-registry-69fbc98d44-fn7fh\" (UID: \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\") " pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" Oct 31 07:30:15.545323 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.544459 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-86n8d\" (UID: \"d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-86n8d" Oct 31 07:30:15.545323 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.544487 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lx427\" (UniqueName: \"kubernetes.io/projected/54296c9d-1c43-44cd-bfb6-d81fe892484c-kube-api-access-lx427\") pod \"volume-data-source-validator-58c9f5f46d-m6sns\" (UID: \"54296c9d-1c43-44cd-bfb6-d81fe892484c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-m6sns" Oct 31 07:30:15.545323 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.544509 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-image-registry-private-configuration\") pod \"image-registry-54854c7897-pc7km\" (UID: \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\") " pod="openshift-image-registry/image-registry-54854c7897-pc7km" Oct 31 07:30:15.545323 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.544530 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-ca-trust-extracted\") pod \"image-registry-54854c7897-pc7km\" (UID: \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\") " pod="openshift-image-registry/image-registry-54854c7897-pc7km" Oct 31 07:30:15.545323 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.544552 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p76jj\" (UniqueName: \"kubernetes.io/projected/b6997c76-e103-4c3f-b39d-a6a17fb98270-kube-api-access-p76jj\") pod \"service-ca-operator-544d749c8c-62tn2\" (UID: \"b6997c76-e103-4c3f-b39d-a6a17fb98270\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-62tn2" Oct 31 07:30:15.545323 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.544575 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-registry-certificates\") pod \"image-registry-54854c7897-pc7km\" (UID: \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\") " pod="openshift-image-registry/image-registry-54854c7897-pc7km" Oct 31 07:30:15.546028 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.544596 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-trusted-ca\") pod \"image-registry-54854c7897-pc7km\" (UID: \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\") " pod="openshift-image-registry/image-registry-54854c7897-pc7km" Oct 31 07:30:15.546028 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.544627 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4dd8884-1d6e-4abe-ad6c-3c947623bb60-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-hz58r\" (UID: \"b4dd8884-1d6e-4abe-ad6c-3c947623bb60\") " pod="openshift-insights/insights-operator-7c4986d449-hz58r" Oct 31 07:30:15.546028 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.544655 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hldt6\" (UniqueName: \"kubernetes.io/projected/4104ba06-fba8-4c3c-b876-cca5087a7d0a-kube-api-access-hldt6\") pod \"image-registry-69fbc98d44-fn7fh\" (UID: \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\") " pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" Oct 31 07:30:15.546028 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.544703 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/58bc3ea1-f58c-4891-a049-1289a99be055-metrics-tls\") pod \"dns-default-z5sfp\" (UID: \"58bc3ea1-f58c-4891-a049-1289a99be055\") " pod="openshift-dns/dns-default-z5sfp" Oct 31 07:30:15.546028 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.544733 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/28be803a-a932-4009-ad61-b356366821ef-default-certificate\") pod \"router-default-5549647ff-cxcdt\" (UID: \"28be803a-a932-4009-ad61-b356366821ef\") " pod="openshift-ingress/router-default-5549647ff-cxcdt" Oct 31 07:30:15.546028 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.544760 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/5e743234-2b5f-4c91-9b06-189f546cbce4-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-282kr\" (UID: \"5e743234-2b5f-4c91-9b06-189f546cbce4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-282kr" Oct 31 07:30:15.546028 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.544784 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28be803a-a932-4009-ad61-b356366821ef-service-ca-bundle\") pod \"router-default-5549647ff-cxcdt\" (UID: \"28be803a-a932-4009-ad61-b356366821ef\") " pod="openshift-ingress/router-default-5549647ff-cxcdt" Oct 31 07:30:15.546028 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.544808 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82jqk\" (UniqueName: \"kubernetes.io/projected/43c59587-4ff8-4546-9c0a-6161ece201d0-kube-api-access-82jqk\") pod \"network-check-source-6f78d59f99-4xtdb\" (UID: \"43c59587-4ff8-4546-9c0a-6161ece201d0\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-4xtdb" Oct 31 07:30:15.546028 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.544829 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-installation-pull-secrets\") pod \"image-registry-54854c7897-pc7km\" (UID: \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\") " pod="openshift-image-registry/image-registry-54854c7897-pc7km" Oct 31 07:30:15.546028 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.544840 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/563d3b8b-0040-439e-b0b2-e858eed5abf6-nginx-conf\") pod \"networking-console-plugin-86c478bb88-s75pm\" (UID: \"563d3b8b-0040-439e-b0b2-e858eed5abf6\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-s75pm" Oct 31 07:30:15.546028 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.544853 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58bc3ea1-f58c-4891-a049-1289a99be055-tmp-dir\") pod \"dns-default-z5sfp\" (UID: \"58bc3ea1-f58c-4891-a049-1289a99be055\") " pod="openshift-dns/dns-default-z5sfp" Oct 31 07:30:15.546028 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.544879 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4104ba06-fba8-4c3c-b876-cca5087a7d0a-trusted-ca\") pod \"image-registry-69fbc98d44-fn7fh\" (UID: \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\") " pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" Oct 31 07:30:15.546028 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.544886 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/b4dd8884-1d6e-4abe-ad6c-3c947623bb60-snapshots\") pod \"insights-operator-7c4986d449-hz58r\" (UID: \"b4dd8884-1d6e-4abe-ad6c-3c947623bb60\") " pod="openshift-insights/insights-operator-7c4986d449-hz58r" Oct 31 07:30:15.546028 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.544972 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/396e1c5e-3700-4fe6-b367-54ad2a244225-trusted-ca\") pod \"console-operator-7cd56b8dbc-jztcz\" (UID: \"396e1c5e-3700-4fe6-b367-54ad2a244225\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jztcz" Oct 31 07:30:15.546028 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:15.543699 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-54854c7897-pc7km: secret "image-registry-tls" not found Oct 31 07:30:15.546028 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:15.545044 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-registry-tls podName:c734ebf9-eba8-4e14-b20d-2d78dd65b10d nodeName:}" failed. No retries permitted until 2026-10-31 07:30:16.045027616 +0000 UTC m=+31.053604038 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-registry-tls") pod "image-registry-54854c7897-pc7km" (UID: "c734ebf9-eba8-4e14-b20d-2d78dd65b10d") : secret "image-registry-tls" not found Oct 31 07:30:15.546028 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:15.544862 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 31 07:30:15.547075 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:15.545335 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 31 07:30:15.547075 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:15.545411 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58bc3ea1-f58c-4891-a049-1289a99be055-metrics-tls podName:58bc3ea1-f58c-4891-a049-1289a99be055 nodeName:}" failed. No retries permitted until 2026-10-31 07:30:16.045394172 +0000 UTC m=+31.053970584 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/58bc3ea1-f58c-4891-a049-1289a99be055-metrics-tls") pod "dns-default-z5sfp" (UID: "58bc3ea1-f58c-4891-a049-1289a99be055") : secret "dns-default-metrics-tls" not found Oct 31 07:30:15.547075 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.545412 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4dd8884-1d6e-4abe-ad6c-3c947623bb60-service-ca-bundle\") pod \"insights-operator-7c4986d449-hz58r\" (UID: \"b4dd8884-1d6e-4abe-ad6c-3c947623bb60\") " pod="openshift-insights/insights-operator-7c4986d449-hz58r" Oct 31 07:30:15.547075 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.545466 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-ca-trust-extracted\") pod \"image-registry-54854c7897-pc7km\" (UID: \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\") " pod="openshift-image-registry/image-registry-54854c7897-pc7km" Oct 31 07:30:15.547075 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.545780 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4104ba06-fba8-4c3c-b876-cca5087a7d0a-ca-trust-extracted\") pod \"image-registry-69fbc98d44-fn7fh\" (UID: \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\") " pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" Oct 31 07:30:15.547075 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.545998 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58bc3ea1-f58c-4891-a049-1289a99be055-tmp-dir\") pod \"dns-default-z5sfp\" (UID: \"58bc3ea1-f58c-4891-a049-1289a99be055\") " pod="openshift-dns/dns-default-z5sfp" Oct 31 07:30:15.547075 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.546036 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-86n8d\" (UID: \"d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-86n8d" Oct 31 07:30:15.547075 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:15.546106 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e743234-2b5f-4c91-9b06-189f546cbce4-cluster-monitoring-operator-tls podName:5e743234-2b5f-4c91-9b06-189f546cbce4 nodeName:}" failed. No retries permitted until 2026-10-31 07:30:16.046088575 +0000 UTC m=+31.054664981 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5e743234-2b5f-4c91-9b06-189f546cbce4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-282kr" (UID: "5e743234-2b5f-4c91-9b06-189f546cbce4") : secret "cluster-monitoring-operator-tls" not found Oct 31 07:30:15.547075 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:15.546178 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/28be803a-a932-4009-ad61-b356366821ef-service-ca-bundle podName:28be803a-a932-4009-ad61-b356366821ef nodeName:}" failed. No retries permitted until 2026-10-31 07:30:16.04616666 +0000 UTC m=+31.054743089 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/28be803a-a932-4009-ad61-b356366821ef-service-ca-bundle") pod "router-default-5549647ff-cxcdt" (UID: "28be803a-a932-4009-ad61-b356366821ef") : configmap references non-existent config key: service-ca.crt Oct 31 07:30:15.547075 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.546579 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/396e1c5e-3700-4fe6-b367-54ad2a244225-config\") pod \"console-operator-7cd56b8dbc-jztcz\" (UID: \"396e1c5e-3700-4fe6-b367-54ad2a244225\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jztcz" Oct 31 07:30:15.547075 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.546851 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4104ba06-fba8-4c3c-b876-cca5087a7d0a-trusted-ca\") pod \"image-registry-69fbc98d44-fn7fh\" (UID: \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\") " pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" Oct 31 07:30:15.548438 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.548409 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/28be803a-a932-4009-ad61-b356366821ef-stats-auth\") pod \"router-default-5549647ff-cxcdt\" (UID: \"28be803a-a932-4009-ad61-b356366821ef\") " pod="openshift-ingress/router-default-5549647ff-cxcdt" Oct 31 07:30:15.548604 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.548586 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f3bf19fd-bc58-4dff-a1a8-4f61381e1fde-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-46qvv\" (UID: \"f3bf19fd-bc58-4dff-a1a8-4f61381e1fde\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-46qvv" Oct 31 07:30:15.548859 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.548842 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4104ba06-fba8-4c3c-b876-cca5087a7d0a-image-registry-private-configuration\") pod \"image-registry-69fbc98d44-fn7fh\" (UID: \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\") " pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" Oct 31 07:30:15.549737 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.549271 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4dd8884-1d6e-4abe-ad6c-3c947623bb60-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-hz58r\" (UID: \"b4dd8884-1d6e-4abe-ad6c-3c947623bb60\") " pod="openshift-insights/insights-operator-7c4986d449-hz58r" Oct 31 07:30:15.549737 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.549378 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/5e743234-2b5f-4c91-9b06-189f546cbce4-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-282kr\" (UID: \"5e743234-2b5f-4c91-9b06-189f546cbce4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-282kr" Oct 31 07:30:15.549737 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:15.549451 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Oct 31 07:30:15.549737 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:15.549493 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28be803a-a932-4009-ad61-b356366821ef-metrics-certs podName:28be803a-a932-4009-ad61-b356366821ef nodeName:}" failed. No retries permitted until 2026-10-31 07:30:16.049478953 +0000 UTC m=+31.058055364 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/28be803a-a932-4009-ad61-b356366821ef-metrics-certs") pod "router-default-5549647ff-cxcdt" (UID: "28be803a-a932-4009-ad61-b356366821ef") : secret "router-metrics-certs-default" not found Oct 31 07:30:15.550529 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:15.545334 2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Oct 31 07:30:15.550529 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:15.549910 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/563d3b8b-0040-439e-b0b2-e858eed5abf6-networking-console-plugin-cert podName:563d3b8b-0040-439e-b0b2-e858eed5abf6 nodeName:}" failed. No retries permitted until 2026-10-31 07:30:16.049896813 +0000 UTC m=+31.058473220 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/563d3b8b-0040-439e-b0b2-e858eed5abf6-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-s75pm" (UID: "563d3b8b-0040-439e-b0b2-e858eed5abf6") : secret "networking-console-plugin-cert" not found Oct 31 07:30:15.550529 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.549944 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b6997c76-e103-4c3f-b39d-a6a17fb98270-serving-cert\") pod \"service-ca-operator-544d749c8c-62tn2\" (UID: \"b6997c76-e103-4c3f-b39d-a6a17fb98270\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-62tn2" Oct 31 07:30:15.550529 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.549982 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4104ba06-fba8-4c3c-b876-cca5087a7d0a-registry-certificates\") pod \"image-registry-69fbc98d44-fn7fh\" (UID: \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\") " pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" Oct 31 07:30:15.550529 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:15.550059 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 31 07:30:15.550529 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.550059 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b4dd8884-1d6e-4abe-ad6c-3c947623bb60-tmp\") pod \"insights-operator-7c4986d449-hz58r\" (UID: \"b4dd8884-1d6e-4abe-ad6c-3c947623bb60\") " pod="openshift-insights/insights-operator-7c4986d449-hz58r" Oct 31 07:30:15.550529 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:15.550069 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69fbc98d44-fn7fh: secret "image-registry-tls" not found Oct 31 07:30:15.550529 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:15.550141 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4104ba06-fba8-4c3c-b876-cca5087a7d0a-registry-tls podName:4104ba06-fba8-4c3c-b876-cca5087a7d0a nodeName:}" failed. No retries permitted until 2026-10-31 07:30:16.050125206 +0000 UTC m=+31.058701612 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4104ba06-fba8-4c3c-b876-cca5087a7d0a-registry-tls") pod "image-registry-69fbc98d44-fn7fh" (UID: "4104ba06-fba8-4c3c-b876-cca5087a7d0a") : secret "image-registry-tls" not found Oct 31 07:30:15.550529 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.550155 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4104ba06-fba8-4c3c-b876-cca5087a7d0a-installation-pull-secrets\") pod \"image-registry-69fbc98d44-fn7fh\" (UID: \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\") " pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" Oct 31 07:30:15.550529 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.550252 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b4dd8884-1d6e-4abe-ad6c-3c947623bb60-serving-cert\") pod \"insights-operator-7c4986d449-hz58r\" (UID: \"b4dd8884-1d6e-4abe-ad6c-3c947623bb60\") " pod="openshift-insights/insights-operator-7c4986d449-hz58r" Oct 31 07:30:15.550529 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.550269 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-trusted-ca\") pod \"image-registry-54854c7897-pc7km\" (UID: \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\") " pod="openshift-image-registry/image-registry-54854c7897-pc7km" Oct 31 07:30:15.550529 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.550392 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/28be803a-a932-4009-ad61-b356366821ef-default-certificate\") pod \"router-default-5549647ff-cxcdt\" (UID: \"28be803a-a932-4009-ad61-b356366821ef\") " pod="openshift-ingress/router-default-5549647ff-cxcdt" Oct 31 07:30:15.550922 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.550575 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/396e1c5e-3700-4fe6-b367-54ad2a244225-serving-cert\") pod \"console-operator-7cd56b8dbc-jztcz\" (UID: \"396e1c5e-3700-4fe6-b367-54ad2a244225\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jztcz" Oct 31 07:30:15.550922 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.550706 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-registry-certificates\") pod \"image-registry-54854c7897-pc7km\" (UID: \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\") " pod="openshift-image-registry/image-registry-54854c7897-pc7km" Oct 31 07:30:15.551312 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.551292 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-image-registry-private-configuration\") pod \"image-registry-54854c7897-pc7km\" (UID: \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\") " pod="openshift-image-registry/image-registry-54854c7897-pc7km" Oct 31 07:30:15.551523 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.551509 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/00a0d978-8020-46df-932d-b0dd22f946f9-cert\") pod \"ingress-canary-22cmd\" (UID: \"00a0d978-8020-46df-932d-b0dd22f946f9\") " pod="openshift-ingress-canary/ingress-canary-22cmd" Oct 31 07:30:15.551866 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.551851 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-installation-pull-secrets\") pod \"image-registry-54854c7897-pc7km\" (UID: \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\") " pod="openshift-image-registry/image-registry-54854c7897-pc7km" Oct 31 07:30:15.565543 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.565017 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsvzt\" (UniqueName: \"kubernetes.io/projected/6fae9634-7abe-40c6-a892-9eb58166e9ad-kube-api-access-wsvzt\") pod \"kas-connection-checker-685895c7c9-xkmbn\" (UID: \"6fae9634-7abe-40c6-a892-9eb58166e9ad\") " pod="kube-system/kas-connection-checker-685895c7c9-xkmbn" Oct 31 07:30:15.565543 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.565065 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cf9zh\" (UniqueName: \"kubernetes.io/projected/28be803a-a932-4009-ad61-b356366821ef-kube-api-access-cf9zh\") pod \"router-default-5549647ff-cxcdt\" (UID: \"28be803a-a932-4009-ad61-b356366821ef\") " pod="openshift-ingress/router-default-5549647ff-cxcdt" Oct 31 07:30:15.565543 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.565335 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4q9t4\" (UniqueName: \"kubernetes.io/projected/58bc3ea1-f58c-4891-a049-1289a99be055-kube-api-access-4q9t4\") pod \"dns-default-z5sfp\" (UID: \"58bc3ea1-f58c-4891-a049-1289a99be055\") " pod="openshift-dns/dns-default-z5sfp" Oct 31 07:30:15.566471 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.566396 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xksf5\" (UniqueName: \"kubernetes.io/projected/e84bef13-a137-49f7-931d-9f94952dacfc-kube-api-access-xksf5\") pod \"kas-connection-checker-685895c7c9-cpkfr\" (UID: \"e84bef13-a137-49f7-931d-9f94952dacfc\") " pod="kube-system/kas-connection-checker-685895c7c9-cpkfr" Oct 31 07:30:15.566673 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.566650 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9k9q8\" (UniqueName: \"kubernetes.io/projected/b4dd8884-1d6e-4abe-ad6c-3c947623bb60-kube-api-access-9k9q8\") pod \"insights-operator-7c4986d449-hz58r\" (UID: \"b4dd8884-1d6e-4abe-ad6c-3c947623bb60\") " pod="openshift-insights/insights-operator-7c4986d449-hz58r" Oct 31 07:30:15.567742 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.567370 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cb5dx\" (UniqueName: \"kubernetes.io/projected/f3bf19fd-bc58-4dff-a1a8-4f61381e1fde-kube-api-access-cb5dx\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-46qvv\" (UID: \"f3bf19fd-bc58-4dff-a1a8-4f61381e1fde\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-46qvv" Oct 31 07:30:15.567742 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.567725 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hldt6\" (UniqueName: \"kubernetes.io/projected/4104ba06-fba8-4c3c-b876-cca5087a7d0a-kube-api-access-hldt6\") pod \"image-registry-69fbc98d44-fn7fh\" (UID: \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\") " pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" Oct 31 07:30:15.567742 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.567737 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lx427\" (UniqueName: \"kubernetes.io/projected/54296c9d-1c43-44cd-bfb6-d81fe892484c-kube-api-access-lx427\") pod \"volume-data-source-validator-58c9f5f46d-m6sns\" (UID: \"54296c9d-1c43-44cd-bfb6-d81fe892484c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-m6sns" Oct 31 07:30:15.568494 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.568468 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-djqrt\" (UniqueName: \"kubernetes.io/projected/5e743234-2b5f-4c91-9b06-189f546cbce4-kube-api-access-djqrt\") pod \"cluster-monitoring-operator-884576c57-282kr\" (UID: \"5e743234-2b5f-4c91-9b06-189f546cbce4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-282kr" Oct 31 07:30:15.569286 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.569263 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jhpf\" (UniqueName: \"kubernetes.io/projected/00a0d978-8020-46df-932d-b0dd22f946f9-kube-api-access-2jhpf\") pod \"ingress-canary-22cmd\" (UID: \"00a0d978-8020-46df-932d-b0dd22f946f9\") " pod="openshift-ingress-canary/ingress-canary-22cmd" Oct 31 07:30:15.569493 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.569473 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlm6c\" (UniqueName: \"kubernetes.io/projected/d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e-kube-api-access-nlm6c\") pod \"cluster-samples-operator-6ddb6c8b94-86n8d\" (UID: \"d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-86n8d" Oct 31 07:30:15.569689 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.569666 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-bound-sa-token\") pod \"image-registry-54854c7897-pc7km\" (UID: \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\") " pod="openshift-image-registry/image-registry-54854c7897-pc7km" Oct 31 07:30:15.569953 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.569935 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4104ba06-fba8-4c3c-b876-cca5087a7d0a-bound-sa-token\") pod \"image-registry-69fbc98d44-fn7fh\" (UID: \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\") " pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" Oct 31 07:30:15.570435 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.570413 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5s89\" (UniqueName: \"kubernetes.io/projected/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-kube-api-access-z5s89\") pod \"image-registry-54854c7897-pc7km\" (UID: \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\") " pod="openshift-image-registry/image-registry-54854c7897-pc7km" Oct 31 07:30:15.571016 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.570991 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p76jj\" (UniqueName: \"kubernetes.io/projected/b6997c76-e103-4c3f-b39d-a6a17fb98270-kube-api-access-p76jj\") pod \"service-ca-operator-544d749c8c-62tn2\" (UID: \"b6997c76-e103-4c3f-b39d-a6a17fb98270\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-62tn2" Oct 31 07:30:15.571360 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.571327 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-82jqk\" (UniqueName: \"kubernetes.io/projected/43c59587-4ff8-4546-9c0a-6161ece201d0-kube-api-access-82jqk\") pod \"network-check-source-6f78d59f99-4xtdb\" (UID: \"43c59587-4ff8-4546-9c0a-6161ece201d0\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-4xtdb" Oct 31 07:30:15.571769 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.571749 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g76kv\" (UniqueName: \"kubernetes.io/projected/396e1c5e-3700-4fe6-b367-54ad2a244225-kube-api-access-g76kv\") pod \"console-operator-7cd56b8dbc-jztcz\" (UID: \"396e1c5e-3700-4fe6-b367-54ad2a244225\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jztcz" Oct 31 07:30:15.592083 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.589964 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-vhcq9" Oct 31 07:30:15.597415 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.597400 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-cpkfr" Oct 31 07:30:15.615803 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.615781 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-xkmbn" Oct 31 07:30:15.643531 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.643504 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-m6sns" Oct 31 07:30:15.649746 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.646654 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-46qvv" Oct 31 07:30:15.652807 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.652788 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-62tn2" Oct 31 07:30:15.672823 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.670525 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-hz58r" Oct 31 07:30:15.694404 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.686909 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-jztcz" Oct 31 07:30:15.697145 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.697123 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-4xtdb" Oct 31 07:30:15.715930 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.712866 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-22cmd" Oct 31 07:30:15.905591 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.905568 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-cpkfr"] Oct 31 07:30:15.907259 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.907241 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-vhcq9"] Oct 31 07:30:15.914131 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:30:15.914104 2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode84bef13_a137_49f7_931d_9f94952dacfc.slice/crio-81bc093b082071b56e9c9f3892c254c3cd10ea34a7f034ea79fe9936867ca71f WatchSource:0}: Error finding container 81bc093b082071b56e9c9f3892c254c3cd10ea34a7f034ea79fe9936867ca71f: Status 404 returned error can't find the container with id 81bc093b082071b56e9c9f3892c254c3cd10ea34a7f034ea79fe9936867ca71f Oct 31 07:30:15.924877 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:30:15.924852 2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cbf4695_0536_45d5_bc48_be56861b87b8.slice/crio-8d5c8c577bce94daefa7f9e842462e7f57e5680293162c66c04f4aa7ebcc56d9 WatchSource:0}: Error finding container 8d5c8c577bce94daefa7f9e842462e7f57e5680293162c66c04f4aa7ebcc56d9: Status 404 returned error can't find the container with id 8d5c8c577bce94daefa7f9e842462e7f57e5680293162c66c04f4aa7ebcc56d9 Oct 31 07:30:15.950278 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:15.950250 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-xkmbn"] Oct 31 07:30:15.956702 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:30:15.956680 2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fae9634_7abe_40c6_a892_9eb58166e9ad.slice/crio-d187de26ca97f373060f15bc026d354398aa7202968b574737f4169340a5bc1a WatchSource:0}: Error finding container d187de26ca97f373060f15bc026d354398aa7202968b574737f4169340a5bc1a: Status 404 returned error can't find the container with id d187de26ca97f373060f15bc026d354398aa7202968b574737f4169340a5bc1a Oct 31 07:30:16.001473 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:16.000668 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-m6sns"] Oct 31 07:30:16.011494 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:30:16.010155 2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54296c9d_1c43_44cd_bfb6_d81fe892484c.slice/crio-a847b89d581384509483d914b77bb38b311ee6414adcbda176abfa587c130864 WatchSource:0}: Error finding container a847b89d581384509483d914b77bb38b311ee6414adcbda176abfa587c130864: Status 404 returned error can't find the container with id a847b89d581384509483d914b77bb38b311ee6414adcbda176abfa587c130864 Oct 31 07:30:16.019755 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:16.017400 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-62tn2"] Oct 31 07:30:16.039109 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:16.039074 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:16.049448 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:16.049426 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/58bc3ea1-f58c-4891-a049-1289a99be055-metrics-tls\") pod \"dns-default-z5sfp\" (UID: \"58bc3ea1-f58c-4891-a049-1289a99be055\") " pod="openshift-dns/dns-default-z5sfp" Oct 31 07:30:16.049448 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:16.049466 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28be803a-a932-4009-ad61-b356366821ef-service-ca-bundle\") pod \"router-default-5549647ff-cxcdt\" (UID: \"28be803a-a932-4009-ad61-b356366821ef\") " pod="openshift-ingress/router-default-5549647ff-cxcdt" Oct 31 07:30:16.049646 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:16.049555 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-86n8d\" (UID: \"d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-86n8d" Oct 31 07:30:16.049646 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:16.049617 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-registry-tls\") pod \"image-registry-54854c7897-pc7km\" (UID: \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\") " pod="openshift-image-registry/image-registry-54854c7897-pc7km" Oct 31 07:30:16.049771 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:16.049677 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 31 07:30:16.049771 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:16.049688 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5e743234-2b5f-4c91-9b06-189f546cbce4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-282kr\" (UID: \"5e743234-2b5f-4c91-9b06-189f546cbce4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-282kr" Oct 31 07:30:16.049771 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:16.049734 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58bc3ea1-f58c-4891-a049-1289a99be055-metrics-tls podName:58bc3ea1-f58c-4891-a049-1289a99be055 nodeName:}" failed. No retries permitted until 2026-10-31 07:30:17.049715968 +0000 UTC m=+32.058292388 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/58bc3ea1-f58c-4891-a049-1289a99be055-metrics-tls") pod "dns-default-z5sfp" (UID: "58bc3ea1-f58c-4891-a049-1289a99be055") : secret "dns-default-metrics-tls" not found Oct 31 07:30:16.049923 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:16.049805 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 31 07:30:16.049923 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:16.049853 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e743234-2b5f-4c91-9b06-189f546cbce4-cluster-monitoring-operator-tls podName:5e743234-2b5f-4c91-9b06-189f546cbce4 nodeName:}" failed. No retries permitted until 2026-10-31 07:30:17.049838844 +0000 UTC m=+32.058415253 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5e743234-2b5f-4c91-9b06-189f546cbce4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-282kr" (UID: "5e743234-2b5f-4c91-9b06-189f546cbce4") : secret "cluster-monitoring-operator-tls" not found Oct 31 07:30:16.049923 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:16.049907 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Oct 31 07:30:16.050137 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:16.049939 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e-samples-operator-tls podName:d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e nodeName:}" failed. No retries permitted until 2026-10-31 07:30:17.049927272 +0000 UTC m=+32.058503680 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-86n8d" (UID: "d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e") : secret "samples-operator-tls" not found Oct 31 07:30:16.050137 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:16.050011 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 31 07:30:16.050137 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:16.050022 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-54854c7897-pc7km: secret "image-registry-tls" not found Oct 31 07:30:16.050137 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:16.050051 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-registry-tls podName:c734ebf9-eba8-4e14-b20d-2d78dd65b10d nodeName:}" failed. No retries permitted until 2026-10-31 07:30:17.050040837 +0000 UTC m=+32.058617243 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-registry-tls") pod "image-registry-54854c7897-pc7km" (UID: "c734ebf9-eba8-4e14-b20d-2d78dd65b10d") : secret "image-registry-tls" not found Oct 31 07:30:16.050137 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:16.050121 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/28be803a-a932-4009-ad61-b356366821ef-service-ca-bundle podName:28be803a-a932-4009-ad61-b356366821ef nodeName:}" failed. No retries permitted until 2026-10-31 07:30:17.050086714 +0000 UTC m=+32.058663124 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/28be803a-a932-4009-ad61-b356366821ef-service-ca-bundle") pod "router-default-5549647ff-cxcdt" (UID: "28be803a-a932-4009-ad61-b356366821ef") : configmap references non-existent config key: service-ca.crt Oct 31 07:30:16.075599 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:16.075575 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-22cmd"] Oct 31 07:30:16.079876 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:16.079858 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-jztcz"] Oct 31 07:30:16.082157 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:30:16.082132 2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00a0d978_8020_46df_932d_b0dd22f946f9.slice/crio-61e6f367bdfe0ea5c7737b4f795fed21ec5aaa3f12e073243a74a976db2b98c6 WatchSource:0}: Error finding container 61e6f367bdfe0ea5c7737b4f795fed21ec5aaa3f12e073243a74a976db2b98c6: Status 404 returned error can't find the container with id 61e6f367bdfe0ea5c7737b4f795fed21ec5aaa3f12e073243a74a976db2b98c6 Oct 31 07:30:16.083830 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:30:16.083808 2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod396e1c5e_3700_4fe6_b367_54ad2a244225.slice/crio-a1f16ffaf583d8408a8973ff31c9322c54494ca1527c33466774e135f0fad17d WatchSource:0}: Error finding container a1f16ffaf583d8408a8973ff31c9322c54494ca1527c33466774e135f0fad17d: Status 404 returned error can't find the container with id a1f16ffaf583d8408a8973ff31c9322c54494ca1527c33466774e135f0fad17d Oct 31 07:30:16.088471 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:16.088450 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-46qvv"] Oct 31 07:30:16.091410 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:16.091388 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-hz58r"] Oct 31 07:30:16.098790 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:30:16.098767 2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3bf19fd_bc58_4dff_a1a8_4f61381e1fde.slice/crio-9e4560e0b325aa756651336d460ead8b3ed6ad858c3db99b05554503a1e4225d WatchSource:0}: Error finding container 9e4560e0b325aa756651336d460ead8b3ed6ad858c3db99b05554503a1e4225d: Status 404 returned error can't find the container with id 9e4560e0b325aa756651336d460ead8b3ed6ad858c3db99b05554503a1e4225d Oct 31 07:30:16.099020 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:30:16.098999 2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4dd8884_1d6e_4abe_ad6c_3c947623bb60.slice/crio-e8964c5013096f6a4215b0a3726ca0ca9205b84920d1200007deb94fecce9b21 WatchSource:0}: Error finding container e8964c5013096f6a4215b0a3726ca0ca9205b84920d1200007deb94fecce9b21: Status 404 returned error can't find the container with id e8964c5013096f6a4215b0a3726ca0ca9205b84920d1200007deb94fecce9b21 Oct 31 07:30:16.118431 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:16.118383 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-4xtdb"] Oct 31 07:30:16.121116 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:30:16.121095 2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43c59587_4ff8_4546_9c0a_6161ece201d0.slice/crio-8110977887cda9cf1367fb243562b8e66c4948348cec19a4e8b33389cf193838 WatchSource:0}: Error finding container 8110977887cda9cf1367fb243562b8e66c4948348cec19a4e8b33389cf193838: Status 404 returned error can't find the container with id 8110977887cda9cf1367fb243562b8e66c4948348cec19a4e8b33389cf193838 Oct 31 07:30:16.150193 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:16.150172 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/563d3b8b-0040-439e-b0b2-e858eed5abf6-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-s75pm\" (UID: \"563d3b8b-0040-439e-b0b2-e858eed5abf6\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-s75pm" Oct 31 07:30:16.150267 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:16.150203 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28be803a-a932-4009-ad61-b356366821ef-metrics-certs\") pod \"router-default-5549647ff-cxcdt\" (UID: \"28be803a-a932-4009-ad61-b356366821ef\") " pod="openshift-ingress/router-default-5549647ff-cxcdt" Oct 31 07:30:16.150267 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:16.150223 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4104ba06-fba8-4c3c-b876-cca5087a7d0a-registry-tls\") pod \"image-registry-69fbc98d44-fn7fh\" (UID: \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\") " pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" Oct 31 07:30:16.150379 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:16.150318 2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Oct 31 07:30:16.150379 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:16.150339 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 31 07:30:16.150379 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:16.150318 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Oct 31 07:30:16.150479 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:16.150390 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/563d3b8b-0040-439e-b0b2-e858eed5abf6-networking-console-plugin-cert podName:563d3b8b-0040-439e-b0b2-e858eed5abf6 nodeName:}" failed. No retries permitted until 2026-10-31 07:30:17.150370791 +0000 UTC m=+32.158947198 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/563d3b8b-0040-439e-b0b2-e858eed5abf6-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-s75pm" (UID: "563d3b8b-0040-439e-b0b2-e858eed5abf6") : secret "networking-console-plugin-cert" not found Oct 31 07:30:16.150479 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:16.150439 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28be803a-a932-4009-ad61-b356366821ef-metrics-certs podName:28be803a-a932-4009-ad61-b356366821ef nodeName:}" failed. No retries permitted until 2026-10-31 07:30:17.150429295 +0000 UTC m=+32.159005700 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/28be803a-a932-4009-ad61-b356366821ef-metrics-certs") pod "router-default-5549647ff-cxcdt" (UID: "28be803a-a932-4009-ad61-b356366821ef") : secret "router-metrics-certs-default" not found Oct 31 07:30:16.150479 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:16.150443 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69fbc98d44-fn7fh: secret "image-registry-tls" not found Oct 31 07:30:16.150479 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:16.150478 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4104ba06-fba8-4c3c-b876-cca5087a7d0a-registry-tls podName:4104ba06-fba8-4c3c-b876-cca5087a7d0a nodeName:}" failed. No retries permitted until 2026-10-31 07:30:17.150467963 +0000 UTC m=+32.159044388 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4104ba06-fba8-4c3c-b876-cca5087a7d0a-registry-tls") pod "image-registry-69fbc98d44-fn7fh" (UID: "4104ba06-fba8-4c3c-b876-cca5087a7d0a") : secret "image-registry-tls" not found Oct 31 07:30:16.292000 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:16.291966 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:16.486142 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:16.486109 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8hpcb" Oct 31 07:30:16.486309 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:16.486106 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ncnd7" Oct 31 07:30:16.486309 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:16.486119 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7thxr" Oct 31 07:30:16.490475 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:16.490448 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Oct 31 07:30:16.490598 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:16.490551 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-x6jvt\"" Oct 31 07:30:16.490598 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:16.490556 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xb7gt\"" Oct 31 07:30:16.490598 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:16.490555 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Oct 31 07:30:16.490598 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:16.490594 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-p6nqs\"" Oct 31 07:30:16.727418 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:16.726649 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-vhcq9" event={"ID":"3cbf4695-0536-45d5-bc48-be56861b87b8","Type":"ContainerStarted","Data":"9655f1b5f2b9e0e2212207139acb8487e5a20c8a90eabf7a0699fc211eb2fa60"} Oct 31 07:30:16.727418 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:16.726744 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-vhcq9" event={"ID":"3cbf4695-0536-45d5-bc48-be56861b87b8","Type":"ContainerStarted","Data":"8d5c8c577bce94daefa7f9e842462e7f57e5680293162c66c04f4aa7ebcc56d9"} Oct 31 07:30:16.729956 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:16.729904 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-46qvv" event={"ID":"f3bf19fd-bc58-4dff-a1a8-4f61381e1fde","Type":"ContainerStarted","Data":"9e4560e0b325aa756651336d460ead8b3ed6ad858c3db99b05554503a1e4225d"} Oct 31 07:30:16.731647 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:16.731623 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-22cmd" event={"ID":"00a0d978-8020-46df-932d-b0dd22f946f9","Type":"ContainerStarted","Data":"61e6f367bdfe0ea5c7737b4f795fed21ec5aaa3f12e073243a74a976db2b98c6"} Oct 31 07:30:16.734473 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:16.734452 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-hz58r" event={"ID":"b4dd8884-1d6e-4abe-ad6c-3c947623bb60","Type":"ContainerStarted","Data":"e8964c5013096f6a4215b0a3726ca0ca9205b84920d1200007deb94fecce9b21"} Oct 31 07:30:16.735958 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:16.735935 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-jztcz" event={"ID":"396e1c5e-3700-4fe6-b367-54ad2a244225","Type":"ContainerStarted","Data":"a1f16ffaf583d8408a8973ff31c9322c54494ca1527c33466774e135f0fad17d"} Oct 31 07:30:16.737243 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:16.737165 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-4xtdb" event={"ID":"43c59587-4ff8-4546-9c0a-6161ece201d0","Type":"ContainerStarted","Data":"8110977887cda9cf1367fb243562b8e66c4948348cec19a4e8b33389cf193838"} Oct 31 07:30:16.738545 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:16.738521 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-62tn2" event={"ID":"b6997c76-e103-4c3f-b39d-a6a17fb98270","Type":"ContainerStarted","Data":"115e6d16cbb3c78a7ada677b2d211ea416d8c8df87e680755a40bb9eaf9d4bb6"} Oct 31 07:30:16.740619 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:16.740587 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-m6sns" event={"ID":"54296c9d-1c43-44cd-bfb6-d81fe892484c","Type":"ContainerStarted","Data":"a847b89d581384509483d914b77bb38b311ee6414adcbda176abfa587c130864"} Oct 31 07:30:16.744004 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:16.743940 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-xkmbn" event={"ID":"6fae9634-7abe-40c6-a892-9eb58166e9ad","Type":"ContainerStarted","Data":"dd485695e5730564dcdefd62fb95b10e72fea6626cb0c9aebea5491c235b016e"} Oct 31 07:30:16.744004 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:16.743979 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-xkmbn" event={"ID":"6fae9634-7abe-40c6-a892-9eb58166e9ad","Type":"ContainerStarted","Data":"d187de26ca97f373060f15bc026d354398aa7202968b574737f4169340a5bc1a"} Oct 31 07:30:16.745488 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:16.745442 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-vhcq9" podStartSLOduration=398.745425766 podStartE2EDuration="6m38.745425766s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:23:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:30:16.744776844 +0000 UTC m=+31.753353290" watchObservedRunningTime="2026-10-31 07:30:16.745425766 +0000 UTC m=+31.754002604" Oct 31 07:30:16.750456 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:16.750429 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-cpkfr" event={"ID":"e84bef13-a137-49f7-931d-9f94952dacfc","Type":"ContainerStarted","Data":"2f928e91840899e1813413bfc7d44a85651245378d69ad025477b7c5dfc4f05c"} Oct 31 07:30:16.750555 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:16.750472 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-cpkfr" event={"ID":"e84bef13-a137-49f7-931d-9f94952dacfc","Type":"ContainerStarted","Data":"81bc093b082071b56e9c9f3892c254c3cd10ea34a7f034ea79fe9936867ca71f"} Oct 31 07:30:16.765738 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:16.765696 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-xkmbn" podStartSLOduration=398.765680328 podStartE2EDuration="6m38.765680328s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:23:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:30:16.764989863 +0000 UTC m=+31.773566364" watchObservedRunningTime="2026-10-31 07:30:16.765680328 +0000 UTC m=+31.774256758" Oct 31 07:30:16.782384 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:16.782320 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-cpkfr" podStartSLOduration=398.782305127 podStartE2EDuration="6m38.782305127s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:23:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:30:16.781526196 +0000 UTC m=+31.790102644" watchObservedRunningTime="2026-10-31 07:30:16.782305127 +0000 UTC m=+31.790881555" Oct 31 07:30:17.060040 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:17.059963 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/58bc3ea1-f58c-4891-a049-1289a99be055-metrics-tls\") pod \"dns-default-z5sfp\" (UID: \"58bc3ea1-f58c-4891-a049-1289a99be055\") " pod="openshift-dns/dns-default-z5sfp" Oct 31 07:30:17.060040 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:17.060016 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28be803a-a932-4009-ad61-b356366821ef-service-ca-bundle\") pod \"router-default-5549647ff-cxcdt\" (UID: \"28be803a-a932-4009-ad61-b356366821ef\") " pod="openshift-ingress/router-default-5549647ff-cxcdt" Oct 31 07:30:17.060239 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:17.060061 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-86n8d\" (UID: \"d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-86n8d" Oct 31 07:30:17.060239 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:17.060101 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-registry-tls\") pod \"image-registry-54854c7897-pc7km\" (UID: \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\") " pod="openshift-image-registry/image-registry-54854c7897-pc7km" Oct 31 07:30:17.060239 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:17.060164 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5e743234-2b5f-4c91-9b06-189f546cbce4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-282kr\" (UID: \"5e743234-2b5f-4c91-9b06-189f546cbce4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-282kr" Oct 31 07:30:17.060417 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:17.060384 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 31 07:30:17.060472 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:17.060456 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e743234-2b5f-4c91-9b06-189f546cbce4-cluster-monitoring-operator-tls podName:5e743234-2b5f-4c91-9b06-189f546cbce4 nodeName:}" failed. No retries permitted until 2026-10-31 07:30:19.060436319 +0000 UTC m=+34.069012739 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5e743234-2b5f-4c91-9b06-189f546cbce4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-282kr" (UID: "5e743234-2b5f-4c91-9b06-189f546cbce4") : secret "cluster-monitoring-operator-tls" not found Oct 31 07:30:17.060924 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:17.060904 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 31 07:30:17.061025 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:17.060957 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58bc3ea1-f58c-4891-a049-1289a99be055-metrics-tls podName:58bc3ea1-f58c-4891-a049-1289a99be055 nodeName:}" failed. No retries permitted until 2026-10-31 07:30:19.06094358 +0000 UTC m=+34.069519995 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/58bc3ea1-f58c-4891-a049-1289a99be055-metrics-tls") pod "dns-default-z5sfp" (UID: "58bc3ea1-f58c-4891-a049-1289a99be055") : secret "dns-default-metrics-tls" not found Oct 31 07:30:17.061025 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:17.061022 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/28be803a-a932-4009-ad61-b356366821ef-service-ca-bundle podName:28be803a-a932-4009-ad61-b356366821ef nodeName:}" failed. No retries permitted until 2026-10-31 07:30:19.061012488 +0000 UTC m=+34.069588908 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/28be803a-a932-4009-ad61-b356366821ef-service-ca-bundle") pod "router-default-5549647ff-cxcdt" (UID: "28be803a-a932-4009-ad61-b356366821ef") : configmap references non-existent config key: service-ca.crt Oct 31 07:30:17.061153 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:17.061133 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Oct 31 07:30:17.061208 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:17.061173 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e-samples-operator-tls podName:d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e nodeName:}" failed. No retries permitted until 2026-10-31 07:30:19.061161926 +0000 UTC m=+34.069738346 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-86n8d" (UID: "d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e") : secret "samples-operator-tls" not found Oct 31 07:30:17.061280 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:17.061236 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 31 07:30:17.061280 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:17.061248 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-54854c7897-pc7km: secret "image-registry-tls" not found Oct 31 07:30:17.061280 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:17.061279 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-registry-tls podName:c734ebf9-eba8-4e14-b20d-2d78dd65b10d nodeName:}" failed. No retries permitted until 2026-10-31 07:30:19.061268482 +0000 UTC m=+34.069844902 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-registry-tls") pod "image-registry-54854c7897-pc7km" (UID: "c734ebf9-eba8-4e14-b20d-2d78dd65b10d") : secret "image-registry-tls" not found Oct 31 07:30:17.161513 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:17.161474 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/563d3b8b-0040-439e-b0b2-e858eed5abf6-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-s75pm\" (UID: \"563d3b8b-0040-439e-b0b2-e858eed5abf6\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-s75pm" Oct 31 07:30:17.161683 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:17.161525 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28be803a-a932-4009-ad61-b356366821ef-metrics-certs\") pod \"router-default-5549647ff-cxcdt\" (UID: \"28be803a-a932-4009-ad61-b356366821ef\") " pod="openshift-ingress/router-default-5549647ff-cxcdt" Oct 31 07:30:17.161683 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:17.161546 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4104ba06-fba8-4c3c-b876-cca5087a7d0a-registry-tls\") pod \"image-registry-69fbc98d44-fn7fh\" (UID: \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\") " pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" Oct 31 07:30:17.161797 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:17.161726 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 31 07:30:17.161797 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:17.161742 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69fbc98d44-fn7fh: secret "image-registry-tls" not found Oct 31 07:30:17.161886 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:17.161798 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4104ba06-fba8-4c3c-b876-cca5087a7d0a-registry-tls podName:4104ba06-fba8-4c3c-b876-cca5087a7d0a nodeName:}" failed. No retries permitted until 2026-10-31 07:30:19.161779978 +0000 UTC m=+34.170356398 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4104ba06-fba8-4c3c-b876-cca5087a7d0a-registry-tls") pod "image-registry-69fbc98d44-fn7fh" (UID: "4104ba06-fba8-4c3c-b876-cca5087a7d0a") : secret "image-registry-tls" not found Oct 31 07:30:17.162217 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:17.162193 2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Oct 31 07:30:17.162340 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:17.162250 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/563d3b8b-0040-439e-b0b2-e858eed5abf6-networking-console-plugin-cert podName:563d3b8b-0040-439e-b0b2-e858eed5abf6 nodeName:}" failed. No retries permitted until 2026-10-31 07:30:19.162236098 +0000 UTC m=+34.170812518 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/563d3b8b-0040-439e-b0b2-e858eed5abf6-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-s75pm" (UID: "563d3b8b-0040-439e-b0b2-e858eed5abf6") : secret "networking-console-plugin-cert" not found Oct 31 07:30:17.162340 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:17.162303 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Oct 31 07:30:17.162340 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:17.162333 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28be803a-a932-4009-ad61-b356366821ef-metrics-certs podName:28be803a-a932-4009-ad61-b356366821ef nodeName:}" failed. No retries permitted until 2026-10-31 07:30:19.162323638 +0000 UTC m=+34.170900060 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/28be803a-a932-4009-ad61-b356366821ef-metrics-certs") pod "router-default-5549647ff-cxcdt" (UID: "28be803a-a932-4009-ad61-b356366821ef") : secret "router-metrics-certs-default" not found Oct 31 07:30:18.767873 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:18.767827 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-m6sns" event={"ID":"54296c9d-1c43-44cd-bfb6-d81fe892484c","Type":"ContainerStarted","Data":"e83f383212b2043521b7b04790eeb87595548a629691c1ba54fc79895e82c590"} Oct 31 07:30:19.004655 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:19.004561 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:19.083705 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:19.083622 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/58bc3ea1-f58c-4891-a049-1289a99be055-metrics-tls\") pod \"dns-default-z5sfp\" (UID: \"58bc3ea1-f58c-4891-a049-1289a99be055\") " pod="openshift-dns/dns-default-z5sfp" Oct 31 07:30:19.083705 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:19.083673 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28be803a-a932-4009-ad61-b356366821ef-service-ca-bundle\") pod \"router-default-5549647ff-cxcdt\" (UID: \"28be803a-a932-4009-ad61-b356366821ef\") " pod="openshift-ingress/router-default-5549647ff-cxcdt" Oct 31 07:30:19.083904 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:19.083718 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-86n8d\" (UID: \"d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-86n8d" Oct 31 07:30:19.083904 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:19.083755 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-registry-tls\") pod \"image-registry-54854c7897-pc7km\" (UID: \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\") " pod="openshift-image-registry/image-registry-54854c7897-pc7km" Oct 31 07:30:19.083904 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:19.083812 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5e743234-2b5f-4c91-9b06-189f546cbce4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-282kr\" (UID: \"5e743234-2b5f-4c91-9b06-189f546cbce4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-282kr" Oct 31 07:30:19.084041 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:19.083980 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 31 07:30:19.084083 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:19.084047 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e743234-2b5f-4c91-9b06-189f546cbce4-cluster-monitoring-operator-tls podName:5e743234-2b5f-4c91-9b06-189f546cbce4 nodeName:}" failed. No retries permitted until 2026-10-31 07:30:23.084027393 +0000 UTC m=+38.092603800 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5e743234-2b5f-4c91-9b06-189f546cbce4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-282kr" (UID: "5e743234-2b5f-4c91-9b06-189f546cbce4") : secret "cluster-monitoring-operator-tls" not found Oct 31 07:30:19.084307 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:19.084227 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Oct 31 07:30:19.084307 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:19.084280 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e-samples-operator-tls podName:d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e nodeName:}" failed. No retries permitted until 2026-10-31 07:30:23.084265109 +0000 UTC m=+38.092841514 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-86n8d" (UID: "d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e") : secret "samples-operator-tls" not found Oct 31 07:30:19.084307 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:19.084288 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 31 07:30:19.084517 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:19.084305 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 31 07:30:19.084517 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:19.084313 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-54854c7897-pc7km: secret "image-registry-tls" not found Oct 31 07:30:19.084517 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:19.084313 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/28be803a-a932-4009-ad61-b356366821ef-service-ca-bundle podName:28be803a-a932-4009-ad61-b356366821ef nodeName:}" failed. No retries permitted until 2026-10-31 07:30:23.084290406 +0000 UTC m=+38.092866830 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/28be803a-a932-4009-ad61-b356366821ef-service-ca-bundle") pod "router-default-5549647ff-cxcdt" (UID: "28be803a-a932-4009-ad61-b356366821ef") : configmap references non-existent config key: service-ca.crt Oct 31 07:30:19.084517 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:19.084401 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58bc3ea1-f58c-4891-a049-1289a99be055-metrics-tls podName:58bc3ea1-f58c-4891-a049-1289a99be055 nodeName:}" failed. No retries permitted until 2026-10-31 07:30:23.084384508 +0000 UTC m=+38.092960929 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/58bc3ea1-f58c-4891-a049-1289a99be055-metrics-tls") pod "dns-default-z5sfp" (UID: "58bc3ea1-f58c-4891-a049-1289a99be055") : secret "dns-default-metrics-tls" not found Oct 31 07:30:19.084517 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:19.084416 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-registry-tls podName:c734ebf9-eba8-4e14-b20d-2d78dd65b10d nodeName:}" failed. No retries permitted until 2026-10-31 07:30:23.084407172 +0000 UTC m=+38.092983586 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-registry-tls") pod "image-registry-54854c7897-pc7km" (UID: "c734ebf9-eba8-4e14-b20d-2d78dd65b10d") : secret "image-registry-tls" not found Oct 31 07:30:19.185282 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:19.184748 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28ec9621-7ec5-421c-a88b-51bb20d3a7bf-metrics-certs\") pod \"network-metrics-daemon-7thxr\" (UID: \"28ec9621-7ec5-421c-a88b-51bb20d3a7bf\") " pod="openshift-multus/network-metrics-daemon-7thxr" Oct 31 07:30:19.185282 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:19.184826 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/563d3b8b-0040-439e-b0b2-e858eed5abf6-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-s75pm\" (UID: \"563d3b8b-0040-439e-b0b2-e858eed5abf6\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-s75pm" Oct 31 07:30:19.185282 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:19.184859 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28be803a-a932-4009-ad61-b356366821ef-metrics-certs\") pod \"router-default-5549647ff-cxcdt\" (UID: \"28be803a-a932-4009-ad61-b356366821ef\") " pod="openshift-ingress/router-default-5549647ff-cxcdt" Oct 31 07:30:19.185282 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:19.184886 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4104ba06-fba8-4c3c-b876-cca5087a7d0a-registry-tls\") pod \"image-registry-69fbc98d44-fn7fh\" (UID: \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\") " pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" Oct 31 07:30:19.185282 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:19.184931 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Oct 31 07:30:19.185282 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:19.185010 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28ec9621-7ec5-421c-a88b-51bb20d3a7bf-metrics-certs podName:28ec9621-7ec5-421c-a88b-51bb20d3a7bf nodeName:}" failed. No retries permitted until 2026-10-31 07:30:51.184987185 +0000 UTC m=+66.193563608 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/28ec9621-7ec5-421c-a88b-51bb20d3a7bf-metrics-certs") pod "network-metrics-daemon-7thxr" (UID: "28ec9621-7ec5-421c-a88b-51bb20d3a7bf") : secret "metrics-daemon-secret" not found Oct 31 07:30:19.185282 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:19.185069 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 31 07:30:19.185282 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:19.185084 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69fbc98d44-fn7fh: secret "image-registry-tls" not found Oct 31 07:30:19.185282 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:19.185150 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4104ba06-fba8-4c3c-b876-cca5087a7d0a-registry-tls podName:4104ba06-fba8-4c3c-b876-cca5087a7d0a nodeName:}" failed. No retries permitted until 2026-10-31 07:30:23.185133861 +0000 UTC m=+38.193710282 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4104ba06-fba8-4c3c-b876-cca5087a7d0a-registry-tls") pod "image-registry-69fbc98d44-fn7fh" (UID: "4104ba06-fba8-4c3c-b876-cca5087a7d0a") : secret "image-registry-tls" not found Oct 31 07:30:19.185282 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:19.185213 2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Oct 31 07:30:19.185282 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:19.185246 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/563d3b8b-0040-439e-b0b2-e858eed5abf6-networking-console-plugin-cert podName:563d3b8b-0040-439e-b0b2-e858eed5abf6 nodeName:}" failed. No retries permitted until 2026-10-31 07:30:23.185235779 +0000 UTC m=+38.193812200 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/563d3b8b-0040-439e-b0b2-e858eed5abf6-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-s75pm" (UID: "563d3b8b-0040-439e-b0b2-e858eed5abf6") : secret "networking-console-plugin-cert" not found Oct 31 07:30:19.185282 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:19.185286 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Oct 31 07:30:19.186008 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:19.185324 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28be803a-a932-4009-ad61-b356366821ef-metrics-certs podName:28be803a-a932-4009-ad61-b356366821ef nodeName:}" failed. No retries permitted until 2026-10-31 07:30:23.185311873 +0000 UTC m=+38.193888294 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/28be803a-a932-4009-ad61-b356366821ef-metrics-certs") pod "router-default-5549647ff-cxcdt" (UID: "28be803a-a932-4009-ad61-b356366821ef") : secret "router-metrics-certs-default" not found Oct 31 07:30:19.205659 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:19.205581 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:19.285936 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:19.285885 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/297b35d7-aa65-49c7-8460-ac3f9860248c-original-pull-secret\") pod \"global-pull-secret-syncer-8hpcb\" (UID: \"297b35d7-aa65-49c7-8460-ac3f9860248c\") " pod="kube-system/global-pull-secret-syncer-8hpcb" Oct 31 07:30:19.286123 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:19.286093 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ckzf\" (UniqueName: \"kubernetes.io/projected/d8b28b9f-37a2-4859-87fd-d27bb37d832b-kube-api-access-2ckzf\") pod \"network-check-target-ncnd7\" (UID: \"d8b28b9f-37a2-4859-87fd-d27bb37d832b\") " pod="openshift-network-diagnostics/network-check-target-ncnd7" Oct 31 07:30:19.292760 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:19.292736 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/297b35d7-aa65-49c7-8460-ac3f9860248c-original-pull-secret\") pod \"global-pull-secret-syncer-8hpcb\" (UID: \"297b35d7-aa65-49c7-8460-ac3f9860248c\") " pod="kube-system/global-pull-secret-syncer-8hpcb" Oct 31 07:30:19.292950 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:19.292928 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ckzf\" (UniqueName: \"kubernetes.io/projected/d8b28b9f-37a2-4859-87fd-d27bb37d832b-kube-api-access-2ckzf\") pod \"network-check-target-ncnd7\" (UID: \"d8b28b9f-37a2-4859-87fd-d27bb37d832b\") " pod="openshift-network-diagnostics/network-check-target-ncnd7" Oct 31 07:30:19.527431 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:19.527395 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8hpcb" Oct 31 07:30:19.545731 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:19.545698 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ncnd7" Oct 31 07:30:19.870905 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:19.870801 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-m6sns" podStartSLOduration=373.84338314 podStartE2EDuration="6m15.870780417s" totalImagesPullingTime="2.027397277s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:24:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:30:18.795448556 +0000 UTC m=+33.804025001" watchObservedRunningTime="2026-10-31 07:30:19.870780417 +0000 UTC m=+34.879356846" Oct 31 07:30:19.872107 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:19.872081 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8hpcb"] Oct 31 07:30:19.881631 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:30:19.881599 2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod297b35d7_aa65_49c7_8460_ac3f9860248c.slice/crio-1b3a1845b16b5781c7c4e35b89646268e616834157f7adca291705d99cc98e77 WatchSource:0}: Error finding container 1b3a1845b16b5781c7c4e35b89646268e616834157f7adca291705d99cc98e77: Status 404 returned error can't find the container with id 1b3a1845b16b5781c7c4e35b89646268e616834157f7adca291705d99cc98e77 Oct 31 07:30:19.910179 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:19.910099 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-ncnd7"] Oct 31 07:30:19.915933 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:30:19.915896 2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8b28b9f_37a2_4859_87fd_d27bb37d832b.slice/crio-023f6b65d0f9f8a68940ab14d162cd1c18a4e84cfe18bc9e02de25381fe84786 WatchSource:0}: Error finding container 023f6b65d0f9f8a68940ab14d162cd1c18a4e84cfe18bc9e02de25381fe84786: Status 404 returned error can't find the container with id 023f6b65d0f9f8a68940ab14d162cd1c18a4e84cfe18bc9e02de25381fe84786 Oct 31 07:30:20.784883 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:20.784857 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jztcz_396e1c5e-3700-4fe6-b367-54ad2a244225/console-operator/0.log" Oct 31 07:30:20.785065 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:20.784901 2577 generic.go:356] "Generic (PLEG): container finished" podID="396e1c5e-3700-4fe6-b367-54ad2a244225" containerID="ac145f554db2fcc07e35299c186cd97c04b22de2a436294ea030e82b19d304b7" exitCode=255 Oct 31 07:30:20.785065 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:20.785011 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-jztcz" event={"ID":"396e1c5e-3700-4fe6-b367-54ad2a244225","Type":"ContainerDied","Data":"ac145f554db2fcc07e35299c186cd97c04b22de2a436294ea030e82b19d304b7"} Oct 31 07:30:20.785224 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:20.785206 2577 scope.go:122] "RemoveContainer" containerID="ac145f554db2fcc07e35299c186cd97c04b22de2a436294ea030e82b19d304b7" Oct 31 07:30:20.788411 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:20.788385 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-ncnd7" event={"ID":"d8b28b9f-37a2-4859-87fd-d27bb37d832b","Type":"ContainerStarted","Data":"023f6b65d0f9f8a68940ab14d162cd1c18a4e84cfe18bc9e02de25381fe84786"} Oct 31 07:30:20.789935 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:20.789910 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8hpcb" event={"ID":"297b35d7-aa65-49c7-8460-ac3f9860248c","Type":"ContainerStarted","Data":"1b3a1845b16b5781c7c4e35b89646268e616834157f7adca291705d99cc98e77"} Oct 31 07:30:21.381501 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:21.381467 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:21.587579 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:21.587490 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:21.790098 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:21.790073 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:21.796079 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:21.796050 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-46qvv" event={"ID":"f3bf19fd-bc58-4dff-a1a8-4f61381e1fde","Type":"ContainerStarted","Data":"a4d14e51cf34ef6255e491bd4714d520f3505fa33dc1a5af8dda8e7ebfd24534"} Oct 31 07:30:21.801856 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:21.801795 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-22cmd" event={"ID":"00a0d978-8020-46df-932d-b0dd22f946f9","Type":"ContainerStarted","Data":"5c77c3faa6ad2087c027e8ebcb296b441e8779711afe95ccbb6501aca7ddbed7"} Oct 31 07:30:21.804288 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:21.804261 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-hz58r" event={"ID":"b4dd8884-1d6e-4abe-ad6c-3c947623bb60","Type":"ContainerStarted","Data":"c163670e087dcc1d3bf484637029cd5dc9bdeacff71daa0efb650167bf38ffd5"} Oct 31 07:30:21.806568 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:21.806548 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jztcz_396e1c5e-3700-4fe6-b367-54ad2a244225/console-operator/1.log" Oct 31 07:30:21.808100 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:21.808081 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jztcz_396e1c5e-3700-4fe6-b367-54ad2a244225/console-operator/0.log" Oct 31 07:30:21.808181 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:21.808118 2577 generic.go:356] "Generic (PLEG): container finished" podID="396e1c5e-3700-4fe6-b367-54ad2a244225" containerID="25aec100393d03aba086bd02f13db3038b24b6ae07422807f4c23edbb520f3f5" exitCode=255 Oct 31 07:30:21.808242 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:21.808223 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-jztcz" event={"ID":"396e1c5e-3700-4fe6-b367-54ad2a244225","Type":"ContainerDied","Data":"25aec100393d03aba086bd02f13db3038b24b6ae07422807f4c23edbb520f3f5"} Oct 31 07:30:21.808293 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:21.808263 2577 scope.go:122] "RemoveContainer" containerID="ac145f554db2fcc07e35299c186cd97c04b22de2a436294ea030e82b19d304b7" Oct 31 07:30:21.808531 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:21.808516 2577 scope.go:122] "RemoveContainer" containerID="25aec100393d03aba086bd02f13db3038b24b6ae07422807f4c23edbb520f3f5" Oct 31 07:30:21.808743 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:21.808722 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-jztcz_openshift-console-operator(396e1c5e-3700-4fe6-b367-54ad2a244225)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-jztcz" podUID="396e1c5e-3700-4fe6-b367-54ad2a244225" Oct 31 07:30:21.812371 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:21.812092 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-62tn2" event={"ID":"b6997c76-e103-4c3f-b39d-a6a17fb98270","Type":"ContainerStarted","Data":"802d75b7d7289ffc5378d73cc340e8c3619c827734d0500160e9b60b2167c7ab"} Oct 31 07:30:21.817818 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:21.817776 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-46qvv" podStartSLOduration=372.615625708 podStartE2EDuration="6m17.817761128s" totalImagesPullingTime="5.20213542s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:24:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:30:21.816726765 +0000 UTC m=+36.825303213" watchObservedRunningTime="2026-10-31 07:30:21.817761128 +0000 UTC m=+36.826337557" Oct 31 07:30:21.846483 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:21.846397 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-62tn2" podStartSLOduration=372.540985064 podStartE2EDuration="6m17.846383111s" totalImagesPullingTime="5.305398047s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:24:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:30:21.845566554 +0000 UTC m=+36.854143013" watchObservedRunningTime="2026-10-31 07:30:21.846383111 +0000 UTC m=+36.854959535" Oct 31 07:30:21.896298 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:21.896236 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-operator-7c4986d449-hz58r" podStartSLOduration=391.692721777 podStartE2EDuration="6m36.896217115s" totalImagesPullingTime="5.203495338s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:23:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:30:21.86947012 +0000 UTC m=+36.878046553" watchObservedRunningTime="2026-10-31 07:30:21.896217115 +0000 UTC m=+36.904793543" Oct 31 07:30:22.258288 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:22.258191 2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-mvmr5"] Oct 31 07:30:22.284037 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:22.284011 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mvmr5" Oct 31 07:30:22.284453 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:22.284429 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mvmr5"] Oct 31 07:30:22.288459 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:22.288436 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9wbww\"" Oct 31 07:30:22.288620 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:22.288603 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Oct 31 07:30:22.288684 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:22.288669 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Oct 31 07:30:22.349315 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:22.349285 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:22.420932 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:22.420903 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f13bc62a-1894-472c-95de-262afa399b38-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mvmr5\" (UID: \"f13bc62a-1894-472c-95de-262afa399b38\") " pod="openshift-insights/insights-runtime-extractor-mvmr5" Oct 31 07:30:22.421411 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:22.420941 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f13bc62a-1894-472c-95de-262afa399b38-crio-socket\") pod \"insights-runtime-extractor-mvmr5\" (UID: \"f13bc62a-1894-472c-95de-262afa399b38\") " pod="openshift-insights/insights-runtime-extractor-mvmr5" Oct 31 07:30:22.421411 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:22.420987 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhvbn\" (UniqueName: \"kubernetes.io/projected/f13bc62a-1894-472c-95de-262afa399b38-kube-api-access-qhvbn\") pod \"insights-runtime-extractor-mvmr5\" (UID: \"f13bc62a-1894-472c-95de-262afa399b38\") " pod="openshift-insights/insights-runtime-extractor-mvmr5" Oct 31 07:30:22.421411 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:22.421052 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f13bc62a-1894-472c-95de-262afa399b38-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mvmr5\" (UID: \"f13bc62a-1894-472c-95de-262afa399b38\") " pod="openshift-insights/insights-runtime-extractor-mvmr5" Oct 31 07:30:22.421411 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:22.421081 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f13bc62a-1894-472c-95de-262afa399b38-data-volume\") pod \"insights-runtime-extractor-mvmr5\" (UID: \"f13bc62a-1894-472c-95de-262afa399b38\") " pod="openshift-insights/insights-runtime-extractor-mvmr5" Oct 31 07:30:22.521741 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:22.521660 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f13bc62a-1894-472c-95de-262afa399b38-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mvmr5\" (UID: \"f13bc62a-1894-472c-95de-262afa399b38\") " pod="openshift-insights/insights-runtime-extractor-mvmr5" Oct 31 07:30:22.521741 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:22.521717 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f13bc62a-1894-472c-95de-262afa399b38-crio-socket\") pod \"insights-runtime-extractor-mvmr5\" (UID: \"f13bc62a-1894-472c-95de-262afa399b38\") " pod="openshift-insights/insights-runtime-extractor-mvmr5" Oct 31 07:30:22.521935 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:22.521787 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhvbn\" (UniqueName: \"kubernetes.io/projected/f13bc62a-1894-472c-95de-262afa399b38-kube-api-access-qhvbn\") pod \"insights-runtime-extractor-mvmr5\" (UID: \"f13bc62a-1894-472c-95de-262afa399b38\") " pod="openshift-insights/insights-runtime-extractor-mvmr5" Oct 31 07:30:22.521935 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:22.521892 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f13bc62a-1894-472c-95de-262afa399b38-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mvmr5\" (UID: \"f13bc62a-1894-472c-95de-262afa399b38\") " pod="openshift-insights/insights-runtime-extractor-mvmr5" Oct 31 07:30:22.522039 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:22.521939 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f13bc62a-1894-472c-95de-262afa399b38-data-volume\") pod \"insights-runtime-extractor-mvmr5\" (UID: \"f13bc62a-1894-472c-95de-262afa399b38\") " pod="openshift-insights/insights-runtime-extractor-mvmr5" Oct 31 07:30:22.523128 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:22.522294 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f13bc62a-1894-472c-95de-262afa399b38-data-volume\") pod \"insights-runtime-extractor-mvmr5\" (UID: \"f13bc62a-1894-472c-95de-262afa399b38\") " pod="openshift-insights/insights-runtime-extractor-mvmr5" Oct 31 07:30:22.523128 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:22.522413 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Oct 31 07:30:22.523128 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:22.522495 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f13bc62a-1894-472c-95de-262afa399b38-kube-rbac-proxy-cm podName:f13bc62a-1894-472c-95de-262afa399b38 nodeName:}" failed. No retries permitted until 2026-10-31 07:30:23.022473971 +0000 UTC m=+38.031050382 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f13bc62a-1894-472c-95de-262afa399b38-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mvmr5" (UID: "f13bc62a-1894-472c-95de-262afa399b38") : configmap "kube-rbac-proxy" not found Oct 31 07:30:22.523128 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:22.522759 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f13bc62a-1894-472c-95de-262afa399b38-crio-socket\") pod \"insights-runtime-extractor-mvmr5\" (UID: \"f13bc62a-1894-472c-95de-262afa399b38\") " pod="openshift-insights/insights-runtime-extractor-mvmr5" Oct 31 07:30:22.523128 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:22.523046 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 31 07:30:22.523128 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:22.523092 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f13bc62a-1894-472c-95de-262afa399b38-insights-runtime-extractor-tls podName:f13bc62a-1894-472c-95de-262afa399b38 nodeName:}" failed. No retries permitted until 2026-10-31 07:30:23.023077749 +0000 UTC m=+38.031654165 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f13bc62a-1894-472c-95de-262afa399b38-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mvmr5" (UID: "f13bc62a-1894-472c-95de-262afa399b38") : secret "insights-runtime-extractor-tls" not found Oct 31 07:30:22.552106 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:22.552054 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhvbn\" (UniqueName: \"kubernetes.io/projected/f13bc62a-1894-472c-95de-262afa399b38-kube-api-access-qhvbn\") pod \"insights-runtime-extractor-mvmr5\" (UID: \"f13bc62a-1894-472c-95de-262afa399b38\") " pod="openshift-insights/insights-runtime-extractor-mvmr5" Oct 31 07:30:22.572616 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:22.572550 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:22.608287 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:22.608257 2577 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-dwlls"] Oct 31 07:30:22.630147 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:22.630118 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-dwlls"] Oct 31 07:30:22.630299 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:22.630235 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-dwlls" Oct 31 07:30:22.716694 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:22.716659 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Oct 31 07:30:22.716694 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:22.716659 2577 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-kqb4j\"" Oct 31 07:30:22.716965 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:22.716673 2577 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Oct 31 07:30:22.723889 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:22.723862 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvpkt\" (UniqueName: \"kubernetes.io/projected/acda6b08-2faa-4836-888e-d4ecf324cb1d-kube-api-access-tvpkt\") pod \"migrator-66df5d496b-dwlls\" (UID: \"acda6b08-2faa-4836-888e-d4ecf324cb1d\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-dwlls" Oct 31 07:30:22.774153 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:22.774077 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:22.818830 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:22.818801 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jztcz_396e1c5e-3700-4fe6-b367-54ad2a244225/console-operator/1.log" Oct 31 07:30:22.819554 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:22.819211 2577 scope.go:122] "RemoveContainer" containerID="25aec100393d03aba086bd02f13db3038b24b6ae07422807f4c23edbb520f3f5" Oct 31 07:30:22.819554 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:22.819469 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-jztcz_openshift-console-operator(396e1c5e-3700-4fe6-b367-54ad2a244225)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-jztcz" podUID="396e1c5e-3700-4fe6-b367-54ad2a244225" Oct 31 07:30:22.821426 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:22.821396 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-4xtdb" event={"ID":"43c59587-4ff8-4546-9c0a-6161ece201d0","Type":"ContainerStarted","Data":"58a2c88d7d8e9d13ddeb9d97ef9d7d786c65a8d144c89cceb7ea6c26bcf1f053"} Oct 31 07:30:22.825088 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:22.825062 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvpkt\" (UniqueName: \"kubernetes.io/projected/acda6b08-2faa-4836-888e-d4ecf324cb1d-kube-api-access-tvpkt\") pod \"migrator-66df5d496b-dwlls\" (UID: \"acda6b08-2faa-4836-888e-d4ecf324cb1d\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-dwlls" Oct 31 07:30:22.837893 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:22.837873 2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvpkt\" (UniqueName: \"kubernetes.io/projected/acda6b08-2faa-4836-888e-d4ecf324cb1d-kube-api-access-tvpkt\") pod \"migrator-66df5d496b-dwlls\" (UID: \"acda6b08-2faa-4836-888e-d4ecf324cb1d\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-dwlls" Oct 31 07:30:22.856579 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:22.856535 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-22cmd" podStartSLOduration=2.261112432 podStartE2EDuration="7.856523766s" totalImagesPullingTime="5.595411334s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:30:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:30:22.856492437 +0000 UTC m=+37.865068890" watchObservedRunningTime="2026-10-31 07:30:22.856523766 +0000 UTC m=+37.865100194" Oct 31 07:30:22.956670 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:22.956637 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-dwlls" Oct 31 07:30:22.975744 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:22.975710 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:23.028957 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:23.028884 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f13bc62a-1894-472c-95de-262afa399b38-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mvmr5\" (UID: \"f13bc62a-1894-472c-95de-262afa399b38\") " pod="openshift-insights/insights-runtime-extractor-mvmr5" Oct 31 07:30:23.029102 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:23.028973 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f13bc62a-1894-472c-95de-262afa399b38-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mvmr5\" (UID: \"f13bc62a-1894-472c-95de-262afa399b38\") " pod="openshift-insights/insights-runtime-extractor-mvmr5" Oct 31 07:30:23.029102 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:23.029070 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 31 07:30:23.029214 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:23.029110 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Oct 31 07:30:23.029214 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:23.029138 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f13bc62a-1894-472c-95de-262afa399b38-insights-runtime-extractor-tls podName:f13bc62a-1894-472c-95de-262afa399b38 nodeName:}" failed. No retries permitted until 2026-10-31 07:30:24.029118334 +0000 UTC m=+39.037694757 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f13bc62a-1894-472c-95de-262afa399b38-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mvmr5" (UID: "f13bc62a-1894-472c-95de-262afa399b38") : secret "insights-runtime-extractor-tls" not found Oct 31 07:30:23.029214 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:23.029171 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f13bc62a-1894-472c-95de-262afa399b38-kube-rbac-proxy-cm podName:f13bc62a-1894-472c-95de-262afa399b38 nodeName:}" failed. No retries permitted until 2026-10-31 07:30:24.029153099 +0000 UTC m=+39.037729520 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f13bc62a-1894-472c-95de-262afa399b38-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mvmr5" (UID: "f13bc62a-1894-472c-95de-262afa399b38") : configmap "kube-rbac-proxy" not found Oct 31 07:30:23.129751 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:23.129720 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-86n8d\" (UID: \"d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-86n8d" Oct 31 07:30:23.129861 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:23.129759 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-registry-tls\") pod \"image-registry-54854c7897-pc7km\" (UID: \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\") " pod="openshift-image-registry/image-registry-54854c7897-pc7km" Oct 31 07:30:23.129861 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:23.129803 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5e743234-2b5f-4c91-9b06-189f546cbce4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-282kr\" (UID: \"5e743234-2b5f-4c91-9b06-189f546cbce4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-282kr" Oct 31 07:30:23.129964 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:23.129864 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/58bc3ea1-f58c-4891-a049-1289a99be055-metrics-tls\") pod \"dns-default-z5sfp\" (UID: \"58bc3ea1-f58c-4891-a049-1289a99be055\") " pod="openshift-dns/dns-default-z5sfp" Oct 31 07:30:23.129964 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:23.129891 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28be803a-a932-4009-ad61-b356366821ef-service-ca-bundle\") pod \"router-default-5549647ff-cxcdt\" (UID: \"28be803a-a932-4009-ad61-b356366821ef\") " pod="openshift-ingress/router-default-5549647ff-cxcdt" Oct 31 07:30:23.130028 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:23.129996 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/28be803a-a932-4009-ad61-b356366821ef-service-ca-bundle podName:28be803a-a932-4009-ad61-b356366821ef nodeName:}" failed. No retries permitted until 2026-10-31 07:30:31.129982315 +0000 UTC m=+46.138558721 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/28be803a-a932-4009-ad61-b356366821ef-service-ca-bundle") pod "router-default-5549647ff-cxcdt" (UID: "28be803a-a932-4009-ad61-b356366821ef") : configmap references non-existent config key: service-ca.crt Oct 31 07:30:23.130305 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:23.130289 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Oct 31 07:30:23.130345 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:23.130330 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e-samples-operator-tls podName:d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e nodeName:}" failed. No retries permitted until 2026-10-31 07:30:31.130320593 +0000 UTC m=+46.138896998 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-86n8d" (UID: "d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e") : secret "samples-operator-tls" not found Oct 31 07:30:23.130421 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:23.130397 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 31 07:30:23.130421 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:23.130408 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-54854c7897-pc7km: secret "image-registry-tls" not found Oct 31 07:30:23.130491 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:23.130448 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-registry-tls podName:c734ebf9-eba8-4e14-b20d-2d78dd65b10d nodeName:}" failed. No retries permitted until 2026-10-31 07:30:31.130437881 +0000 UTC m=+46.139014302 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-registry-tls") pod "image-registry-54854c7897-pc7km" (UID: "c734ebf9-eba8-4e14-b20d-2d78dd65b10d") : secret "image-registry-tls" not found Oct 31 07:30:23.130491 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:23.130459 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 31 07:30:23.130491 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:23.130456 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 31 07:30:23.130491 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:23.130490 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e743234-2b5f-4c91-9b06-189f546cbce4-cluster-monitoring-operator-tls podName:5e743234-2b5f-4c91-9b06-189f546cbce4 nodeName:}" failed. No retries permitted until 2026-10-31 07:30:31.130482824 +0000 UTC m=+46.139059243 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5e743234-2b5f-4c91-9b06-189f546cbce4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-282kr" (UID: "5e743234-2b5f-4c91-9b06-189f546cbce4") : secret "cluster-monitoring-operator-tls" not found Oct 31 07:30:23.130658 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:23.130516 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58bc3ea1-f58c-4891-a049-1289a99be055-metrics-tls podName:58bc3ea1-f58c-4891-a049-1289a99be055 nodeName:}" failed. No retries permitted until 2026-10-31 07:30:31.130498902 +0000 UTC m=+46.139075326 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/58bc3ea1-f58c-4891-a049-1289a99be055-metrics-tls") pod "dns-default-z5sfp" (UID: "58bc3ea1-f58c-4891-a049-1289a99be055") : secret "dns-default-metrics-tls" not found Oct 31 07:30:23.230589 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:23.230558 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/563d3b8b-0040-439e-b0b2-e858eed5abf6-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-s75pm\" (UID: \"563d3b8b-0040-439e-b0b2-e858eed5abf6\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-s75pm" Oct 31 07:30:23.230589 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:23.230601 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28be803a-a932-4009-ad61-b356366821ef-metrics-certs\") pod \"router-default-5549647ff-cxcdt\" (UID: \"28be803a-a932-4009-ad61-b356366821ef\") " pod="openshift-ingress/router-default-5549647ff-cxcdt" Oct 31 07:30:23.230815 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:23.230625 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4104ba06-fba8-4c3c-b876-cca5087a7d0a-registry-tls\") pod \"image-registry-69fbc98d44-fn7fh\" (UID: \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\") " pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" Oct 31 07:30:23.230815 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:23.230716 2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Oct 31 07:30:23.230815 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:23.230738 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Oct 31 07:30:23.230815 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:23.230775 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/563d3b8b-0040-439e-b0b2-e858eed5abf6-networking-console-plugin-cert podName:563d3b8b-0040-439e-b0b2-e858eed5abf6 nodeName:}" failed. No retries permitted until 2026-10-31 07:30:31.230758847 +0000 UTC m=+46.239335273 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/563d3b8b-0040-439e-b0b2-e858eed5abf6-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-s75pm" (UID: "563d3b8b-0040-439e-b0b2-e858eed5abf6") : secret "networking-console-plugin-cert" not found Oct 31 07:30:23.230815 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:23.230793 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 31 07:30:23.230815 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:23.230810 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69fbc98d44-fn7fh: secret "image-registry-tls" not found Oct 31 07:30:23.230815 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:23.230816 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28be803a-a932-4009-ad61-b356366821ef-metrics-certs podName:28be803a-a932-4009-ad61-b356366821ef nodeName:}" failed. No retries permitted until 2026-10-31 07:30:31.230806332 +0000 UTC m=+46.239382741 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/28be803a-a932-4009-ad61-b356366821ef-metrics-certs") pod "router-default-5549647ff-cxcdt" (UID: "28be803a-a932-4009-ad61-b356366821ef") : secret "router-metrics-certs-default" not found Oct 31 07:30:23.231080 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:23.230849 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4104ba06-fba8-4c3c-b876-cca5087a7d0a-registry-tls podName:4104ba06-fba8-4c3c-b876-cca5087a7d0a nodeName:}" failed. No retries permitted until 2026-10-31 07:30:31.230836847 +0000 UTC m=+46.239413253 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4104ba06-fba8-4c3c-b876-cca5087a7d0a-registry-tls") pod "image-registry-69fbc98d44-fn7fh" (UID: "4104ba06-fba8-4c3c-b876-cca5087a7d0a") : secret "image-registry-tls" not found Oct 31 07:30:23.330569 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:23.330517 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-4xtdb" podStartSLOduration=366.234751854 podStartE2EDuration="6m12.330498622s" totalImagesPullingTime="6.095746768s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:24:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:30:22.876286803 +0000 UTC m=+37.884863234" watchObservedRunningTime="2026-10-31 07:30:23.330498622 +0000 UTC m=+38.339075050" Oct 31 07:30:23.330701 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:23.330682 2577 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-dwlls"] Oct 31 07:30:23.333249 ip-10-0-130-48 kubenswrapper[2577]: W1031 07:30:23.333218 2577 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacda6b08_2faa_4836_888e_d4ecf324cb1d.slice/crio-7c6136cafd20d7dc5483470c86e6c4a5d40582798db219bed088e3a4430e6d60 WatchSource:0}: Error finding container 7c6136cafd20d7dc5483470c86e6c4a5d40582798db219bed088e3a4430e6d60: Status 404 returned error can't find the container with id 7c6136cafd20d7dc5483470c86e6c4a5d40582798db219bed088e3a4430e6d60 Oct 31 07:30:23.791118 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:23.791085 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:23.826899 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:23.826864 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-ncnd7" event={"ID":"d8b28b9f-37a2-4859-87fd-d27bb37d832b","Type":"ContainerStarted","Data":"2a7a9d203e3b11a2056a3a50296127d1b521934f5fc9edd5fca721b3f0f70325"} Oct 31 07:30:23.827037 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:23.826959 2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-ncnd7" Oct 31 07:30:23.828237 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:23.828203 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-dwlls" event={"ID":"acda6b08-2faa-4836-888e-d4ecf324cb1d","Type":"ContainerStarted","Data":"7c6136cafd20d7dc5483470c86e6c4a5d40582798db219bed088e3a4430e6d60"} Oct 31 07:30:23.844105 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:23.844051 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-ncnd7" podStartSLOduration=35.698723061 podStartE2EDuration="38.844037361s" totalImagesPullingTime="3.1453143s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:29:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:30:23.843835538 +0000 UTC m=+38.852411966" watchObservedRunningTime="2026-10-31 07:30:23.844037361 +0000 UTC m=+38.852613791" Oct 31 07:30:24.038262 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:24.037930 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f13bc62a-1894-472c-95de-262afa399b38-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mvmr5\" (UID: \"f13bc62a-1894-472c-95de-262afa399b38\") " pod="openshift-insights/insights-runtime-extractor-mvmr5" Oct 31 07:30:24.038262 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:24.038044 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f13bc62a-1894-472c-95de-262afa399b38-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mvmr5\" (UID: \"f13bc62a-1894-472c-95de-262afa399b38\") " pod="openshift-insights/insights-runtime-extractor-mvmr5" Oct 31 07:30:24.038262 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:24.038063 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 31 07:30:24.038262 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:24.038140 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f13bc62a-1894-472c-95de-262afa399b38-insights-runtime-extractor-tls podName:f13bc62a-1894-472c-95de-262afa399b38 nodeName:}" failed. No retries permitted until 2026-10-31 07:30:26.038119204 +0000 UTC m=+41.046695617 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f13bc62a-1894-472c-95de-262afa399b38-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mvmr5" (UID: "f13bc62a-1894-472c-95de-262afa399b38") : secret "insights-runtime-extractor-tls" not found Oct 31 07:30:24.038262 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:24.038172 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Oct 31 07:30:24.038262 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:24.038229 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f13bc62a-1894-472c-95de-262afa399b38-kube-rbac-proxy-cm podName:f13bc62a-1894-472c-95de-262afa399b38 nodeName:}" failed. No retries permitted until 2026-10-31 07:30:26.038210543 +0000 UTC m=+41.046786971 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f13bc62a-1894-472c-95de-262afa399b38-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mvmr5" (UID: "f13bc62a-1894-472c-95de-262afa399b38") : configmap "kube-rbac-proxy" not found Oct 31 07:30:25.691736 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:25.691706 2577 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-jztcz" Oct 31 07:30:25.692092 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:25.691755 2577 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-jztcz" Oct 31 07:30:25.692092 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:25.692030 2577 scope.go:122] "RemoveContainer" containerID="25aec100393d03aba086bd02f13db3038b24b6ae07422807f4c23edbb520f3f5" Oct 31 07:30:25.692203 ip-10-0-130-48 kubenswrapper[2577]: E1031 07:30:25.692184 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-jztcz_openshift-console-operator(396e1c5e-3700-4fe6-b367-54ad2a244225)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-jztcz" podUID="396e1c5e-3700-4fe6-b367-54ad2a244225" Oct 31 07:30:25.837412 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:25.837383 2577 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-dwlls" event={"ID":"acda6b08-2faa-4836-888e-d4ecf324cb1d","Type":"ContainerStarted","Data":"d951356e4e4cd112b92eb121e1ed0d469a49a44461fd3cd44f9b7364317423a7"} Oct 31 07:30:25.856450 ip-10-0-130-48 kubenswrapper[2577]: I1031 07:30:25.856404 2577 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-dwlls" podStartSLOduration=2.252607889 podStartE2EDuration="3.856389337s" totalImagesPullingTime="1.603781448s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:30:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:30:25.855528722 +0000 UTC m=+40.864105173" watchObservedRunningTime="2026-10-31 07:30:25.856389337 +0000 UTC m=+40.864965765" Oct 31 07:30:26.056319 ip-10-0-130-48 systemd[1]: Stopping Kubernetes Kubelet... Oct 31 07:30:26.074733 ip-10-0-130-48 systemd[1]: kubelet.service: Deactivated successfully. Oct 31 07:30:26.074901 ip-10-0-130-48 systemd[1]: Stopped Kubernetes Kubelet. Oct 31 07:30:26.074947 ip-10-0-130-48 systemd[1]: kubelet.service: Consumed 2.902s CPU time. Oct 31 07:30:26.086806 ip-10-0-130-48 systemd[1]: Starting Kubernetes Kubelet... Oct 31 07:30:26.157151 ip-10-0-130-48 kubenswrapper[4935]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 31 07:30:26.157151 ip-10-0-130-48 kubenswrapper[4935]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Oct 31 07:30:26.157151 ip-10-0-130-48 kubenswrapper[4935]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 31 07:30:26.157151 ip-10-0-130-48 kubenswrapper[4935]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 31 07:30:26.159249 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159164 4935 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 31 07:30:26.159249 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159248 4935 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 31 07:30:26.159249 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159251 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 31 07:30:26.159249 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159254 4935 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 31 07:30:26.159414 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159258 4935 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 31 07:30:26.159414 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159260 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 31 07:30:26.159414 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159263 4935 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 31 07:30:26.159414 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159265 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 31 07:30:26.159414 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159268 4935 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 31 07:30:26.159414 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159271 4935 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 31 07:30:26.159414 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159274 4935 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 31 07:30:26.159414 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159276 4935 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 31 07:30:26.159414 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159279 4935 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 31 07:30:26.159414 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159282 4935 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 31 07:30:26.159414 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159285 4935 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 31 07:30:26.159414 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159289 4935 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 31 07:30:26.159414 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159292 4935 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 31 07:30:26.159414 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159295 4935 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 31 07:30:26.159414 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159298 4935 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 31 07:30:26.159414 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159300 4935 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 31 07:30:26.159414 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159303 4935 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 31 07:30:26.159414 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159305 4935 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 31 07:30:26.159414 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159308 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 31 07:30:26.159414 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159310 4935 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 31 07:30:26.160011 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159312 4935 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 31 07:30:26.160011 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159315 4935 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 31 07:30:26.160011 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159317 4935 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 31 07:30:26.160011 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159320 4935 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 31 07:30:26.160011 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159322 4935 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 31 07:30:26.160011 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159324 4935 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 31 07:30:26.160011 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159327 4935 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 31 07:30:26.160011 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159329 4935 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 31 07:30:26.160011 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159333 4935 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 31 07:30:26.160011 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159335 4935 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 31 07:30:26.160011 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159338 4935 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 31 07:30:26.160011 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159340 4935 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 31 07:30:26.160011 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159343 4935 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 31 07:30:26.160011 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159366 4935 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 31 07:30:26.160011 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159370 4935 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 31 07:30:26.160011 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159374 4935 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 31 07:30:26.160011 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159378 4935 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 31 07:30:26.160011 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159382 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 31 07:30:26.160011 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159385 4935 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 31 07:30:26.160011 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159388 4935 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 31 07:30:26.160620 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159391 4935 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 31 07:30:26.160620 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159393 4935 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 31 07:30:26.160620 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159396 4935 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 31 07:30:26.160620 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159399 4935 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 31 07:30:26.160620 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159401 4935 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 31 07:30:26.160620 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159404 4935 feature_gate.go:386] unrecognized feature gate: Example2 Oct 31 07:30:26.160620 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159407 4935 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 31 07:30:26.160620 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159410 4935 feature_gate.go:386] unrecognized feature gate: EVPN Oct 31 07:30:26.160620 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159412 4935 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 31 07:30:26.160620 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159416 4935 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 31 07:30:26.160620 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159418 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 31 07:30:26.160620 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159421 4935 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 31 07:30:26.160620 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159424 4935 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 31 07:30:26.160620 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159427 4935 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 31 07:30:26.160620 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159429 4935 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 31 07:30:26.160620 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159434 4935 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 31 07:30:26.160620 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159439 4935 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 31 07:30:26.160620 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159442 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 31 07:30:26.160620 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159445 4935 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 31 07:30:26.160620 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159447 4935 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 31 07:30:26.161126 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159450 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 31 07:30:26.161126 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159452 4935 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 31 07:30:26.161126 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159455 4935 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 31 07:30:26.161126 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159457 4935 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 31 07:30:26.161126 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159460 4935 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 31 07:30:26.161126 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159462 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 31 07:30:26.161126 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159465 4935 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 31 07:30:26.161126 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159468 4935 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 31 07:30:26.161126 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159470 4935 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 31 07:30:26.161126 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159473 4935 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 31 07:30:26.161126 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159475 4935 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 31 07:30:26.161126 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159477 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 31 07:30:26.161126 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159480 4935 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 31 07:30:26.161126 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159482 4935 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 31 07:30:26.161126 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159486 4935 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 31 07:30:26.161126 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159489 4935 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 31 07:30:26.161126 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159491 4935 feature_gate.go:386] unrecognized feature gate: Example Oct 31 07:30:26.161126 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159494 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 31 07:30:26.161126 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159498 4935 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 31 07:30:26.161643 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159501 4935 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 31 07:30:26.161643 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159503 4935 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 31 07:30:26.161643 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159506 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 31 07:30:26.161643 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159508 4935 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 31 07:30:26.161643 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159511 4935 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 31 07:30:26.161643 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159513 4935 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 31 07:30:26.161643 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159515 4935 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 31 07:30:26.161643 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159518 4935 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 31 07:30:26.161643 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159520 4935 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 31 07:30:26.161643 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159522 4935 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 31 07:30:26.161643 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159526 4935 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 31 07:30:26.161643 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159531 4935 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 31 07:30:26.161643 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159534 4935 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 31 07:30:26.161643 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159537 4935 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 31 07:30:26.161643 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159539 4935 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 31 07:30:26.161643 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159542 4935 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 31 07:30:26.161643 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159544 4935 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 31 07:30:26.161643 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159547 4935 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 31 07:30:26.161643 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159549 4935 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 31 07:30:26.162137 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159551 4935 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 31 07:30:26.162137 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159554 4935 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 31 07:30:26.162137 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159557 4935 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 31 07:30:26.162137 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.159560 4935 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 31 07:30:26.162137 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160336 4935 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 31 07:30:26.162137 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160343 4935 feature_gate.go:386] unrecognized feature gate: Example2 Oct 31 07:30:26.162137 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160364 4935 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 31 07:30:26.162137 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160368 4935 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 31 07:30:26.162137 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160372 4935 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 31 07:30:26.162137 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160377 4935 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 31 07:30:26.162137 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160381 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 31 07:30:26.162137 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160383 4935 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 31 07:30:26.162137 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160388 4935 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 31 07:30:26.162137 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160393 4935 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 31 07:30:26.162137 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160397 4935 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 31 07:30:26.162137 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160401 4935 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 31 07:30:26.162137 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160406 4935 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 31 07:30:26.162137 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160410 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 31 07:30:26.162137 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160414 4935 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 31 07:30:26.162137 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160416 4935 feature_gate.go:386] unrecognized feature gate: EVPN Oct 31 07:30:26.162719 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160419 4935 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 31 07:30:26.162719 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160421 4935 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 31 07:30:26.162719 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160424 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 31 07:30:26.162719 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160426 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 31 07:30:26.162719 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160429 4935 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 31 07:30:26.162719 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160432 4935 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 31 07:30:26.162719 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160434 4935 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 31 07:30:26.162719 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160437 4935 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 31 07:30:26.162719 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160440 4935 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 31 07:30:26.162719 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160442 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 31 07:30:26.162719 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160444 4935 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 31 07:30:26.162719 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160447 4935 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 31 07:30:26.162719 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160449 4935 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 31 07:30:26.162719 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160452 4935 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 31 07:30:26.162719 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160454 4935 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 31 07:30:26.162719 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160457 4935 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 31 07:30:26.162719 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160459 4935 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 31 07:30:26.162719 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160461 4935 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 31 07:30:26.162719 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160464 4935 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 31 07:30:26.163197 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160467 4935 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 31 07:30:26.163197 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160469 4935 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 31 07:30:26.163197 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160472 4935 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 31 07:30:26.163197 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160474 4935 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 31 07:30:26.163197 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160477 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 31 07:30:26.163197 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160480 4935 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 31 07:30:26.163197 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160483 4935 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 31 07:30:26.163197 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160486 4935 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 31 07:30:26.163197 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160488 4935 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 31 07:30:26.163197 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160491 4935 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 31 07:30:26.163197 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160493 4935 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 31 07:30:26.163197 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160496 4935 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 31 07:30:26.163197 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160499 4935 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 31 07:30:26.163197 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160501 4935 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 31 07:30:26.163197 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160503 4935 feature_gate.go:386] unrecognized feature gate: Example Oct 31 07:30:26.163197 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160506 4935 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 31 07:30:26.163197 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160508 4935 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 31 07:30:26.163197 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160510 4935 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 31 07:30:26.163197 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160513 4935 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 31 07:30:26.163197 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160516 4935 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 31 07:30:26.163728 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160518 4935 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 31 07:30:26.163728 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160520 4935 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 31 07:30:26.163728 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160523 4935 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 31 07:30:26.163728 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160525 4935 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 31 07:30:26.163728 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160528 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 31 07:30:26.163728 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160531 4935 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 31 07:30:26.163728 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160533 4935 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 31 07:30:26.163728 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160535 4935 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 31 07:30:26.163728 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160538 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 31 07:30:26.163728 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160541 4935 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 31 07:30:26.163728 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160544 4935 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 31 07:30:26.163728 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160546 4935 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 31 07:30:26.163728 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160549 4935 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 31 07:30:26.163728 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160552 4935 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 31 07:30:26.163728 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160556 4935 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 31 07:30:26.163728 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160559 4935 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 31 07:30:26.163728 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160562 4935 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 31 07:30:26.163728 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160565 4935 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 31 07:30:26.163728 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160568 4935 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 31 07:30:26.164193 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160571 4935 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 31 07:30:26.164193 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160574 4935 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 31 07:30:26.164193 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160576 4935 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 31 07:30:26.164193 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160578 4935 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 31 07:30:26.164193 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160581 4935 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 31 07:30:26.164193 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160583 4935 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 31 07:30:26.164193 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160586 4935 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 31 07:30:26.164193 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160588 4935 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 31 07:30:26.164193 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160592 4935 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 31 07:30:26.164193 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160596 4935 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 31 07:30:26.164193 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160599 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 31 07:30:26.164193 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160601 4935 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 31 07:30:26.164193 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160604 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 31 07:30:26.164193 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160607 4935 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 31 07:30:26.164193 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160610 4935 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 31 07:30:26.164193 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160612 4935 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 31 07:30:26.164193 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160615 4935 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 31 07:30:26.164193 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160618 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 31 07:30:26.164193 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160621 4935 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 31 07:30:26.164690 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160624 4935 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 31 07:30:26.164690 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160627 4935 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 31 07:30:26.164690 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160629 4935 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 31 07:30:26.164690 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160632 4935 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 31 07:30:26.164690 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160635 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 31 07:30:26.164690 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160638 4935 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 31 07:30:26.164690 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160640 4935 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 31 07:30:26.164690 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160643 4935 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 31 07:30:26.164690 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160645 4935 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 31 07:30:26.164690 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160648 4935 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 31 07:30:26.164690 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160650 4935 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 31 07:30:26.164690 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160653 4935 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 31 07:30:26.164690 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.160655 4935 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 31 07:30:26.164690 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161150 4935 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 31 07:30:26.164690 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161156 4935 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 31 07:30:26.164690 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161159 4935 feature_gate.go:386] unrecognized feature gate: EVPN Oct 31 07:30:26.164690 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161162 4935 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 31 07:30:26.164690 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161165 4935 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 31 07:30:26.164690 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161167 4935 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 31 07:30:26.164690 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161170 4935 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 31 07:30:26.165187 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161173 4935 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 31 07:30:26.165187 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161177 4935 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 31 07:30:26.165187 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161180 4935 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 31 07:30:26.165187 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161183 4935 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 31 07:30:26.165187 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161186 4935 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 31 07:30:26.165187 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161189 4935 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 31 07:30:26.165187 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161191 4935 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 31 07:30:26.165187 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161193 4935 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 31 07:30:26.165187 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161196 4935 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 31 07:30:26.165187 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161199 4935 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 31 07:30:26.165187 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161202 4935 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 31 07:30:26.165187 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161205 4935 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 31 07:30:26.165187 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161207 4935 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 31 07:30:26.165187 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161210 4935 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 31 07:30:26.165187 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161212 4935 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 31 07:30:26.165187 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161215 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 31 07:30:26.165187 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161217 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 31 07:30:26.165187 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161220 4935 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 31 07:30:26.165187 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161222 4935 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 31 07:30:26.165705 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161225 4935 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 31 07:30:26.165705 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161227 4935 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 31 07:30:26.165705 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161230 4935 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 31 07:30:26.165705 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161232 4935 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 31 07:30:26.165705 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161234 4935 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 31 07:30:26.165705 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161237 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 31 07:30:26.165705 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161239 4935 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 31 07:30:26.165705 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161242 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 31 07:30:26.165705 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161246 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 31 07:30:26.165705 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161249 4935 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 31 07:30:26.165705 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161251 4935 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 31 07:30:26.165705 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161254 4935 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 31 07:30:26.165705 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161257 4935 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 31 07:30:26.165705 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161259 4935 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 31 07:30:26.165705 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161262 4935 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 31 07:30:26.165705 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161264 4935 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 31 07:30:26.165705 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161267 4935 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 31 07:30:26.165705 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161270 4935 feature_gate.go:386] unrecognized feature gate: Example Oct 31 07:30:26.165705 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161272 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 31 07:30:26.166841 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161274 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 31 07:30:26.166841 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161277 4935 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 31 07:30:26.166841 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161279 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 31 07:30:26.166841 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161282 4935 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 31 07:30:26.166841 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161284 4935 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 31 07:30:26.166841 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161286 4935 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 31 07:30:26.166841 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161289 4935 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 31 07:30:26.166841 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161291 4935 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 31 07:30:26.166841 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161293 4935 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 31 07:30:26.166841 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161296 4935 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 31 07:30:26.166841 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161298 4935 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 31 07:30:26.166841 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161301 4935 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 31 07:30:26.166841 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161303 4935 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 31 07:30:26.166841 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161306 4935 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 31 07:30:26.166841 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161308 4935 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 31 07:30:26.166841 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161311 4935 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 31 07:30:26.166841 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161313 4935 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 31 07:30:26.166841 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161315 4935 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 31 07:30:26.166841 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161320 4935 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 31 07:30:26.167336 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161324 4935 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 31 07:30:26.167336 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161328 4935 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 31 07:30:26.167336 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161331 4935 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 31 07:30:26.167336 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161334 4935 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 31 07:30:26.167336 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161337 4935 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 31 07:30:26.167336 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161339 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 31 07:30:26.167336 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161342 4935 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 31 07:30:26.167336 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161344 4935 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 31 07:30:26.167336 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161365 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 31 07:30:26.167336 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161368 4935 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 31 07:30:26.167336 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161371 4935 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 31 07:30:26.167336 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161373 4935 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 31 07:30:26.167336 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161375 4935 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 31 07:30:26.167336 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161378 4935 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 31 07:30:26.167336 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161380 4935 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 31 07:30:26.167336 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161383 4935 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 31 07:30:26.167336 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161385 4935 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 31 07:30:26.167336 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161388 4935 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 31 07:30:26.167336 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161391 4935 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 31 07:30:26.167336 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161393 4935 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 31 07:30:26.167885 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161396 4935 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 31 07:30:26.167885 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161398 4935 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 31 07:30:26.167885 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161400 4935 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 31 07:30:26.167885 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161403 4935 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 31 07:30:26.167885 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161405 4935 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 31 07:30:26.167885 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161408 4935 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 31 07:30:26.167885 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161410 4935 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 31 07:30:26.167885 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161413 4935 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 31 07:30:26.167885 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161415 4935 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 31 07:30:26.167885 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161418 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 31 07:30:26.167885 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161421 4935 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 31 07:30:26.167885 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161423 4935 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 31 07:30:26.167885 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161426 4935 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 31 07:30:26.167885 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161428 4935 feature_gate.go:386] unrecognized feature gate: Example2 Oct 31 07:30:26.167885 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161431 4935 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 31 07:30:26.167885 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161433 4935 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 31 07:30:26.167885 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161437 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 31 07:30:26.167885 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161441 4935 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 31 07:30:26.167885 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161443 4935 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 31 07:30:26.167885 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161446 4935 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 31 07:30:26.168404 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161448 4935 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 31 07:30:26.168404 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.161450 4935 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 31 07:30:26.168404 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161533 4935 flags.go:64] FLAG: --address="0.0.0.0" Oct 31 07:30:26.168404 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161540 4935 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Oct 31 07:30:26.168404 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161546 4935 flags.go:64] FLAG: --anonymous-auth="true" Oct 31 07:30:26.168404 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161550 4935 flags.go:64] FLAG: --application-metrics-count-limit="100" Oct 31 07:30:26.168404 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161554 4935 flags.go:64] FLAG: --authentication-token-webhook="false" Oct 31 07:30:26.168404 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161557 4935 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Oct 31 07:30:26.168404 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161562 4935 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Oct 31 07:30:26.168404 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161569 4935 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Oct 31 07:30:26.168404 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161572 4935 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Oct 31 07:30:26.168404 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161575 4935 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Oct 31 07:30:26.168404 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161578 4935 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Oct 31 07:30:26.168404 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161581 4935 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Oct 31 07:30:26.168404 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161584 4935 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Oct 31 07:30:26.168404 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161587 4935 flags.go:64] FLAG: --cgroup-root="" Oct 31 07:30:26.168404 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161589 4935 flags.go:64] FLAG: --cgroups-per-qos="true" Oct 31 07:30:26.168404 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161593 4935 flags.go:64] FLAG: --client-ca-file="" Oct 31 07:30:26.168404 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161595 4935 flags.go:64] FLAG: --cloud-provider="external" Oct 31 07:30:26.168404 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161598 4935 flags.go:64] FLAG: --cluster-dns="[]" Oct 31 07:30:26.168404 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161602 4935 flags.go:64] FLAG: --cluster-domain="" Oct 31 07:30:26.168404 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161604 4935 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Oct 31 07:30:26.168404 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161608 4935 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Oct 31 07:30:26.168404 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161611 4935 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Oct 31 07:30:26.169130 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161614 4935 flags.go:64] FLAG: --container-log-max-files="5" Oct 31 07:30:26.169130 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161618 4935 flags.go:64] FLAG: --container-log-max-size="10Mi" Oct 31 07:30:26.169130 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161620 4935 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Oct 31 07:30:26.169130 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161623 4935 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Oct 31 07:30:26.169130 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161626 4935 flags.go:64] FLAG: --containerd-namespace="k8s.io" Oct 31 07:30:26.169130 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161630 4935 flags.go:64] FLAG: --contention-profiling="false" Oct 31 07:30:26.169130 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161633 4935 flags.go:64] FLAG: --cpu-cfs-quota="true" Oct 31 07:30:26.169130 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161635 4935 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Oct 31 07:30:26.169130 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161638 4935 flags.go:64] FLAG: --cpu-manager-policy="none" Oct 31 07:30:26.169130 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161642 4935 flags.go:64] FLAG: --cpu-manager-policy-options="" Oct 31 07:30:26.169130 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161647 4935 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Oct 31 07:30:26.169130 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161650 4935 flags.go:64] FLAG: --enable-controller-attach-detach="true" Oct 31 07:30:26.169130 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161653 4935 flags.go:64] FLAG: --enable-debugging-handlers="true" Oct 31 07:30:26.169130 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161656 4935 flags.go:64] FLAG: --enable-load-reader="false" Oct 31 07:30:26.169130 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161659 4935 flags.go:64] FLAG: --enable-server="true" Oct 31 07:30:26.169130 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161662 4935 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Oct 31 07:30:26.169130 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161667 4935 flags.go:64] FLAG: --event-burst="100" Oct 31 07:30:26.169130 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161671 4935 flags.go:64] FLAG: --event-qps="50" Oct 31 07:30:26.169130 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161674 4935 flags.go:64] FLAG: --event-storage-age-limit="default=0" Oct 31 07:30:26.169130 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161677 4935 flags.go:64] FLAG: --event-storage-event-limit="default=0" Oct 31 07:30:26.169130 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161680 4935 flags.go:64] FLAG: --eviction-hard="" Oct 31 07:30:26.169130 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161684 4935 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Oct 31 07:30:26.169130 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161687 4935 flags.go:64] FLAG: --eviction-minimum-reclaim="" Oct 31 07:30:26.169130 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161690 4935 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Oct 31 07:30:26.169130 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161692 4935 flags.go:64] FLAG: --eviction-soft="" Oct 31 07:30:26.169766 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161695 4935 flags.go:64] FLAG: --eviction-soft-grace-period="" Oct 31 07:30:26.169766 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161698 4935 flags.go:64] FLAG: --exit-on-lock-contention="false" Oct 31 07:30:26.169766 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161701 4935 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Oct 31 07:30:26.169766 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161704 4935 flags.go:64] FLAG: --experimental-mounter-path="" Oct 31 07:30:26.169766 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161706 4935 flags.go:64] FLAG: --fail-cgroupv1="true" Oct 31 07:30:26.169766 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161709 4935 flags.go:64] FLAG: --fail-swap-on="true" Oct 31 07:30:26.169766 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161712 4935 flags.go:64] FLAG: --feature-gates="" Oct 31 07:30:26.169766 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161716 4935 flags.go:64] FLAG: --file-check-frequency="20s" Oct 31 07:30:26.169766 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161719 4935 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Oct 31 07:30:26.169766 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161722 4935 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Oct 31 07:30:26.169766 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161725 4935 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Oct 31 07:30:26.169766 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161727 4935 flags.go:64] FLAG: --healthz-port="10248" Oct 31 07:30:26.169766 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161730 4935 flags.go:64] FLAG: --help="false" Oct 31 07:30:26.169766 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161734 4935 flags.go:64] FLAG: --hostname-override="ip-10-0-130-48.us-east-2.compute.internal" Oct 31 07:30:26.169766 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161737 4935 flags.go:64] FLAG: --housekeeping-interval="10s" Oct 31 07:30:26.169766 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161740 4935 flags.go:64] FLAG: --http-check-frequency="20s" Oct 31 07:30:26.169766 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161743 4935 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Oct 31 07:30:26.169766 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161747 4935 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Oct 31 07:30:26.169766 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161750 4935 flags.go:64] FLAG: --image-gc-high-threshold="85" Oct 31 07:30:26.169766 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161753 4935 flags.go:64] FLAG: --image-gc-low-threshold="80" Oct 31 07:30:26.169766 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161755 4935 flags.go:64] FLAG: --image-service-endpoint="" Oct 31 07:30:26.169766 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161758 4935 flags.go:64] FLAG: --kernel-memcg-notification="false" Oct 31 07:30:26.169766 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161761 4935 flags.go:64] FLAG: --kube-api-burst="100" Oct 31 07:30:26.170400 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161764 4935 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Oct 31 07:30:26.170400 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161770 4935 flags.go:64] FLAG: --kube-api-qps="50" Oct 31 07:30:26.170400 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161772 4935 flags.go:64] FLAG: --kube-reserved="" Oct 31 07:30:26.170400 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161775 4935 flags.go:64] FLAG: --kube-reserved-cgroup="" Oct 31 07:30:26.170400 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161788 4935 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Oct 31 07:30:26.170400 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161791 4935 flags.go:64] FLAG: --kubelet-cgroups="" Oct 31 07:30:26.170400 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161794 4935 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Oct 31 07:30:26.170400 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161797 4935 flags.go:64] FLAG: --lock-file="" Oct 31 07:30:26.170400 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161800 4935 flags.go:64] FLAG: --log-cadvisor-usage="false" Oct 31 07:30:26.170400 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161802 4935 flags.go:64] FLAG: --log-flush-frequency="5s" Oct 31 07:30:26.170400 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161805 4935 flags.go:64] FLAG: --log-json-info-buffer-size="0" Oct 31 07:30:26.170400 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161810 4935 flags.go:64] FLAG: --log-json-split-stream="false" Oct 31 07:30:26.170400 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161813 4935 flags.go:64] FLAG: --log-text-info-buffer-size="0" Oct 31 07:30:26.170400 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161816 4935 flags.go:64] FLAG: --log-text-split-stream="false" Oct 31 07:30:26.170400 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161819 4935 flags.go:64] FLAG: --logging-format="text" Oct 31 07:30:26.170400 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161822 4935 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Oct 31 07:30:26.170400 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161825 4935 flags.go:64] FLAG: --make-iptables-util-chains="true" Oct 31 07:30:26.170400 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161827 4935 flags.go:64] FLAG: --manifest-url="" Oct 31 07:30:26.170400 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161830 4935 flags.go:64] FLAG: --manifest-url-header="" Oct 31 07:30:26.170400 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161834 4935 flags.go:64] FLAG: --max-housekeeping-interval="15s" Oct 31 07:30:26.170400 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161837 4935 flags.go:64] FLAG: --max-open-files="1000000" Oct 31 07:30:26.170400 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161841 4935 flags.go:64] FLAG: --max-pods="110" Oct 31 07:30:26.170400 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161844 4935 flags.go:64] FLAG: --maximum-dead-containers="-1" Oct 31 07:30:26.170400 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161847 4935 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Oct 31 07:30:26.170400 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161850 4935 flags.go:64] FLAG: --memory-manager-policy="None" Oct 31 07:30:26.171018 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161854 4935 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Oct 31 07:30:26.171018 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161857 4935 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Oct 31 07:30:26.171018 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161860 4935 flags.go:64] FLAG: --node-ip="0.0.0.0" Oct 31 07:30:26.171018 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161863 4935 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Oct 31 07:30:26.171018 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161869 4935 flags.go:64] FLAG: --node-status-max-images="50" Oct 31 07:30:26.171018 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161872 4935 flags.go:64] FLAG: --node-status-update-frequency="10s" Oct 31 07:30:26.171018 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161875 4935 flags.go:64] FLAG: --oom-score-adj="-999" Oct 31 07:30:26.171018 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161878 4935 flags.go:64] FLAG: --pod-cidr="" Oct 31 07:30:26.171018 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161882 4935 flags.go:64] FLAG: --pod-manifest-path="" Oct 31 07:30:26.171018 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161885 4935 flags.go:64] FLAG: --pod-max-pids="-1" Oct 31 07:30:26.171018 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161888 4935 flags.go:64] FLAG: --pods-per-core="0" Oct 31 07:30:26.171018 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161890 4935 flags.go:64] FLAG: --port="10250" Oct 31 07:30:26.171018 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161893 4935 flags.go:64] FLAG: --protect-kernel-defaults="false" Oct 31 07:30:26.171018 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161896 4935 flags.go:64] FLAG: --provider-id="aws:///us-east-2a/i-0113f1ef7509cd15b" Oct 31 07:30:26.171018 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161899 4935 flags.go:64] FLAG: --qos-reserved="" Oct 31 07:30:26.171018 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161902 4935 flags.go:64] FLAG: --read-only-port="10255" Oct 31 07:30:26.171018 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161904 4935 flags.go:64] FLAG: --register-node="true" Oct 31 07:30:26.171018 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161907 4935 flags.go:64] FLAG: --register-with-taints="" Oct 31 07:30:26.171018 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161911 4935 flags.go:64] FLAG: --registry-burst="10" Oct 31 07:30:26.171018 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161917 4935 flags.go:64] FLAG: --registry-qps="5" Oct 31 07:30:26.171018 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161920 4935 flags.go:64] FLAG: --reserved-cpus="" Oct 31 07:30:26.171018 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161923 4935 flags.go:64] FLAG: --reserved-memory="" Oct 31 07:30:26.171018 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161926 4935 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Oct 31 07:30:26.171018 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161929 4935 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Oct 31 07:30:26.171018 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161932 4935 flags.go:64] FLAG: --rotate-certificates="false" Oct 31 07:30:26.171018 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161935 4935 flags.go:64] FLAG: --rotate-server-certificates="false" Oct 31 07:30:26.171775 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161938 4935 flags.go:64] FLAG: --runonce="false" Oct 31 07:30:26.171775 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161941 4935 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Oct 31 07:30:26.171775 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161944 4935 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Oct 31 07:30:26.171775 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161947 4935 flags.go:64] FLAG: --seccomp-default="false" Oct 31 07:30:26.171775 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161949 4935 flags.go:64] FLAG: --serialize-image-pulls="true" Oct 31 07:30:26.171775 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161952 4935 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Oct 31 07:30:26.171775 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161956 4935 flags.go:64] FLAG: --storage-driver-db="cadvisor" Oct 31 07:30:26.171775 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161959 4935 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Oct 31 07:30:26.171775 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161962 4935 flags.go:64] FLAG: --storage-driver-password="root" Oct 31 07:30:26.171775 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161965 4935 flags.go:64] FLAG: --storage-driver-secure="false" Oct 31 07:30:26.171775 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161968 4935 flags.go:64] FLAG: --storage-driver-table="stats" Oct 31 07:30:26.171775 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161971 4935 flags.go:64] FLAG: --storage-driver-user="root" Oct 31 07:30:26.171775 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161973 4935 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Oct 31 07:30:26.171775 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161976 4935 flags.go:64] FLAG: --sync-frequency="1m0s" Oct 31 07:30:26.171775 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161980 4935 flags.go:64] FLAG: --system-cgroups="" Oct 31 07:30:26.171775 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161983 4935 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Oct 31 07:30:26.171775 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161988 4935 flags.go:64] FLAG: --system-reserved-cgroup="" Oct 31 07:30:26.171775 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161990 4935 flags.go:64] FLAG: --tls-cert-file="" Oct 31 07:30:26.171775 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161993 4935 flags.go:64] FLAG: --tls-cipher-suites="[]" Oct 31 07:30:26.171775 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161997 4935 flags.go:64] FLAG: --tls-min-version="" Oct 31 07:30:26.171775 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.161999 4935 flags.go:64] FLAG: --tls-private-key-file="" Oct 31 07:30:26.171775 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.162002 4935 flags.go:64] FLAG: --topology-manager-policy="none" Oct 31 07:30:26.171775 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.162005 4935 flags.go:64] FLAG: --topology-manager-policy-options="" Oct 31 07:30:26.171775 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.162007 4935 flags.go:64] FLAG: --topology-manager-scope="container" Oct 31 07:30:26.171775 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.162010 4935 flags.go:64] FLAG: --v="2" Oct 31 07:30:26.172467 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.162014 4935 flags.go:64] FLAG: --version="false" Oct 31 07:30:26.172467 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.162018 4935 flags.go:64] FLAG: --vmodule="" Oct 31 07:30:26.172467 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.162022 4935 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Oct 31 07:30:26.172467 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.162025 4935 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Oct 31 07:30:26.172467 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162146 4935 feature_gate.go:386] unrecognized feature gate: Example Oct 31 07:30:26.172467 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162151 4935 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 31 07:30:26.172467 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162153 4935 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 31 07:30:26.172467 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162156 4935 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 31 07:30:26.172467 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162159 4935 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 31 07:30:26.172467 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162161 4935 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 31 07:30:26.172467 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162164 4935 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 31 07:30:26.172467 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162166 4935 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 31 07:30:26.172467 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162169 4935 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 31 07:30:26.172467 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162173 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 31 07:30:26.172467 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162175 4935 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 31 07:30:26.172467 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162178 4935 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 31 07:30:26.172467 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162180 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 31 07:30:26.172467 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162183 4935 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 31 07:30:26.172467 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162185 4935 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 31 07:30:26.172467 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162188 4935 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 31 07:30:26.172467 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162190 4935 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 31 07:30:26.173167 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162194 4935 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 31 07:30:26.173167 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162196 4935 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 31 07:30:26.173167 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162199 4935 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 31 07:30:26.173167 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162201 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 31 07:30:26.173167 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162204 4935 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 31 07:30:26.173167 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162206 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 31 07:30:26.173167 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162209 4935 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 31 07:30:26.173167 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162211 4935 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 31 07:30:26.173167 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162214 4935 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 31 07:30:26.173167 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162216 4935 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 31 07:30:26.173167 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162219 4935 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 31 07:30:26.173167 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162222 4935 feature_gate.go:386] unrecognized feature gate: Example2 Oct 31 07:30:26.173167 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162224 4935 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 31 07:30:26.173167 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162226 4935 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 31 07:30:26.173167 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162229 4935 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 31 07:30:26.173167 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162231 4935 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 31 07:30:26.173167 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162234 4935 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 31 07:30:26.173167 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162236 4935 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 31 07:30:26.173167 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162238 4935 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 31 07:30:26.173738 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162241 4935 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 31 07:30:26.173738 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162243 4935 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 31 07:30:26.173738 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162246 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 31 07:30:26.173738 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162249 4935 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 31 07:30:26.173738 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162251 4935 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 31 07:30:26.173738 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162255 4935 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 31 07:30:26.173738 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162258 4935 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 31 07:30:26.173738 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162260 4935 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 31 07:30:26.173738 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162263 4935 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 31 07:30:26.173738 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162265 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 31 07:30:26.173738 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162268 4935 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 31 07:30:26.173738 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162270 4935 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 31 07:30:26.173738 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162273 4935 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 31 07:30:26.173738 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162277 4935 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 31 07:30:26.173738 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162279 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 31 07:30:26.173738 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162282 4935 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 31 07:30:26.173738 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162284 4935 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 31 07:30:26.173738 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162287 4935 feature_gate.go:386] unrecognized feature gate: EVPN Oct 31 07:30:26.173738 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162289 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 31 07:30:26.173738 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162292 4935 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 31 07:30:26.174235 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162294 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 31 07:30:26.174235 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162297 4935 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 31 07:30:26.174235 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162299 4935 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 31 07:30:26.174235 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162301 4935 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 31 07:30:26.174235 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162304 4935 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 31 07:30:26.174235 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162307 4935 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 31 07:30:26.174235 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162309 4935 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 31 07:30:26.174235 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162312 4935 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 31 07:30:26.174235 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162314 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 31 07:30:26.174235 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162316 4935 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 31 07:30:26.174235 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162320 4935 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 31 07:30:26.174235 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162324 4935 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 31 07:30:26.174235 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162327 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 31 07:30:26.174235 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162330 4935 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 31 07:30:26.174235 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162332 4935 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 31 07:30:26.174235 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162335 4935 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 31 07:30:26.174235 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162337 4935 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 31 07:30:26.174235 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162341 4935 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 31 07:30:26.174235 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162343 4935 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 31 07:30:26.174769 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162363 4935 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 31 07:30:26.174769 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162367 4935 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 31 07:30:26.174769 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162373 4935 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 31 07:30:26.174769 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162378 4935 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 31 07:30:26.174769 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162383 4935 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 31 07:30:26.174769 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162388 4935 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 31 07:30:26.174769 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162392 4935 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 31 07:30:26.174769 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162395 4935 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 31 07:30:26.174769 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162397 4935 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 31 07:30:26.174769 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162400 4935 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 31 07:30:26.174769 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162402 4935 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 31 07:30:26.174769 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162405 4935 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 31 07:30:26.174769 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162409 4935 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 31 07:30:26.174769 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162413 4935 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 31 07:30:26.174769 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162417 4935 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 31 07:30:26.174769 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162421 4935 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 31 07:30:26.174769 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162425 4935 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 31 07:30:26.174769 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162429 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 31 07:30:26.174769 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162433 4935 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 31 07:30:26.175301 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162436 4935 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 31 07:30:26.175301 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162439 4935 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 31 07:30:26.175301 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162441 4935 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 31 07:30:26.175301 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162444 4935 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 31 07:30:26.175301 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162446 4935 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 31 07:30:26.175301 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162449 4935 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 31 07:30:26.175301 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162451 4935 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 31 07:30:26.175301 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162454 4935 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 31 07:30:26.175301 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162457 4935 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 31 07:30:26.175301 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162459 4935 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 31 07:30:26.175301 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162462 4935 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 31 07:30:26.175301 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162466 4935 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 31 07:30:26.175301 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.162493 4935 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 31 07:30:26.175301 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162527 4935 feature_gate.go:386] unrecognized feature gate: Example Oct 31 07:30:26.175301 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162532 4935 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 31 07:30:26.175301 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162535 4935 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 31 07:30:26.175914 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162538 4935 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 31 07:30:26.175914 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162541 4935 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 31 07:30:26.175914 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162543 4935 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 31 07:30:26.175914 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162546 4935 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 31 07:30:26.175914 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162549 4935 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 31 07:30:26.175914 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162552 4935 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 31 07:30:26.175914 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162554 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 31 07:30:26.175914 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162557 4935 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 31 07:30:26.175914 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162559 4935 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 31 07:30:26.175914 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162562 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 31 07:30:26.175914 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162564 4935 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 31 07:30:26.175914 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162567 4935 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 31 07:30:26.175914 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162569 4935 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 31 07:30:26.175914 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162571 4935 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 31 07:30:26.175914 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162574 4935 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 31 07:30:26.175914 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162577 4935 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 31 07:30:26.175914 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162579 4935 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 31 07:30:26.175914 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162582 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 31 07:30:26.175914 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162584 4935 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 31 07:30:26.176474 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162587 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 31 07:30:26.176474 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162589 4935 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 31 07:30:26.176474 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162592 4935 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 31 07:30:26.176474 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162595 4935 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 31 07:30:26.176474 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162597 4935 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 31 07:30:26.176474 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162600 4935 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 31 07:30:26.176474 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162602 4935 feature_gate.go:386] unrecognized feature gate: Example2 Oct 31 07:30:26.176474 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162605 4935 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 31 07:30:26.176474 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162609 4935 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 31 07:30:26.176474 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162611 4935 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 31 07:30:26.176474 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162614 4935 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 31 07:30:26.176474 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162616 4935 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 31 07:30:26.176474 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162619 4935 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 31 07:30:26.176474 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162622 4935 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 31 07:30:26.176474 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162624 4935 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 31 07:30:26.176474 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162626 4935 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 31 07:30:26.176474 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162629 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 31 07:30:26.176474 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162633 4935 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 31 07:30:26.176474 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162635 4935 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 31 07:30:26.176474 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162638 4935 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 31 07:30:26.176969 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162640 4935 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 31 07:30:26.176969 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162643 4935 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 31 07:30:26.176969 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162645 4935 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 31 07:30:26.176969 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162647 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 31 07:30:26.176969 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162650 4935 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 31 07:30:26.176969 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162652 4935 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 31 07:30:26.176969 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162655 4935 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 31 07:30:26.176969 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162657 4935 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 31 07:30:26.176969 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162660 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 31 07:30:26.176969 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162662 4935 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 31 07:30:26.176969 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162665 4935 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 31 07:30:26.176969 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162667 4935 feature_gate.go:386] unrecognized feature gate: EVPN Oct 31 07:30:26.176969 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162670 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 31 07:30:26.176969 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162672 4935 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 31 07:30:26.176969 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162674 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 31 07:30:26.176969 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162677 4935 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 31 07:30:26.176969 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162679 4935 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 31 07:30:26.176969 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162682 4935 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 31 07:30:26.176969 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162684 4935 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 31 07:30:26.176969 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162686 4935 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 31 07:30:26.177482 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162691 4935 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 31 07:30:26.177482 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162694 4935 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 31 07:30:26.177482 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162696 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 31 07:30:26.177482 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162699 4935 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 31 07:30:26.177482 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162702 4935 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 31 07:30:26.177482 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162706 4935 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 31 07:30:26.177482 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162709 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 31 07:30:26.177482 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162711 4935 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 31 07:30:26.177482 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162714 4935 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 31 07:30:26.177482 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162718 4935 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 31 07:30:26.177482 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162721 4935 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 31 07:30:26.177482 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162724 4935 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 31 07:30:26.177482 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162726 4935 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 31 07:30:26.177482 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162729 4935 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 31 07:30:26.177482 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162731 4935 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 31 07:30:26.177482 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162735 4935 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 31 07:30:26.177482 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162739 4935 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 31 07:30:26.177482 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162742 4935 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 31 07:30:26.177914 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162744 4935 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 31 07:30:26.177914 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162747 4935 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 31 07:30:26.177914 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162749 4935 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 31 07:30:26.177914 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162752 4935 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 31 07:30:26.177914 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162754 4935 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 31 07:30:26.177914 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162757 4935 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 31 07:30:26.177914 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162759 4935 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 31 07:30:26.177914 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162762 4935 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 31 07:30:26.177914 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162764 4935 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 31 07:30:26.177914 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162767 4935 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 31 07:30:26.177914 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162769 4935 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 31 07:30:26.177914 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162772 4935 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 31 07:30:26.177914 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162774 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 31 07:30:26.177914 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162777 4935 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 31 07:30:26.177914 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162780 4935 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 31 07:30:26.177914 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162783 4935 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 31 07:30:26.177914 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162785 4935 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 31 07:30:26.177914 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162788 4935 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 31 07:30:26.177914 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162790 4935 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 31 07:30:26.177914 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162792 4935 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 31 07:30:26.178424 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162795 4935 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 31 07:30:26.178424 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162798 4935 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 31 07:30:26.178424 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162801 4935 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 31 07:30:26.178424 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162804 4935 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 31 07:30:26.178424 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162807 4935 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 31 07:30:26.178424 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.162809 4935 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 31 07:30:26.178424 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.167839 4935 server.go:525] "Kubelet version" kubeletVersion="v1.35.9" Oct 31 07:30:26.178424 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.167856 4935 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Oct 31 07:30:26.178424 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.167877 4935 watchdog_linux.go:95] "Systemd watchdog is not enabled" Oct 31 07:30:26.178424 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.167881 4935 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Oct 31 07:30:26.178424 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.167938 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 31 07:30:26.178424 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.167944 4935 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 31 07:30:26.178424 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.167947 4935 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 31 07:30:26.178424 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.167950 4935 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 31 07:30:26.178424 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.167952 4935 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 31 07:30:26.178424 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.167955 4935 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 31 07:30:26.178424 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.167958 4935 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 31 07:30:26.178424 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.167960 4935 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 31 07:30:26.178424 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.167963 4935 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 31 07:30:26.178939 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.167965 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 31 07:30:26.178939 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.167968 4935 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 31 07:30:26.178939 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.167971 4935 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 31 07:30:26.178939 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.167974 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 31 07:30:26.178939 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.167976 4935 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 31 07:30:26.178939 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.167979 4935 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 31 07:30:26.178939 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.167981 4935 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 31 07:30:26.178939 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.167984 4935 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 31 07:30:26.178939 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.167986 4935 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 31 07:30:26.178939 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.167989 4935 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 31 07:30:26.178939 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.167993 4935 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 31 07:30:26.178939 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.167997 4935 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 31 07:30:26.178939 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168001 4935 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 31 07:30:26.178939 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168003 4935 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 31 07:30:26.178939 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168006 4935 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 31 07:30:26.178939 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168008 4935 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 31 07:30:26.178939 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168011 4935 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 31 07:30:26.178939 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168014 4935 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 31 07:30:26.178939 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168016 4935 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 31 07:30:26.179674 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168020 4935 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 31 07:30:26.179674 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168022 4935 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 31 07:30:26.179674 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168025 4935 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 31 07:30:26.179674 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168028 4935 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 31 07:30:26.179674 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168031 4935 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 31 07:30:26.179674 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168033 4935 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 31 07:30:26.179674 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168036 4935 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 31 07:30:26.179674 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168039 4935 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 31 07:30:26.179674 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168041 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 31 07:30:26.179674 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168043 4935 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 31 07:30:26.179674 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168046 4935 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 31 07:30:26.179674 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168048 4935 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 31 07:30:26.179674 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168051 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 31 07:30:26.179674 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168053 4935 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 31 07:30:26.179674 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168056 4935 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 31 07:30:26.179674 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168058 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 31 07:30:26.179674 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168061 4935 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 31 07:30:26.179674 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168063 4935 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 31 07:30:26.179674 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168066 4935 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 31 07:30:26.180432 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168069 4935 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 31 07:30:26.180432 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168071 4935 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 31 07:30:26.180432 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168073 4935 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 31 07:30:26.180432 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168076 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 31 07:30:26.180432 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168078 4935 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 31 07:30:26.180432 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168080 4935 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 31 07:30:26.180432 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168083 4935 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 31 07:30:26.180432 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168085 4935 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 31 07:30:26.180432 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168088 4935 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 31 07:30:26.180432 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168091 4935 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 31 07:30:26.180432 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168093 4935 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 31 07:30:26.180432 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168095 4935 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 31 07:30:26.180432 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168098 4935 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 31 07:30:26.180432 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168101 4935 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 31 07:30:26.180432 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168103 4935 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 31 07:30:26.180432 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168106 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 31 07:30:26.180432 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168108 4935 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 31 07:30:26.180432 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168110 4935 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 31 07:30:26.180432 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168113 4935 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 31 07:30:26.180432 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168116 4935 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 31 07:30:26.181180 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168118 4935 feature_gate.go:386] unrecognized feature gate: Example Oct 31 07:30:26.181180 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168120 4935 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 31 07:30:26.181180 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168123 4935 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 31 07:30:26.181180 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168125 4935 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 31 07:30:26.181180 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168128 4935 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 31 07:30:26.181180 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168131 4935 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 31 07:30:26.181180 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168134 4935 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 31 07:30:26.181180 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168137 4935 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 31 07:30:26.181180 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168140 4935 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 31 07:30:26.181180 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168143 4935 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 31 07:30:26.181180 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168145 4935 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 31 07:30:26.181180 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168147 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 31 07:30:26.181180 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168151 4935 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 31 07:30:26.181180 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168153 4935 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 31 07:30:26.181180 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168156 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 31 07:30:26.181180 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168158 4935 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 31 07:30:26.181180 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168161 4935 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 31 07:30:26.181180 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168163 4935 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 31 07:30:26.181180 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168165 4935 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 31 07:30:26.181180 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168168 4935 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 31 07:30:26.181774 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168171 4935 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 31 07:30:26.181774 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168173 4935 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 31 07:30:26.181774 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168175 4935 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 31 07:30:26.181774 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168178 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 31 07:30:26.181774 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168181 4935 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 31 07:30:26.181774 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168184 4935 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 31 07:30:26.181774 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168186 4935 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 31 07:30:26.181774 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168189 4935 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 31 07:30:26.181774 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168191 4935 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 31 07:30:26.181774 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168194 4935 feature_gate.go:386] unrecognized feature gate: Example2 Oct 31 07:30:26.181774 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168196 4935 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 31 07:30:26.181774 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168199 4935 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 31 07:30:26.181774 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168201 4935 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 31 07:30:26.181774 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168204 4935 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 31 07:30:26.181774 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168206 4935 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 31 07:30:26.181774 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168209 4935 feature_gate.go:386] unrecognized feature gate: EVPN Oct 31 07:30:26.181774 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168211 4935 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 31 07:30:26.181774 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168213 4935 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 31 07:30:26.181774 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168216 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 31 07:30:26.182265 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.168243 4935 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 31 07:30:26.182265 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168394 4935 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 31 07:30:26.182265 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168399 4935 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 31 07:30:26.182265 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168402 4935 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 31 07:30:26.182265 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168406 4935 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 31 07:30:26.182265 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168410 4935 feature_gate.go:386] unrecognized feature gate: EVPN Oct 31 07:30:26.182265 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168414 4935 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 31 07:30:26.182265 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168417 4935 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 31 07:30:26.182265 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168420 4935 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 31 07:30:26.182265 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168423 4935 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 31 07:30:26.182265 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168426 4935 feature_gate.go:386] unrecognized feature gate: Example Oct 31 07:30:26.182265 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168428 4935 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 31 07:30:26.182265 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168431 4935 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 31 07:30:26.182265 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168433 4935 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 31 07:30:26.182265 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168436 4935 feature_gate.go:386] unrecognized feature gate: Example2 Oct 31 07:30:26.182265 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168438 4935 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 31 07:30:26.182679 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168440 4935 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 31 07:30:26.182679 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168444 4935 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 31 07:30:26.182679 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168446 4935 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 31 07:30:26.182679 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168449 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 31 07:30:26.182679 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168451 4935 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 31 07:30:26.182679 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168454 4935 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 31 07:30:26.182679 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168457 4935 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 31 07:30:26.182679 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168459 4935 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 31 07:30:26.182679 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168461 4935 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 31 07:30:26.182679 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168464 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 31 07:30:26.182679 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168466 4935 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 31 07:30:26.182679 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168469 4935 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 31 07:30:26.182679 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168471 4935 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 31 07:30:26.182679 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168474 4935 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 31 07:30:26.182679 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168476 4935 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 31 07:30:26.182679 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168479 4935 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 31 07:30:26.182679 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168481 4935 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 31 07:30:26.182679 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168484 4935 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 31 07:30:26.182679 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168487 4935 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 31 07:30:26.182679 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168489 4935 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 31 07:30:26.183204 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168492 4935 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 31 07:30:26.183204 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168494 4935 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 31 07:30:26.183204 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168496 4935 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 31 07:30:26.183204 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168500 4935 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 31 07:30:26.183204 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168502 4935 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 31 07:30:26.183204 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168505 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 31 07:30:26.183204 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168507 4935 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 31 07:30:26.183204 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168509 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 31 07:30:26.183204 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168512 4935 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 31 07:30:26.183204 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168514 4935 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 31 07:30:26.183204 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168517 4935 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 31 07:30:26.183204 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168519 4935 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 31 07:30:26.183204 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168522 4935 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 31 07:30:26.183204 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168525 4935 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 31 07:30:26.183204 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168527 4935 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 31 07:30:26.183204 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168530 4935 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 31 07:30:26.183204 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168532 4935 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 31 07:30:26.183204 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168534 4935 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 31 07:30:26.183204 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168537 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 31 07:30:26.183766 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168539 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 31 07:30:26.183766 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168542 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 31 07:30:26.183766 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168544 4935 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 31 07:30:26.183766 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168546 4935 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 31 07:30:26.183766 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168549 4935 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 31 07:30:26.183766 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168551 4935 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 31 07:30:26.183766 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168553 4935 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 31 07:30:26.183766 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168557 4935 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 31 07:30:26.183766 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168560 4935 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 31 07:30:26.183766 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168565 4935 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 31 07:30:26.183766 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168568 4935 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 31 07:30:26.183766 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168570 4935 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 31 07:30:26.183766 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168573 4935 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 31 07:30:26.183766 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168575 4935 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 31 07:30:26.183766 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168577 4935 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 31 07:30:26.183766 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168580 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 31 07:30:26.183766 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168582 4935 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 31 07:30:26.183766 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168586 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 31 07:30:26.183766 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168588 4935 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 31 07:30:26.184227 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168591 4935 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 31 07:30:26.184227 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168593 4935 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 31 07:30:26.184227 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168596 4935 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 31 07:30:26.184227 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168598 4935 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 31 07:30:26.184227 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168600 4935 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 31 07:30:26.184227 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168603 4935 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 31 07:30:26.184227 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168605 4935 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 31 07:30:26.184227 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168608 4935 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 31 07:30:26.184227 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168610 4935 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 31 07:30:26.184227 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168613 4935 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 31 07:30:26.184227 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168615 4935 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 31 07:30:26.184227 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168618 4935 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 31 07:30:26.184227 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168620 4935 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 31 07:30:26.184227 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168623 4935 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 31 07:30:26.184227 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168625 4935 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 31 07:30:26.184227 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168628 4935 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 31 07:30:26.184227 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168630 4935 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 31 07:30:26.184227 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168633 4935 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 31 07:30:26.184227 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168635 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 31 07:30:26.184227 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168637 4935 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 31 07:30:26.184787 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168640 4935 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 31 07:30:26.184787 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168642 4935 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 31 07:30:26.184787 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168645 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 31 07:30:26.184787 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168647 4935 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 31 07:30:26.184787 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168649 4935 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 31 07:30:26.184787 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168651 4935 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 31 07:30:26.184787 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168654 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 31 07:30:26.184787 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168656 4935 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 31 07:30:26.184787 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168658 4935 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 31 07:30:26.184787 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168661 4935 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 31 07:30:26.184787 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168665 4935 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 31 07:30:26.184787 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168669 4935 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 31 07:30:26.184787 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168674 4935 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 31 07:30:26.184787 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.168705 4935 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 31 07:30:26.184787 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168740 4935 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 31 07:30:26.184787 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168744 4935 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 31 07:30:26.185209 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168746 4935 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 31 07:30:26.185209 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168749 4935 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 31 07:30:26.185209 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168752 4935 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 31 07:30:26.185209 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168754 4935 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 31 07:30:26.185209 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168757 4935 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 31 07:30:26.185209 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168759 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 31 07:30:26.185209 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168762 4935 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 31 07:30:26.185209 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168764 4935 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 31 07:30:26.185209 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168767 4935 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 31 07:30:26.185209 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168769 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 31 07:30:26.185209 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168771 4935 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 31 07:30:26.185209 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168774 4935 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 31 07:30:26.185209 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168776 4935 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 31 07:30:26.185209 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168778 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 31 07:30:26.185209 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168781 4935 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 31 07:30:26.185209 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168783 4935 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 31 07:30:26.185209 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168785 4935 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 31 07:30:26.185209 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168788 4935 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 31 07:30:26.185209 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168790 4935 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 31 07:30:26.185707 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168793 4935 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 31 07:30:26.185707 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168795 4935 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 31 07:30:26.185707 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168797 4935 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 31 07:30:26.185707 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168800 4935 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 31 07:30:26.185707 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168804 4935 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 31 07:30:26.185707 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168808 4935 feature_gate.go:386] unrecognized feature gate: EVPN Oct 31 07:30:26.185707 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168811 4935 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 31 07:30:26.185707 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168814 4935 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 31 07:30:26.185707 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168816 4935 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 31 07:30:26.185707 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168819 4935 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 31 07:30:26.185707 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168822 4935 feature_gate.go:386] unrecognized feature gate: Example Oct 31 07:30:26.185707 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168825 4935 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 31 07:30:26.185707 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168827 4935 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 31 07:30:26.185707 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168829 4935 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 31 07:30:26.185707 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168832 4935 feature_gate.go:386] unrecognized feature gate: Example2 Oct 31 07:30:26.185707 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168834 4935 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 31 07:30:26.185707 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168836 4935 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 31 07:30:26.185707 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168839 4935 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 31 07:30:26.185707 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168841 4935 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 31 07:30:26.185707 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168844 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 31 07:30:26.186194 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168847 4935 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 31 07:30:26.186194 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168849 4935 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 31 07:30:26.186194 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168852 4935 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 31 07:30:26.186194 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168854 4935 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 31 07:30:26.186194 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168856 4935 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 31 07:30:26.186194 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168859 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 31 07:30:26.186194 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168861 4935 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 31 07:30:26.186194 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168863 4935 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 31 07:30:26.186194 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168866 4935 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 31 07:30:26.186194 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168868 4935 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 31 07:30:26.186194 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168870 4935 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 31 07:30:26.186194 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168873 4935 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 31 07:30:26.186194 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168875 4935 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 31 07:30:26.186194 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168878 4935 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 31 07:30:26.186194 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168880 4935 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 31 07:30:26.186194 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168882 4935 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 31 07:30:26.186194 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168888 4935 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 31 07:30:26.186194 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168890 4935 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 31 07:30:26.186194 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168893 4935 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 31 07:30:26.186700 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168896 4935 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 31 07:30:26.186700 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168898 4935 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 31 07:30:26.186700 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168901 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 31 07:30:26.186700 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168904 4935 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 31 07:30:26.186700 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168907 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 31 07:30:26.186700 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168909 4935 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 31 07:30:26.186700 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168912 4935 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 31 07:30:26.186700 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168914 4935 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 31 07:30:26.186700 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168917 4935 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 31 07:30:26.186700 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168919 4935 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 31 07:30:26.186700 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168921 4935 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 31 07:30:26.186700 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168924 4935 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 31 07:30:26.186700 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168926 4935 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 31 07:30:26.186700 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168929 4935 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 31 07:30:26.186700 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168931 4935 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 31 07:30:26.186700 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168933 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 31 07:30:26.186700 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168936 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 31 07:30:26.186700 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168938 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 31 07:30:26.186700 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168940 4935 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 31 07:30:26.186700 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168943 4935 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 31 07:30:26.187442 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168945 4935 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 31 07:30:26.187442 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168948 4935 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 31 07:30:26.187442 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168950 4935 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 31 07:30:26.187442 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168953 4935 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 31 07:30:26.187442 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168956 4935 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 31 07:30:26.187442 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168960 4935 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 31 07:30:26.187442 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168963 4935 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 31 07:30:26.187442 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168966 4935 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 31 07:30:26.187442 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168969 4935 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 31 07:30:26.187442 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168973 4935 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 31 07:30:26.187442 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168976 4935 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 31 07:30:26.187442 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168978 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 31 07:30:26.187442 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168981 4935 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 31 07:30:26.187442 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168983 4935 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 31 07:30:26.187442 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168985 4935 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 31 07:30:26.187442 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168988 4935 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 31 07:30:26.187442 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168991 4935 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 31 07:30:26.187442 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168993 4935 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 31 07:30:26.188180 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168995 4935 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 31 07:30:26.188180 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.168998 4935 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 31 07:30:26.188180 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.169001 4935 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 31 07:30:26.188180 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.169003 4935 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 31 07:30:26.188180 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.169006 4935 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 31 07:30:26.188180 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.169008 4935 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 31 07:30:26.188180 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.169012 4935 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 31 07:30:26.188180 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:26.169015 4935 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 31 07:30:26.188180 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.169072 4935 server.go:957] "Client rotation is on, will bootstrap in background" Oct 31 07:30:26.188180 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.170017 4935 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary" Oct 31 07:30:26.188180 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.170075 4935 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem" Oct 31 07:30:26.188180 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.170258 4935 server.go:1014] "Starting client certificate rotation" Oct 31 07:30:26.188180 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.170374 4935 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Oct 31 07:30:26.188180 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.170415 4935 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-30 07:24:45 +0000 UTC" deadline="2028-06-15 15:58:19.542428114 +0000 UTC" Oct 31 07:30:26.188180 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.170451 4935 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14240h27m53.371978806s" Oct 31 07:30:26.188180 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.170780 4935 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 31 07:30:26.188927 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.171891 4935 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 31 07:30:26.188927 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.173080 4935 log.go:25] "Validated CRI v1 runtime API" Oct 31 07:30:26.188927 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.174778 4935 log.go:25] "Validated CRI v1 image API" Oct 31 07:30:26.188927 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.175512 4935 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Oct 31 07:30:26.188927 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.177750 4935 fs.go:135] Filesystem UUIDs: map[492eb354-d37c-4e17-9e39-73d450d41f70:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 87747e47-46e9-4bc1-9b59-cfb442c7132b:/dev/nvme0n1p4] Oct 31 07:30:26.189175 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.177771 4935 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/023f6b65d0f9f8a68940ab14d162cd1c18a4e84cfe18bc9e02de25381fe84786/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/023f6b65d0f9f8a68940ab14d162cd1c18a4e84cfe18bc9e02de25381fe84786/userdata/shm major:0 minor:296 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1040e401a45072871ed858c2c92c752e820ba1e6887613456bd5083ef8a83877/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1040e401a45072871ed858c2c92c752e820ba1e6887613456bd5083ef8a83877/userdata/shm major:0 minor:93 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/115e6d16cbb3c78a7ada677b2d211ea416d8c8df87e680755a40bb9eaf9d4bb6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/115e6d16cbb3c78a7ada677b2d211ea416d8c8df87e680755a40bb9eaf9d4bb6/userdata/shm major:0 minor:236 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1b3a1845b16b5781c7c4e35b89646268e616834157f7adca291705d99cc98e77/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1b3a1845b16b5781c7c4e35b89646268e616834157f7adca291705d99cc98e77/userdata/shm major:0 minor:294 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/49788a1d09e2bb192e4b46ecea1b481cb3df28ad0cbb7feb731e2765844db5e2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/49788a1d09e2bb192e4b46ecea1b481cb3df28ad0cbb7feb731e2765844db5e2/userdata/shm major:0 minor:76 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/61e6f367bdfe0ea5c7737b4f795fed21ec5aaa3f12e073243a74a976db2b98c6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/61e6f367bdfe0ea5c7737b4f795fed21ec5aaa3f12e073243a74a976db2b98c6/userdata/shm major:0 minor:245 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7c6136cafd20d7dc5483470c86e6c4a5d40582798db219bed088e3a4430e6d60/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7c6136cafd20d7dc5483470c86e6c4a5d40582798db219bed088e3a4430e6d60/userdata/shm major:0 minor:334 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7d73c3717618a3632d06b92aba13225d3694b19f95fb058748881de13dbcfd6e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7d73c3717618a3632d06b92aba13225d3694b19f95fb058748881de13dbcfd6e/userdata/shm major:0 minor:86 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8110977887cda9cf1367fb243562b8e66c4948348cec19a4e8b33389cf193838/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8110977887cda9cf1367fb243562b8e66c4948348cec19a4e8b33389cf193838/userdata/shm major:0 minor:238 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/81bc093b082071b56e9c9f3892c254c3cd10ea34a7f034ea79fe9936867ca71f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/81bc093b082071b56e9c9f3892c254c3cd10ea34a7f034ea79fe9936867ca71f/userdata/shm major:0 minor:229 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8d5c8c577bce94daefa7f9e842462e7f57e5680293162c66c04f4aa7ebcc56d9/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8d5c8c577bce94daefa7f9e842462e7f57e5680293162c66c04f4aa7ebcc56d9/userdata/shm major:0 minor:228 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9a2eb5642a9bcd6ecb79d58106e89b4b13ee221824aba714bee38dfaf9f57a8a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9a2eb5642a9bcd6ecb79d58106e89b4b13ee221824aba714bee38dfaf9f57a8a/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9e4560e0b325aa756651336d460ead8b3ed6ad858c3db99b05554503a1e4225d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9e4560e0b325aa756651336d460ead8b3ed6ad858c3db99b05554503a1e4225d/userdata/shm major:0 minor:237 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a1f16ffaf583d8408a8973ff31c9322c54494ca1527c33466774e135f0fad17d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a1f16ffaf583d8408a8973ff31c9322c54494ca1527c33466774e135f0fad17d/userdata/shm major:0 minor:241 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a32ed9582736eace8657e4c5e9e1521b6a93be7e5fb2ff212d0a71e3a22fc402/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a32ed9582736eace8657e4c5e9e1521b6a93be7e5fb2ff212d0a71e3a22fc402/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a6992e8cfd7f4c537e4ca2c5d75ff78aaf7e9d831eb4dbaeec9dec17e03ec726/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a6992e8cfd7f4c537e4ca2c5d75ff78aaf7e9d831eb4dbaeec9dec17e03ec726/userdata/shm major:0 minor:82 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a847b89d581384509483d914b77bb38b311ee6414adcbda176abfa587c130864/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a847b89d581384509483d914b77bb38b311ee6414adcbda176abfa587c130864/userdata/shm major:0 minor:234 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b4a16403faa883c4260e5c78aead776d888c71eb79b7d356dd8b397d49d3b4da/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b4a16403faa883c4260e5c78aead776d888c71eb79b7d356dd8b397d49d3b4da/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c4b192911d11550f7de0ff5884848f30c9f162951df39fa0707f7b8b1f85bc45/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c4b192911d11550f7de0ff5884848f30c9f162951df39fa0707f7b8b1f85bc45/userdata/shm major:0 minor:48 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c8e97bb91fa7a4579a6eb8f24f665103772028210f2a68df470a03c9f5841638/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c8e97bb91fa7a4579a6eb8f24f665103772028210f2a68df470a03c9f5841638/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d187de26ca97f373060f15bc026d354398aa7202968b574737f4169340a5bc1a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d187de26ca97f373060f15bc026d354398aa7202968b574737f4169340a5bc1a/userdata/shm major:0 minor:232 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e4c449b22aa9ecd2802282cc3b7a684d789c2edc99dbea854c45798757d6012f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e4c449b22aa9ecd2802282cc3b7a684d789c2edc99dbea854c45798757d6012f/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e8964c5013096f6a4215b0a3726ca0ca9205b84920d1200007deb94fecce9b21/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e8964c5013096f6a4215b0a3726ca0ca9205b84920d1200007deb94fecce9b21/userdata/shm major:0 minor:242 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/00a0d978-8020-46df-932d-b0dd22f946f9/volumes/kubernetes.io~projected/kube-api-access-2jhpf:{mountpoint:/var/lib/kubelet/pods/00a0d978-8020-46df-932d-b0dd22f946f9/volumes/kubernetes.io~projected/kube-api-access-2jhpf major:0 minor:221 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/00a0d978-8020-46df-932d-b0dd22f946f9/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/00a0d978-8020-46df-932d-b0dd22f946f9/volumes/kubernetes.io~secret/cert major:0 minor:209 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/046b1de8-d709-4729-9aec-391a2b3294d5/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/046b1de8-d709-4729-9aec-391a2b3294d5/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/046b1de8-d709-4729-9aec-391a2b3294d5/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/046b1de8-d709-4729-9aec-391a2b3294d5/volumes/kubernetes.io~empty-dir/tmp major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/046b1de8-d709-4729-9aec-391a2b3294d5/volumes/kubernetes.io~projected/kube-api-access-76x7d:{mountpoint:/var/lib/kubelet/pods/046b1de8-d709-4729-9aec-391a2b3294d5/volumes/kubernetes.io~projected/kube-api-access-76x7d major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/236238a7-255d-40f7-a99b-d65903b3bcb7/volumes/kubernetes.io~projected/kube-api-access-ftf9v:{mountpoint:/var/lib/kubelet/pods/236238a7-255d-40f7-a99b-d65903b3bcb7/volumes/kubernetes.io~projected/kube-api-access-ftf9v major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/28be803a-a932-4009-ad61-b356366821ef/volumes/kubernetes.io~projected/kube-api-access-cf9zh:{mountpoint:/var/lib/kubelet/pods/28be803a-a932-4009-ad61-b356366821ef/volumes/kubernetes.io~projected/kube-api-access-cf9zh major:0 minor:215 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/28be803a-a932-4009-ad61-b356366821ef/volumes/kubernetes.io~secret/default-certificate:{mountpoint:/var/lib/kubelet/pods/28be803a-a932-4009-ad61-b356366821ef/volumes/kubernetes.io~secret/default-certificate major:0 minor:206 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/28be803a-a932-4009-ad61-b356366821ef/volumes/kubernetes.io~secret/stats-auth:{mountpoint:/var/lib/kubelet/pods/28be803a-a932-4009-ad61-b356366821ef/volumes/kubernetes.io~secret/stats-auth major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/28ec9621-7ec5-421c-a88b-51bb20d3a7bf/volumes/kubernetes.io~projected/kube-api-access-8mqks:{mountpoint:/var/lib/kubelet/pods/28ec9621-7ec5-421c-a88b-51bb20d3a7bf/volumes/kubernetes.io~projected/kube-api-access-8mqks major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/297b35d7-aa65-49c7-8460-ac3f9860248c/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/297b35d7-aa65-49c7-8460-ac3f9860248c/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:288 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/396e1c5e-3700-4fe6-b367-54ad2a244225/volumes/kubernetes.io~projected/kube-api-access-g76kv:{mountpoint:/var/lib/kubelet/pods/396e1c5e-3700-4fe6-b367-54ad2a244225/volumes/kubernetes.io~projected/kube-api-access-g76kv major:0 minor:227 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/396e1c5e-3700-4fe6-b367-54ad2a244225/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/396e1c5e-3700-4fe6-b367-54ad2a244225/volumes/kubernetes.io~secret/serving-cert major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3cbf4695-0536-45d5-bc48-be56861b87b8/volumes/kubernetes.io~projected/kube-api-access-v4zgz:{mountpoint:/var/lib/kubelet/pods/3cbf4695-0536-45d5-bc48-be56861b87b8/volumes/kubernetes.io~projected/kube-api-access-v4zgz major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4104ba06-fba8-4c3c-b876-cca5087a7d0a/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/4104ba06-fba8-4c3c-b876-cca5087a7d0a/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:223 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4104ba06-fba8-4c3c-b876-cca5087a7d0a/volumes/kubernetes.io~projected/kube-api-access-hldt6:{mountpoint:/var/lib/kubelet/pods/4104ba06-fba8-4c3c-b876-cca5087a7d0a/volumes/kubernetes.io~projected/kube-api-access-hldt6 major:0 minor:218 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4104ba06-fba8-4c3c-b876-cca5087a7d0a/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/4104ba06-fba8-4c3c-b876-cca5087a7d0a/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4104ba06-fba8-4c3c-b876-cca5087a7d0a/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/4104ba06-fba8-4c3c-b876-cca5087a7d0a/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:207 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/43c59587-4ff8-4546-9c0a-6161ece201d0/volumes/kubernetes.io~projected/kube-api-access-82jqk:{mountpoint:/var/lib/kubelet/pods/43c59587-4ff8-4546-9c0a-6161ece201d0/volumes/kubernetes.io~projected/kube-api-access-82jqk major:0 minor:226 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/53c36e19-9ef6-47a2-9bc2-387115539850/volumes/kubernetes.io~projected/kube-api-access-zzvgd:{mountpoint:/var/lib/kubelet/pods/53c36e19-9ef6-47a2-9bc2-387115539850/volumes/kubernetes.io~projected/kube-api-access-zzvgd major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/54296c9d-1c43-44cd-bfb6-d81fe892484c/volumes/kubernetes.io~projected/kube-api-access-lx427:{mountpoint:/var/lib/kubelet/pods/54296c9d-1c43-44cd-bfb6-d81fe892484c/volumes/kubernetes.io~projected/kube-api-access-lx427 major:0 minor:217 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/58bc3ea1-f58c-4891-a049-1289a99be055/volumes/kubernetes.io~projected/kube-api-access-4q9t4:{mountpoint:/var/lib/kubelet/pods/58bc3ea1-f58c-4891-a049-1289a99be055/volumes/kubernetes.io~projected/kube-api-access-4q9t4 major:0 minor:213 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5e743234-2b5f-4c91-9b06-189f546cbce4/volumes/kubernetes.io~projected/kube-api-access-djqrt:{mountpoint:/var/lib/kubelet/pods/5e743234-2b5f-4c91-9b06-189f546cbce4/volumes/kubernetes.io~projected/kube-api-access-djqrt major:0 minor:219 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6ae10413-8f49-4dfe-8aad-00dcb9d390c8/volumes/kubernetes.io~projected/kube-api-access-997f5:{mountpoint:/var/lib/kubelet/pods/6ae10413-8f49-4dfe-8aad-00dcb9d390c8/volumes/kubernetes.io~projected/kube-api-access-997f5 major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6fae9634-7abe-40c6-a892-9eb58166e9ad/volumes/kubernetes.io~projected/kube-api-access-wsvzt:{mountpoint:/var/lib/kubelet/pods/6fae9634-7abe-40c6-a892-9eb58166e9ad/volumes/kubernetes.io~projected/kube-api-access-wsvzt major:0 minor:214 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ac4bf75a-377c-49b2-818a-7d2436c21e9c/volumes/kubernetes.io~projected/kube-api-access-xv5b5:{mountpoint:/var/lib/kubelet/pods/ac4bf75a-377c-49b2-818a-7d2436c21e9c/volumes/kubernetes.io~projected/kube-api-access-xv5b5 major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/acda6b08-2faa-4836-888e-d4ecf324cb1d/volumes/kubernetes.io~projected/kube-api-access-tvpkt:{mountpoint:/var/lib/kubelet/pods/acda6b08-2faa-4836-888e-d4ecf324cb1d/volumes/kubernetes.io~projected/kube-api-access-tvpkt major:0 minor:333 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b4dd8884-1d6e-4abe-ad6c-3c947623bb60/volumes/kubernetes.io~projected/kube-api-access-9k9q8:{mountpoint:/var/lib/kubelet/pods/b4dd8884-1d6e-4abe-ad6c-3c947623bb60/volumes/kubernetes.io~projected/kube-api-access-9k9q8 major:0 minor:211 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b4dd8884-1d6e-4abe-ad6c-3c947623bb60/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/b4dd8884-1d6e-4abe-ad6c-3c947623bb60/volumes/kubernetes.io~secret/serving-cert major:0 minor:205 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b6997c76-e103-4c3f-b39d-a6a17fb98270/volumes/kubernetes.io~projected/kube-api-access-p76jj:{mountpoint:/var/lib/kubelet/pods/b6997c76-e103-4c3f-b39d-a6a17fb98270/volumes/kubernetes.io~projected/kube-api-access-p76jj major:0 minor:225 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b6997c76-e103-4c3f-b39d-a6a17fb98270/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/b6997c76-e103-4c3f-b39d-a6a17fb98270/volumes/kubernetes.io~secret/serving-cert major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c734ebf9-eba8-4e14-b20d-2d78dd65b10d/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/c734ebf9-eba8-4e14-b20d-2d78dd65b10d/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:222 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c734ebf9-eba8-4e14-b20d-2d78dd65b10d/volumes/kubernetes.io~projected/kube-api-access-z5s89:{mountpoint:/var/lib/kubelet/pods/c734ebf9-eba8-4e14-b20d-2d78dd65b10d/volumes/kubernetes.io~projected/kube-api-access-z5s89 major:0 minor:224 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c734ebf9-eba8-4e14-b20d-2d78dd65b10d/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/c734ebf9-eba8-4e14-b20d-2d78dd65b10d/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:208 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c734ebf9-eba8-4e14-b20d-2d78dd65b10d/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/c734ebf9-eba8-4e14-b20d-2d78dd65b10d/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:210 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d25f2854-fd38-44e7-ad7e-da744d57e671/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/d25f2854-fd38-44e7-ad7e-da744d57e671/volumes/kubernetes.io~secret/agent-certs major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e/volumes/kubernetes.io~projected/kube-api-access-nlm6c:{mountpoint:/var/lib/kubelet/pods/d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e/volumes/kubernetes.io~projected/kube-api-access-nlm6c major:0 minor:220 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d8b28b9f-37a2-4859-87fd-d27bb37d832b/volumes/kubernetes.io~projected/kube-api-access-2ckzf:{mountpoint:/var/lib/kubelet/pods/d8b28b9f-37a2-4859-87fd-d27bb37d832b/volumes/kubernetes.io~projected/kube-api-access-2ckzf major:0 minor:293 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e00eb9e3-0d3c-4541-b238-4130fe1de2f1/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/e00eb9e3-0d3c-4541-b238-4130fe1de2f1/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e00eb9e3-0d3c-4541-b238-4130fe1de2f1/volumes/kubernetes.io~projected/kube-api-access-mrvrg:{mountpoint:/var/lib/kubelet/pods/e00eb9e3-0d3c-4541-b238-4130fe1de2f1/volumes/kubernetes.io~projected/kube-api-access-mrvrg major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e00eb9e3-0d3c-4541-b238-4130fe1de2f1/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/e00eb9e3-0d3c-4541-b238-4130fe1de2f1/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e84bef13-a137-49f7-931d-9f94952dacfc/volumes/kubernetes.io~projected/kube-api-access-xksf5:{mountpoint:/var/lib/kubelet/pods/e84bef13-a137-49f7-931d-9f94952dacfc/volumes/kubernetes.io~projected/kube-api-access-xksf5 major:0 minor:212 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ea9beb38-1491-4cf0-83ca-fd79935fa6a9/volumes/kubernetes.io~projected/kube-api-access-wrj54:{mountpoint:/var/lib/kubelet/pods/ea9beb38-1491-4cf0-83ca-fd79935fa6a9/volumes/kubernetes.io~projected/kube-api-access-wrj54 major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ed522b38-611a-452a-b201-3ca416428d42/volumes/kubernetes.io~projected/kube-api-access-w4tdk:{mountpoint:/var/lib/kubelet/pods/ed522b38-611a-452a-b201-3ca416428d42/volumes/kubernetes.io~projected/kube-api-access-w4tdk major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f13bc62a-1894-472c-95de-262afa399b38/volumes/kubernetes.io~projected/kube-api-access-qhvbn:{mountpoint:/var/lib/kubelet/pods/f13bc62a-1894-472c-95de-262afa399b38/volumes/kubernetes.io~projected/kube-api-access-qhvbn major:0 minor:316 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f3bf19fd-bc58-4dff-a1a8-4f61381e1fde/volumes/kubernetes.io~projected/kube-api-access-cb5dx:{mountpoint:/var/lib/kubelet/pods/f3bf19fd-bc58-4dff-a1a8-4f61381e1fde/volumes/kubernetes.io~projected/kube-api-access-cb5dx major:0 minor:216 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f3bf19fd-bc58-4dff-a1a8-4f61381e1fde/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/f3bf19fd-bc58-4dff-a1a8-4f61381e1fde/volumes/kubernetes.io~secret/serving-cert major:0 minor:201 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/573f932c3b1a739d462ca32f1acdca5cf8222bad977b4313a6d031001ed80cae/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/4130ef341a41f4cd988c7dc398a526e95dc4ae8eb26f4f9cb720168c87baf677/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/94669077a5f0c1c5970cb80eae47ab8125b69a34a6ba48f664b2371abbb9526b/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/968eaff9e8a12c9450bbfbfd1b3fe82393b17d67ce07040c6c81595920559e69/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/0fbfc331895f976541315165a7dbd787504c724997597247c22c1697735a6832/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/456d6a38c8def8b20e1594a39261736f245a1b3e1a875ba923344e69e1eff77d/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/93a18edc41d8c7448db2e8ee0f435ffde1e261b79c9f87112ad85ef25e0d4874/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/2e21a76faf65044a2dc8a47aedb3b6444c1edefa58515adfd4df68f2c83e0d7b/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/0cf01713e5194ad2ee33ca33fbe757e22470bce71692f091a77ef6c422b4d306/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-137:{mountpoint:/var/lib/containers/storage/overlay/844f1b1b8efb76371085cea10130211974656ca922c095b85c504b57cbb3f514/merged major:0 minor:137 fsType:overlay blockSize:0} overlay_0-142:{mountpoint:/var/lib/containers/storage/overlay/396ed9613e3796114f839b5899790c26f4a73639f237dce9f06dbde1d52283c2/merged major:0 minor:142 fsType:overlay blockSize:0} overlay_0-144:{mountpoint:/var/lib/containers/storage/overlay/799f1376cb51f4a5e45d8d36d6aae380acb09be6d0d1589ded3723d86004b807/merged major:0 minor:144 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/a25e66dce55092b89b9e68cbc143f7634dfd907119dcfbcc2e213059330ce125/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/c3dd89775697e9c9ba5ed10af5e7cf3f8d989252f0e8d73e13dd5b87e52ac80a/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/b85acf9ebaf6427ded7a2200ffb11c742178907281b0a2431069b965bc27ecd9/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/b16cf3f7c5095b4f36dd5ebbddde1a989104662c24d72fb797a63132857dfb02/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/e8afd524cf25ae6f7db8b61b18731edaeb4d6654cdcd6294124b7e378c89463a/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/e74a77c6ae7a32b67e59fe1e0c33c86f7f4fc6978b44c08e0703fe1f50080e5f/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/b161b1cc52b359edf46831b02432e51d938676ceee27eb72f141090aa27661a0/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/0203482bf2c192572e7fdc100eb4a9a2874f2de9a405f609698bb828a0bc6032/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/005ed45d0ee7e43ea568390a962c05c393b37284ec0670f9aa3cd8a30a1500e9/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-248:{mountpoint:/var/lib/containers/storage/overlay/a6a7fa57414c05ec307f876d8f4944b1d0fd1a5069c81996a2a43c35a56fa42b/merged major:0 minor:248 fsType:overlay blockSize:0} overlay_0-250:{mountpoint:/var/lib/containers/storage/overlay/62cc663bdb36b96b61343f1afb8cc4241a19d8d4fb05293b43083dcfa9bed805/merged major:0 minor:250 fsType:overlay blockSize:0} overlay_0-252:{mountpoint:/var/lib/containers/storage/overlay/91387be9f3143928664709e6bc586e6816dac698bc14f727029e4392a047d45a/merged major:0 minor:252 fsType:overlay blockSize:0} overlay_0-254:{mountpoint:/var/lib/containers/storage/overlay/779b1679f6f76783099844b1199b2a7d619a3fdee7429aba6fc6cc4ba69e2778/merged major:0 minor:254 fsType:overlay blockSize:0} overlay_0-256:{mountpoint:/var/lib/containers/storage/overlay/89cb762362d4fa47ccf05cfc824f521dd2b24472d6f7fad16285d8a9307041f5/merged major:0 minor:256 fsType:overlay blockSize:0} overlay_0-258:{mountpoint:/var/lib/containers/storage/overlay/895d1d4e1d1b97d782c6a109648e48e6ae9e10e851757e569bc84fe6caf1756c/merged major:0 minor:258 fsType:overlay blockSize:0} overlay_0-260:{mountpoint:/var/lib/containers/storage/overlay/7e1bd989d9c86bf9b55be1c8f915548aba2fe3751065a6a4702a1d02865e1156/merged major:0 minor:260 fsType:overlay blockSize:0} overlay_0-262:{mountpoint:/var/lib/containers/storage/overlay/5f91d9eed120565ad14a41a0640b8f626dcdafd6d5f713e760d343777059ccc2/merged major:0 minor:262 fsType:overlay blockSize:0} overlay_0-264:{mountpoint:/var/lib/containers/storage/overlay/2c960afa22a19114934fa8ce9f38c85fb04ab4943a2b38e76f52c20b30ce7707/merged major:0 minor:264 fsType:overlay blockSize:0} overlay_0-274:{mountpoint:/var/lib/containers/storage/overlay/a6e47797fe6c9dcdaff18b68ac56c43bbd6a89fc45760982af59dd0a603c6db3/merged major:0 minor:274 fsType:overlay blockSize:0} overlay_0-276:{mountpoint:/var/lib/containers/storage/overlay/8e0528a60a86a27bc0a92e5a4e947a279ec359d34959c1082ae066e6e301998c/merged major:0 minor:276 fsType:overlay blockSize:0} overlay_0-277:{mountpoint:/var/lib/containers/storage/overlay/a65e55cd96991f97335d7cc8625154255b7a07d7c43b72d64be82896852fc310/merged major:0 minor:277 fsType:overlay blockSize:0} overlay_0-279:{mountpoint:/var/lib/containers/storage/overlay/57ce77472810c0bfbc6792d6d80ed6916861c5bd89a04c3e48b0e378fb89e49f/merged major:0 minor:279 fsType:overlay blockSize:0} overlay_0-286:{mountpoint:/var/lib/containers/storage/overlay/c7362ab62d2cceb97017d6675f2cfc2ef49e05716f9bd3e247fb23c2a2c940e0/merged major:0 minor:286 fsType:overlay blockSize:0} overlay_0-298:{mountpoint:/var/lib/containers/storage/overlay/b90302672db503847f22239eae94d18f46ca194d685c0f4e602942d1d18c0d4b/merged major:0 minor:298 fsType:overlay blockSize:0} overlay_0-300:{mountpoint:/var/lib/containers/storage/overlay/e521ef511bdfe0390af25770a07a4054f56707c8b18097d4410c1ad87096a11f/merged major:0 minor:300 fsType:overlay blockSize:0} overlay_0-302:{mountpoint:/var/lib/containers/storage/overlay/e8673c7ab1695b04a1b7f94ff5039e4d76d1a0276f3bf5bbd8d610700ef16598/merged major:0 minor:302 fsType:overlay blockSize:0} overlay_0-304:{mountpoint:/var/lib/containers/storage/overlay/acfe0e13d8e8ca801e77a1ee9221cbdc847d54f2eaba99fb7e84f7b340023611/merged major:0 minor:304 fsType:overlay blockSize:0} overlay_0-306:{mountpoint:/var/lib/containers/storage/overlay/d7a66f6f6961c2c492f5259221771c9edc1b7a33cc4baaa9198e7b856a75b639/merged major:0 minor:306 fsType:overlay blockSize:0} overlay_0-308:{mountpoint:/var/lib/containers/storage/overlay/b3b7ada4f5053b7e630e6d5aa912a8c02f71dc1a642dd5eff15daa20ecb54918/merged major:0 minor:308 fsType:overlay blockSize:0} overlay_0-310:{mountpoint:/var/lib/containers/storage/overlay/02b2676cdae96ff8d8cc5b46acdd70ff8cd639d4e53b261a14eab9acf11a3f99/merged major:0 minor:310 fsType:overlay blockSize:0} overlay_0-336:{mountpoint:/var/lib/containers/storage/overlay/ba494ffa07c1ffd41d9ed3418f565343b14f7290961832e52c3feefdbc2c7d06/merged major:0 minor:336 fsType:overlay blockSize:0} overlay_0-338:{mountpoint:/var/lib/containers/storage/overlay/97c5d6a0944e635f345808a1cbf7a44462a88bae8a0860510ee5331fe2420e99/merged major:0 minor:338 fsType:overlay blockSize:0} overlay_0-344:{mountpoint:/var/lib/containers/storage/overlay/bbe9c16ab194b2d31a43773e0382db8f44ab21ededac20ec5194232103592a3b/merged major:0 minor:344 fsType:overlay blockSize:0} overlay_0-346:{mountpoint:/var/lib/containers/storage/overlay/c831b2f12b92fa5c8ddc680a31820cdb49a31cf3ac52e555ea02f0223d7ec104/merged major:0 minor:346 fsType:overlay blockSize:0} overlay_0-51:{mountpoint:/var/lib/containers/storage/overlay/b8c5eec64b5ffc87da44226081fe9c1e14756b61c40fb59f30bd8e50423e99cc/merged major:0 minor:51 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/fe5410e70b0f9f3faa62c176b8bb4f1a75f58147419559d7f10a09e291260a1b/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/e385d7a407c6b79414af2b1f12ce6384d1b5e22f7e9b1937a55d336f0ff5d855/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/c4f3ba200d1caba40c8d6f650a9d352b35f5082e5b0c15ae7b5dff1d3bb0893c/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-78:{mountpoint:/var/lib/containers/storage/overlay/164e85526b4e998b235cf3cccbb78d8882df0021d848adf6be674275748be24c/merged major:0 minor:78 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/aba74d5f0eb1eb3dedea305860ec43fb38a51751fb608ae48bf6735ffe8034b2/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/e3132307b67033afe83ce42dd1128e7da78d91c5ebab7d90546e56c73f8608a0/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/6a47a1b2afb968204cc113913ffb5fc0c1631f2d80041b731f843919a06f97b1/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/032a6d44079e9651b8c9894d34fe66d54f0c4bc762c6f6580b9eb131273860dc/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/314c6b163efd960bee7dc00db7b213f7a8c0d0667dea08f99f4812d2711d6153/merged major:0 minor:96 fsType:overlay blockSize:0}] Oct 31 07:30:26.189447 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.186473 4935 manager.go:217] Machine: {Timestamp:2026-10-31 07:30:26.186007573 +0000 UTC m=+0.056539102 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098719 MemoryCapacity:32812011520 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec269141edead14f4cc16acc2450ceec SystemUUID:ec269141-edea-d14f-4cc1-6acc2450ceec BootID:86951882-5a2d-46ec-9ec7-d012a538784f Filesystems:[{Device:/var/lib/kubelet/pods/53c36e19-9ef6-47a2-9bc2-387115539850/volumes/kubernetes.io~projected/kube-api-access-zzvgd DeviceMajor:0 DeviceMinor:63 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4104ba06-fba8-4c3c-b876-cca5087a7d0a/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:223 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406007808 Type:vfs Inodes:1048576 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c734ebf9-eba8-4e14-b20d-2d78dd65b10d/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:210 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/00a0d978-8020-46df-932d-b0dd22f946f9/volumes/kubernetes.io~projected/kube-api-access-2jhpf DeviceMajor:0 DeviceMinor:221 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/81bc093b082071b56e9c9f3892c254c3cd10ea34a7f034ea79fe9936867ca71f/userdata/shm DeviceMajor:0 DeviceMinor:229 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-260 DeviceMajor:0 DeviceMinor:260 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-276 DeviceMajor:0 DeviceMinor:276 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562402304 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/28ec9621-7ec5-421c-a88b-51bb20d3a7bf/volumes/kubernetes.io~projected/kube-api-access-8mqks DeviceMajor:0 DeviceMinor:58 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/396e1c5e-3700-4fe6-b367-54ad2a244225/volumes/kubernetes.io~projected/kube-api-access-g76kv DeviceMajor:0 DeviceMinor:227 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-338 DeviceMajor:0 DeviceMinor:338 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/6fae9634-7abe-40c6-a892-9eb58166e9ad/volumes/kubernetes.io~projected/kube-api-access-wsvzt DeviceMajor:0 DeviceMinor:214 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-250 DeviceMajor:0 DeviceMinor:250 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-252 DeviceMajor:0 DeviceMinor:252 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ed522b38-611a-452a-b201-3ca416428d42/volumes/kubernetes.io~projected/kube-api-access-w4tdk DeviceMajor:0 DeviceMinor:59 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/c734ebf9-eba8-4e14-b20d-2d78dd65b10d/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:208 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7d73c3717618a3632d06b92aba13225d3694b19f95fb058748881de13dbcfd6e/userdata/shm DeviceMajor:0 DeviceMinor:86 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f3bf19fd-bc58-4dff-a1a8-4f61381e1fde/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:201 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/58bc3ea1-f58c-4891-a049-1289a99be055/volumes/kubernetes.io~projected/kube-api-access-4q9t4 DeviceMajor:0 DeviceMinor:213 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-258 DeviceMajor:0 DeviceMinor:258 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/023f6b65d0f9f8a68940ab14d162cd1c18a4e84cfe18bc9e02de25381fe84786/userdata/shm DeviceMajor:0 DeviceMinor:296 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/046b1de8-d709-4729-9aec-391a2b3294d5/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:61 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/f3bf19fd-bc58-4dff-a1a8-4f61381e1fde/volumes/kubernetes.io~projected/kube-api-access-cb5dx DeviceMajor:0 DeviceMinor:216 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/acda6b08-2faa-4836-888e-d4ecf324cb1d/volumes/kubernetes.io~projected/kube-api-access-tvpkt DeviceMajor:0 DeviceMinor:333 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/396e1c5e-3700-4fe6-b367-54ad2a244225/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:204 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/28be803a-a932-4009-ad61-b356366821ef/volumes/kubernetes.io~secret/default-certificate DeviceMajor:0 DeviceMinor:206 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/e00eb9e3-0d3c-4541-b238-4130fe1de2f1/volumes/kubernetes.io~projected/kube-api-access-mrvrg DeviceMajor:0 DeviceMinor:56 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/28be803a-a932-4009-ad61-b356366821ef/volumes/kubernetes.io~projected/kube-api-access-cf9zh DeviceMajor:0 DeviceMinor:215 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1040e401a45072871ed858c2c92c752e820ba1e6887613456bd5083ef8a83877/userdata/shm DeviceMajor:0 DeviceMinor:93 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9e4560e0b325aa756651336d460ead8b3ed6ad858c3db99b05554503a1e4225d/userdata/shm DeviceMajor:0 DeviceMinor:237 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-254 DeviceMajor:0 DeviceMinor:254 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/var/lib/kubelet/pods/d25f2854-fd38-44e7-ad7e-da744d57e671/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:60 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406003712 Type:vfs Inodes:4005372 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4104ba06-fba8-4c3c-b876-cca5087a7d0a/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:207 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e4c449b22aa9ecd2802282cc3b7a684d789c2edc99dbea854c45798757d6012f/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/43c59587-4ff8-4546-9c0a-6161ece201d0/volumes/kubernetes.io~projected/kube-api-access-82jqk DeviceMajor:0 DeviceMinor:226 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d187de26ca97f373060f15bc026d354398aa7202968b574737f4169340a5bc1a/userdata/shm DeviceMajor:0 DeviceMinor:232 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a847b89d581384509483d914b77bb38b311ee6414adcbda176abfa587c130864/userdata/shm DeviceMajor:0 DeviceMinor:234 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c4b192911d11550f7de0ff5884848f30c9f162951df39fa0707f7b8b1f85bc45/userdata/shm DeviceMajor:0 DeviceMinor:48 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/e84bef13-a137-49f7-931d-9f94952dacfc/volumes/kubernetes.io~projected/kube-api-access-xksf5 DeviceMajor:0 DeviceMinor:212 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1b3a1845b16b5781c7c4e35b89646268e616834157f7adca291705d99cc98e77/userdata/shm DeviceMajor:0 DeviceMinor:294 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-306 DeviceMajor:0 DeviceMinor:306 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/046b1de8-d709-4729-9aec-391a2b3294d5/volumes/kubernetes.io~projected/kube-api-access-76x7d DeviceMajor:0 DeviceMinor:67 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-137 DeviceMajor:0 DeviceMinor:137 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/28be803a-a932-4009-ad61-b356366821ef/volumes/kubernetes.io~secret/stats-auth DeviceMajor:0 DeviceMinor:202 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a6992e8cfd7f4c537e4ca2c5d75ff78aaf7e9d831eb4dbaeec9dec17e03ec726/userdata/shm DeviceMajor:0 DeviceMinor:82 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-144 DeviceMajor:0 DeviceMinor:144 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3cbf4695-0536-45d5-bc48-be56861b87b8/volumes/kubernetes.io~projected/kube-api-access-v4zgz DeviceMajor:0 DeviceMinor:45 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/c734ebf9-eba8-4e14-b20d-2d78dd65b10d/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:222 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-256 DeviceMajor:0 DeviceMinor:256 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c8e97bb91fa7a4579a6eb8f24f665103772028210f2a68df470a03c9f5841638/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/4104ba06-fba8-4c3c-b876-cca5087a7d0a/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:197 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-262 DeviceMajor:0 DeviceMinor:262 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d8b28b9f-37a2-4859-87fd-d27bb37d832b/volumes/kubernetes.io~projected/kube-api-access-2ckzf DeviceMajor:0 DeviceMinor:293 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/ea9beb38-1491-4cf0-83ca-fd79935fa6a9/volumes/kubernetes.io~projected/kube-api-access-wrj54 DeviceMajor:0 DeviceMinor:64 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9a2eb5642a9bcd6ecb79d58106e89b4b13ee221824aba714bee38dfaf9f57a8a/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b6997c76-e103-4c3f-b39d-a6a17fb98270/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:203 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/5e743234-2b5f-4c91-9b06-189f546cbce4/volumes/kubernetes.io~projected/kube-api-access-djqrt DeviceMajor:0 DeviceMinor:219 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-346 DeviceMajor:0 DeviceMinor:346 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/6ae10413-8f49-4dfe-8aad-00dcb9d390c8/volumes/kubernetes.io~projected/kube-api-access-997f5 DeviceMajor:0 DeviceMinor:57 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b4dd8884-1d6e-4abe-ad6c-3c947623bb60/volumes/kubernetes.io~projected/kube-api-access-9k9q8 DeviceMajor:0 DeviceMinor:211 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/54296c9d-1c43-44cd-bfb6-d81fe892484c/volumes/kubernetes.io~projected/kube-api-access-lx427 DeviceMajor:0 DeviceMinor:217 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/61e6f367bdfe0ea5c7737b4f795fed21ec5aaa3f12e073243a74a976db2b98c6/userdata/shm DeviceMajor:0 DeviceMinor:245 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-277 DeviceMajor:0 DeviceMinor:277 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-344 DeviceMajor:0 DeviceMinor:344 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/046b1de8-d709-4729-9aec-391a2b3294d5/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:62 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b6997c76-e103-4c3f-b39d-a6a17fb98270/volumes/kubernetes.io~projected/kube-api-access-p76jj DeviceMajor:0 DeviceMinor:225 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/49788a1d09e2bb192e4b46ecea1b481cb3df28ad0cbb7feb731e2765844db5e2/userdata/shm DeviceMajor:0 DeviceMinor:76 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e/volumes/kubernetes.io~projected/kube-api-access-nlm6c DeviceMajor:0 DeviceMinor:220 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-302 DeviceMajor:0 DeviceMinor:302 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e8964c5013096f6a4215b0a3726ca0ca9205b84920d1200007deb94fecce9b21/userdata/shm DeviceMajor:0 DeviceMinor:242 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-298 DeviceMajor:0 DeviceMinor:298 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ac4bf75a-377c-49b2-818a-7d2436c21e9c/volumes/kubernetes.io~projected/kube-api-access-xv5b5 DeviceMajor:0 DeviceMinor:65 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/b4dd8884-1d6e-4abe-ad6c-3c947623bb60/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:205 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8110977887cda9cf1367fb243562b8e66c4948348cec19a4e8b33389cf193838/userdata/shm DeviceMajor:0 DeviceMinor:238 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/297b35d7-aa65-49c7-8460-ac3f9860248c/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:288 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7c6136cafd20d7dc5483470c86e6c4a5d40582798db219bed088e3a4430e6d60/userdata/shm DeviceMajor:0 DeviceMinor:334 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b4a16403faa883c4260e5c78aead776d888c71eb79b7d356dd8b397d49d3b4da/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/115e6d16cbb3c78a7ada677b2d211ea416d8c8df87e680755a40bb9eaf9d4bb6/userdata/shm DeviceMajor:0 DeviceMinor:236 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-264 DeviceMajor:0 DeviceMinor:264 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-274 DeviceMajor:0 DeviceMinor:274 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-78 DeviceMajor:0 DeviceMinor:78 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-51 DeviceMajor:0 DeviceMinor:51 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e00eb9e3-0d3c-4541-b238-4130fe1de2f1/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6562402304 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/00a0d978-8020-46df-932d-b0dd22f946f9/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:209 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/e00eb9e3-0d3c-4541-b238-4130fe1de2f1/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:55 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/236238a7-255d-40f7-a99b-d65903b3bcb7/volumes/kubernetes.io~projected/kube-api-access-ftf9v DeviceMajor:0 DeviceMinor:66 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8d5c8c577bce94daefa7f9e842462e7f57e5680293162c66c04f4aa7ebcc56d9/userdata/shm DeviceMajor:0 DeviceMinor:228 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-248 DeviceMajor:0 DeviceMinor:248 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-286 DeviceMajor:0 DeviceMinor:286 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-310 DeviceMajor:0 DeviceMinor:310 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a1f16ffaf583d8408a8973ff31c9322c54494ca1527c33466774e135f0fad17d/userdata/shm DeviceMajor:0 DeviceMinor:241 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/f13bc62a-1894-472c-95de-262afa399b38/volumes/kubernetes.io~projected/kube-api-access-qhvbn DeviceMajor:0 DeviceMinor:316 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-336 DeviceMajor:0 DeviceMinor:336 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-279 DeviceMajor:0 DeviceMinor:279 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-300 DeviceMajor:0 DeviceMinor:300 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-304 DeviceMajor:0 DeviceMinor:304 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a32ed9582736eace8657e4c5e9e1521b6a93be7e5fb2ff212d0a71e3a22fc402/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-142 DeviceMajor:0 DeviceMinor:142 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4104ba06-fba8-4c3c-b876-cca5087a7d0a/volumes/kubernetes.io~projected/kube-api-access-hldt6 DeviceMajor:0 DeviceMinor:218 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/c734ebf9-eba8-4e14-b20d-2d78dd65b10d/volumes/kubernetes.io~projected/kube-api-access-z5s89 DeviceMajor:0 DeviceMinor:224 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-308 DeviceMajor:0 DeviceMinor:308 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:023f6b65d0f9f8a MacAddress:3e:bf:aa:40:b7:dc Speed:10000 Mtu:8901} {Name:115e6d16cbb3c78 MacAddress:a6:81:8a:33:4d:41 Speed:10000 Mtu:8901} {Name:1b3a1845b16b578 MacAddress:a2:4f:2e:4b:40:5d Speed:10000 Mtu:8901} {Name:61e6f367bdfe0ea MacAddress:e2:d8:7c:37:86:6a Speed:10000 Mtu:8901} {Name:7c6136cafd20d7d MacAddress:22:b9:71:24:0f:f4 Speed:10000 Mtu:8901} {Name:8110977887cda9c MacAddress:b6:a0:b3:9f:80:fd Speed:10000 Mtu:8901} {Name:81bc093b082071b MacAddress:ae:77:39:c5:57:73 Speed:10000 Mtu:8901} {Name:8d5c8c577bce94d MacAddress:2e:07:7b:2f:1d:97 Speed:10000 Mtu:8901} {Name:9e4560e0b325aa7 MacAddress:aa:74:53:4c:04:60 Speed:10000 Mtu:8901} {Name:a1f16ffaf583d84 MacAddress:52:57:6e:b4:64:41 Speed:10000 Mtu:8901} {Name:a847b89d5813845 MacAddress:e6:42:ae:5f:d9:93 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:ff:e9:2f:c4:23 Speed:0 Mtu:9001} {Name:br-int MacAddress:86:af:af:5c:6a:3e Speed:0 Mtu:8901} {Name:d187de26ca97f37 MacAddress:ea:63:c3:35:29:89 Speed:10000 Mtu:8901} {Name:e8964c5013096f6 MacAddress:9e:80:02:e2:05:29 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:ff:e9:2f:c4:23 Speed:0 Mtu:9001} {Name:genev_sys_6081 MacAddress:9e:a3:fb:06:9d:1f Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:84:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:5e:5e:7c:af:96:85 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812011520 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Oct 31 07:30:26.189447 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.186804 4935 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Oct 31 07:30:26.189447 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.186847 4935 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Oct 31 07:30:26.189447 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.187182 4935 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Oct 31 07:30:26.189447 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.187218 4935 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-48.us-east-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Oct 31 07:30:26.189447 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.187424 4935 topology_manager.go:143] "Creating topology manager with none policy" Oct 31 07:30:26.189447 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.187436 4935 container_manager_linux.go:311] "Creating device plugin manager" Oct 31 07:30:26.189447 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.187454 4935 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 31 07:30:26.189447 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.187480 4935 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 31 07:30:26.189447 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.187505 4935 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager" Oct 31 07:30:26.189447 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.187720 4935 state_mem.go:41] "Initialized" logger="CPUManager state memory" Oct 31 07:30:26.189447 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.187853 4935 server.go:1235] "Using root directory" path="/var/lib/kubelet" Oct 31 07:30:26.189447 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.187899 4935 kubelet.go:493] "Attempting to sync node with API server" Oct 31 07:30:26.189447 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.187913 4935 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests" Oct 31 07:30:26.189447 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.187928 4935 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Oct 31 07:30:26.189447 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.187941 4935 kubelet.go:405] "Adding apiserver pod source" Oct 31 07:30:26.189447 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.187952 4935 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Oct 31 07:30:26.189447 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.188899 4935 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint" Oct 31 07:30:26.189447 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.188912 4935 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 31 07:30:26.189447 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.189427 4935 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1" Oct 31 07:30:26.191586 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.191570 4935 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint" Oct 31 07:30:26.191586 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.191583 4935 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 31 07:30:26.191705 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.191683 4935 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem" Oct 31 07:30:26.191870 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.191859 4935 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Oct 31 07:30:26.191902 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.191887 4935 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Oct 31 07:30:26.191984 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.191974 4935 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Oct 31 07:30:26.192029 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.191990 4935 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Oct 31 07:30:26.192029 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.191996 4935 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Oct 31 07:30:26.192029 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.192001 4935 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Oct 31 07:30:26.192029 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.192007 4935 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Oct 31 07:30:26.192029 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.192013 4935 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Oct 31 07:30:26.192029 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.192022 4935 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Oct 31 07:30:26.192029 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.192030 4935 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Oct 31 07:30:26.192260 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.192037 4935 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Oct 31 07:30:26.192260 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.192043 4935 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Oct 31 07:30:26.192260 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.192056 4935 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Oct 31 07:30:26.192260 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.192065 4935 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Oct 31 07:30:26.192260 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.192093 4935 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Oct 31 07:30:26.192260 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.192104 4935 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Oct 31 07:30:26.195785 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.195771 4935 server.go:1263] "Started kubelet" Oct 31 07:30:26.196300 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.195845 4935 server.go:182] "Starting to listen" address="0.0.0.0" port=10250 Oct 31 07:30:26.196300 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.195854 4935 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Oct 31 07:30:26.196300 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.195917 4935 server_v1.go:49] "podresources" method="list" useActivePods=true Oct 31 07:30:26.196975 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.196939 4935 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Oct 31 07:30:26.197123 ip-10-0-130-48 systemd[1]: Started Kubernetes Kubelet. Oct 31 07:30:26.199417 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.198004 4935 server.go:317] "Adding debug handlers to kubelet server" Oct 31 07:30:26.203617 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.203600 4935 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer" Oct 31 07:30:26.203712 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.203640 4935 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Oct 31 07:30:26.203712 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.203678 4935 volume_manager.go:309] "The desired_state_of_world populator starts" Oct 31 07:30:26.203712 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.203692 4935 volume_manager.go:311] "Starting Kubelet Volume Manager" Oct 31 07:30:26.203913 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.203719 4935 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Oct 31 07:30:26.203913 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.203716 4935 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-30 07:24:47 +0000 UTC" deadline="2028-05-22 14:06:36.406484731 +0000 UTC" Oct 31 07:30:26.203913 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.203748 4935 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13662h36m10.202740429s" Oct 31 07:30:26.203913 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:26.203784 4935 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-48.us-east-2.compute.internal\" not found" Oct 31 07:30:26.205756 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.205737 4935 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Oct 31 07:30:26.206594 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.206560 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="00a0d978-8020-46df-932d-b0dd22f946f9" volumeName="kubernetes.io/projected/00a0d978-8020-46df-932d-b0dd22f946f9-kube-api-access-2jhpf" seLinuxMountContext="" Oct 31 07:30:26.206666 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.206597 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4104ba06-fba8-4c3c-b876-cca5087a7d0a" volumeName="kubernetes.io/secret/4104ba06-fba8-4c3c-b876-cca5087a7d0a-image-registry-private-configuration" seLinuxMountContext="" Oct 31 07:30:26.206666 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.206612 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="58bc3ea1-f58c-4891-a049-1289a99be055" volumeName="kubernetes.io/projected/58bc3ea1-f58c-4891-a049-1289a99be055-kube-api-access-4q9t4" seLinuxMountContext="" Oct 31 07:30:26.206666 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.206625 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d25f2854-fd38-44e7-ad7e-da744d57e671" volumeName="kubernetes.io/secret/d25f2854-fd38-44e7-ad7e-da744d57e671-agent-certs" seLinuxMountContext="" Oct 31 07:30:26.206666 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.206637 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e" volumeName="kubernetes.io/configmap/d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e-samples-operator-config" seLinuxMountContext="" Oct 31 07:30:26.206666 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.206649 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e00eb9e3-0d3c-4541-b238-4130fe1de2f1" volumeName="kubernetes.io/projected/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-kube-api-access-mrvrg" seLinuxMountContext="" Oct 31 07:30:26.206666 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.206664 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b6997c76-e103-4c3f-b39d-a6a17fb98270" volumeName="kubernetes.io/configmap/b6997c76-e103-4c3f-b39d-a6a17fb98270-config" seLinuxMountContext="" Oct 31 07:30:26.206863 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.206676 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e00eb9e3-0d3c-4541-b238-4130fe1de2f1" volumeName="kubernetes.io/configmap/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-ovnkube-config" seLinuxMountContext="" Oct 31 07:30:26.206863 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.206692 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ed522b38-611a-452a-b201-3ca416428d42" volumeName="kubernetes.io/projected/ed522b38-611a-452a-b201-3ca416428d42-kube-api-access-w4tdk" seLinuxMountContext="" Oct 31 07:30:26.206863 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.206703 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f13bc62a-1894-472c-95de-262afa399b38" volumeName="kubernetes.io/empty-dir/f13bc62a-1894-472c-95de-262afa399b38-data-volume" seLinuxMountContext="" Oct 31 07:30:26.206863 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.206717 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="28be803a-a932-4009-ad61-b356366821ef" volumeName="kubernetes.io/secret/28be803a-a932-4009-ad61-b356366821ef-default-certificate" seLinuxMountContext="" Oct 31 07:30:26.206863 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.206741 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="53c36e19-9ef6-47a2-9bc2-387115539850" volumeName="kubernetes.io/empty-dir/53c36e19-9ef6-47a2-9bc2-387115539850-tmp-dir" seLinuxMountContext="" Oct 31 07:30:26.206863 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.206752 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b6997c76-e103-4c3f-b39d-a6a17fb98270" volumeName="kubernetes.io/secret/b6997c76-e103-4c3f-b39d-a6a17fb98270-serving-cert" seLinuxMountContext="" Oct 31 07:30:26.206863 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.206764 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e00eb9e3-0d3c-4541-b238-4130fe1de2f1" volumeName="kubernetes.io/configmap/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-ovnkube-script-lib" seLinuxMountContext="" Oct 31 07:30:26.206863 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.206779 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="28ec9621-7ec5-421c-a88b-51bb20d3a7bf" volumeName="kubernetes.io/projected/28ec9621-7ec5-421c-a88b-51bb20d3a7bf-kube-api-access-8mqks" seLinuxMountContext="" Oct 31 07:30:26.206863 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.206788 4935 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161" Oct 31 07:30:26.206863 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.206791 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="396e1c5e-3700-4fe6-b367-54ad2a244225" volumeName="kubernetes.io/projected/396e1c5e-3700-4fe6-b367-54ad2a244225-kube-api-access-g76kv" seLinuxMountContext="" Oct 31 07:30:26.206863 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.206832 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="58bc3ea1-f58c-4891-a049-1289a99be055" volumeName="kubernetes.io/empty-dir/58bc3ea1-f58c-4891-a049-1289a99be055-tmp-dir" seLinuxMountContext="" Oct 31 07:30:26.206863 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.206854 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="acda6b08-2faa-4836-888e-d4ecf324cb1d" volumeName="kubernetes.io/projected/acda6b08-2faa-4836-888e-d4ecf324cb1d-kube-api-access-tvpkt" seLinuxMountContext="" Oct 31 07:30:26.207243 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.206869 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b4dd8884-1d6e-4abe-ad6c-3c947623bb60" volumeName="kubernetes.io/secret/b4dd8884-1d6e-4abe-ad6c-3c947623bb60-serving-cert" seLinuxMountContext="" Oct 31 07:30:26.207243 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.206882 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="53c36e19-9ef6-47a2-9bc2-387115539850" volumeName="kubernetes.io/projected/53c36e19-9ef6-47a2-9bc2-387115539850-kube-api-access-zzvgd" seLinuxMountContext="" Oct 31 07:30:26.207243 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.206868 4935 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Oct 31 07:30:26.207243 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.206908 4935 factory.go:55] Registering systemd factory Oct 31 07:30:26.207243 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.206917 4935 factory.go:223] Registration of the systemd container factory successfully Oct 31 07:30:26.207243 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.206895 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5e743234-2b5f-4c91-9b06-189f546cbce4" volumeName="kubernetes.io/projected/5e743234-2b5f-4c91-9b06-189f546cbce4-kube-api-access-djqrt" seLinuxMountContext="" Oct 31 07:30:26.207243 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.206984 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ac4bf75a-377c-49b2-818a-7d2436c21e9c" volumeName="kubernetes.io/projected/ac4bf75a-377c-49b2-818a-7d2436c21e9c-kube-api-access-xv5b5" seLinuxMountContext="" Oct 31 07:30:26.207243 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.207003 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e84bef13-a137-49f7-931d-9f94952dacfc" volumeName="kubernetes.io/projected/e84bef13-a137-49f7-931d-9f94952dacfc-kube-api-access-xksf5" seLinuxMountContext="" Oct 31 07:30:26.207243 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.207018 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="046b1de8-d709-4729-9aec-391a2b3294d5" volumeName="kubernetes.io/empty-dir/046b1de8-d709-4729-9aec-391a2b3294d5-tmp" seLinuxMountContext="" Oct 31 07:30:26.207243 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.207058 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4104ba06-fba8-4c3c-b876-cca5087a7d0a" volumeName="kubernetes.io/configmap/4104ba06-fba8-4c3c-b876-cca5087a7d0a-registry-certificates" seLinuxMountContext="" Oct 31 07:30:26.207243 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.207072 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4104ba06-fba8-4c3c-b876-cca5087a7d0a" volumeName="kubernetes.io/projected/4104ba06-fba8-4c3c-b876-cca5087a7d0a-kube-api-access-hldt6" seLinuxMountContext="" Oct 31 07:30:26.207243 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.207085 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6fae9634-7abe-40c6-a892-9eb58166e9ad" volumeName="kubernetes.io/projected/6fae9634-7abe-40c6-a892-9eb58166e9ad-kube-api-access-wsvzt" seLinuxMountContext="" Oct 31 07:30:26.207243 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.207098 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ea9beb38-1491-4cf0-83ca-fd79935fa6a9" volumeName="kubernetes.io/projected/ea9beb38-1491-4cf0-83ca-fd79935fa6a9-kube-api-access-wrj54" seLinuxMountContext="" Oct 31 07:30:26.207243 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.207119 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ed522b38-611a-452a-b201-3ca416428d42" volumeName="kubernetes.io/configmap/ed522b38-611a-452a-b201-3ca416428d42-cni-sysctl-allowlist" seLinuxMountContext="" Oct 31 07:30:26.207243 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.207132 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="00a0d978-8020-46df-932d-b0dd22f946f9" volumeName="kubernetes.io/secret/00a0d978-8020-46df-932d-b0dd22f946f9-cert" seLinuxMountContext="" Oct 31 07:30:26.207243 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.207145 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="297b35d7-aa65-49c7-8460-ac3f9860248c" volumeName="kubernetes.io/secret/297b35d7-aa65-49c7-8460-ac3f9860248c-original-pull-secret" seLinuxMountContext="" Oct 31 07:30:26.207243 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.207157 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4104ba06-fba8-4c3c-b876-cca5087a7d0a" volumeName="kubernetes.io/secret/4104ba06-fba8-4c3c-b876-cca5087a7d0a-installation-pull-secrets" seLinuxMountContext="" Oct 31 07:30:26.207243 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.207201 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="54296c9d-1c43-44cd-bfb6-d81fe892484c" volumeName="kubernetes.io/projected/54296c9d-1c43-44cd-bfb6-d81fe892484c-kube-api-access-lx427" seLinuxMountContext="" Oct 31 07:30:26.207243 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.207224 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b4dd8884-1d6e-4abe-ad6c-3c947623bb60" volumeName="kubernetes.io/empty-dir/b4dd8884-1d6e-4abe-ad6c-3c947623bb60-snapshots" seLinuxMountContext="" Oct 31 07:30:26.208138 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.207259 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c734ebf9-eba8-4e14-b20d-2d78dd65b10d" volumeName="kubernetes.io/secret/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-image-registry-private-configuration" seLinuxMountContext="" Oct 31 07:30:26.208138 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.207272 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c734ebf9-eba8-4e14-b20d-2d78dd65b10d" volumeName="kubernetes.io/secret/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-installation-pull-secrets" seLinuxMountContext="" Oct 31 07:30:26.208138 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.207284 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f13bc62a-1894-472c-95de-262afa399b38" volumeName="kubernetes.io/projected/f13bc62a-1894-472c-95de-262afa399b38-kube-api-access-qhvbn" seLinuxMountContext="" Oct 31 07:30:26.208138 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.207295 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4104ba06-fba8-4c3c-b876-cca5087a7d0a" volumeName="kubernetes.io/empty-dir/4104ba06-fba8-4c3c-b876-cca5087a7d0a-ca-trust-extracted" seLinuxMountContext="" Oct 31 07:30:26.208138 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.207325 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c734ebf9-eba8-4e14-b20d-2d78dd65b10d" volumeName="kubernetes.io/configmap/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-trusted-ca" seLinuxMountContext="" Oct 31 07:30:26.208138 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.207339 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c734ebf9-eba8-4e14-b20d-2d78dd65b10d" volumeName="kubernetes.io/projected/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-bound-sa-token" seLinuxMountContext="" Oct 31 07:30:26.208138 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.207397 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e00eb9e3-0d3c-4541-b238-4130fe1de2f1" volumeName="kubernetes.io/configmap/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-env-overrides" seLinuxMountContext="" Oct 31 07:30:26.208138 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.207415 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6ae10413-8f49-4dfe-8aad-00dcb9d390c8" volumeName="kubernetes.io/projected/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-kube-api-access-997f5" seLinuxMountContext="" Oct 31 07:30:26.208138 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.207427 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="396e1c5e-3700-4fe6-b367-54ad2a244225" volumeName="kubernetes.io/configmap/396e1c5e-3700-4fe6-b367-54ad2a244225-trusted-ca" seLinuxMountContext="" Oct 31 07:30:26.208138 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.207164 4935 factory.go:153] Registering CRI-O factory Oct 31 07:30:26.208138 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.207441 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="046b1de8-d709-4729-9aec-391a2b3294d5" volumeName="kubernetes.io/projected/046b1de8-d709-4729-9aec-391a2b3294d5-kube-api-access-76x7d" seLinuxMountContext="" Oct 31 07:30:26.208138 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.207462 4935 factory.go:223] Registration of the crio container factory successfully Oct 31 07:30:26.208138 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.207483 4935 factory.go:103] Registering Raw factory Oct 31 07:30:26.208138 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.207498 4935 manager.go:1199] Started watching for new ooms in manager Oct 31 07:30:26.208138 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.207462 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="236238a7-255d-40f7-a99b-d65903b3bcb7" volumeName="kubernetes.io/configmap/236238a7-255d-40f7-a99b-d65903b3bcb7-serviceca" seLinuxMountContext="" Oct 31 07:30:26.208138 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.207603 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="28be803a-a932-4009-ad61-b356366821ef" volumeName="kubernetes.io/projected/28be803a-a932-4009-ad61-b356366821ef-kube-api-access-cf9zh" seLinuxMountContext="" Oct 31 07:30:26.208138 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.207623 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="396e1c5e-3700-4fe6-b367-54ad2a244225" volumeName="kubernetes.io/secret/396e1c5e-3700-4fe6-b367-54ad2a244225-serving-cert" seLinuxMountContext="" Oct 31 07:30:26.208138 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.207638 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d25f2854-fd38-44e7-ad7e-da744d57e671" volumeName="kubernetes.io/configmap/d25f2854-fd38-44e7-ad7e-da744d57e671-konnectivity-ca" seLinuxMountContext="" Oct 31 07:30:26.208138 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.207653 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f3bf19fd-bc58-4dff-a1a8-4f61381e1fde" volumeName="kubernetes.io/secret/f3bf19fd-bc58-4dff-a1a8-4f61381e1fde-serving-cert" seLinuxMountContext="" Oct 31 07:30:26.208138 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.207667 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="046b1de8-d709-4729-9aec-391a2b3294d5" volumeName="kubernetes.io/empty-dir/046b1de8-d709-4729-9aec-391a2b3294d5-etc-tuned" seLinuxMountContext="" Oct 31 07:30:26.208138 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.207680 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="28be803a-a932-4009-ad61-b356366821ef" volumeName="kubernetes.io/secret/28be803a-a932-4009-ad61-b356366821ef-stats-auth" seLinuxMountContext="" Oct 31 07:30:26.208138 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.207693 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="563d3b8b-0040-439e-b0b2-e858eed5abf6" volumeName="kubernetes.io/configmap/563d3b8b-0040-439e-b0b2-e858eed5abf6-nginx-conf" seLinuxMountContext="" Oct 31 07:30:26.208138 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.207707 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5e743234-2b5f-4c91-9b06-189f546cbce4" volumeName="kubernetes.io/configmap/5e743234-2b5f-4c91-9b06-189f546cbce4-telemetry-config" seLinuxMountContext="" Oct 31 07:30:26.208138 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.207719 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6ae10413-8f49-4dfe-8aad-00dcb9d390c8" volumeName="kubernetes.io/configmap/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-cni-binary-copy" seLinuxMountContext="" Oct 31 07:30:26.208138 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.207733 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b4dd8884-1d6e-4abe-ad6c-3c947623bb60" volumeName="kubernetes.io/configmap/b4dd8884-1d6e-4abe-ad6c-3c947623bb60-service-ca-bundle" seLinuxMountContext="" Oct 31 07:30:26.208138 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.207744 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b4dd8884-1d6e-4abe-ad6c-3c947623bb60" volumeName="kubernetes.io/configmap/b4dd8884-1d6e-4abe-ad6c-3c947623bb60-trusted-ca-bundle" seLinuxMountContext="" Oct 31 07:30:26.208138 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.207768 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b6997c76-e103-4c3f-b39d-a6a17fb98270" volumeName="kubernetes.io/projected/b6997c76-e103-4c3f-b39d-a6a17fb98270-kube-api-access-p76jj" seLinuxMountContext="" Oct 31 07:30:26.208138 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.207781 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="236238a7-255d-40f7-a99b-d65903b3bcb7" volumeName="kubernetes.io/projected/236238a7-255d-40f7-a99b-d65903b3bcb7-kube-api-access-ftf9v" seLinuxMountContext="" Oct 31 07:30:26.208138 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.207795 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="43c59587-4ff8-4546-9c0a-6161ece201d0" volumeName="kubernetes.io/projected/43c59587-4ff8-4546-9c0a-6161ece201d0-kube-api-access-82jqk" seLinuxMountContext="" Oct 31 07:30:26.208138 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.207820 4935 manager.go:319] Starting recovery of all containers Oct 31 07:30:26.208138 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.207806 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d8b28b9f-37a2-4859-87fd-d27bb37d832b" volumeName="kubernetes.io/projected/d8b28b9f-37a2-4859-87fd-d27bb37d832b-kube-api-access-2ckzf" seLinuxMountContext="" Oct 31 07:30:26.208138 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.207840 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e00eb9e3-0d3c-4541-b238-4130fe1de2f1" volumeName="kubernetes.io/secret/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-ovn-node-metrics-cert" seLinuxMountContext="" Oct 31 07:30:26.208138 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.207852 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ed522b38-611a-452a-b201-3ca416428d42" volumeName="kubernetes.io/configmap/ed522b38-611a-452a-b201-3ca416428d42-whereabouts-flatfile-configmap" seLinuxMountContext="" Oct 31 07:30:26.208138 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.207865 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ed522b38-611a-452a-b201-3ca416428d42" volumeName="kubernetes.io/configmap/ed522b38-611a-452a-b201-3ca416428d42-cni-binary-copy" seLinuxMountContext="" Oct 31 07:30:26.208138 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.207876 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="396e1c5e-3700-4fe6-b367-54ad2a244225" volumeName="kubernetes.io/configmap/396e1c5e-3700-4fe6-b367-54ad2a244225-config" seLinuxMountContext="" Oct 31 07:30:26.208138 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.207890 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e" volumeName="kubernetes.io/projected/d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e-kube-api-access-nlm6c" seLinuxMountContext="" Oct 31 07:30:26.208138 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.207906 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4104ba06-fba8-4c3c-b876-cca5087a7d0a" volumeName="kubernetes.io/projected/4104ba06-fba8-4c3c-b876-cca5087a7d0a-bound-sa-token" seLinuxMountContext="" Oct 31 07:30:26.208138 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.207919 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3cbf4695-0536-45d5-bc48-be56861b87b8" volumeName="kubernetes.io/projected/3cbf4695-0536-45d5-bc48-be56861b87b8-kube-api-access-v4zgz" seLinuxMountContext="" Oct 31 07:30:26.208138 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.207932 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4104ba06-fba8-4c3c-b876-cca5087a7d0a" volumeName="kubernetes.io/configmap/4104ba06-fba8-4c3c-b876-cca5087a7d0a-trusted-ca" seLinuxMountContext="" Oct 31 07:30:26.208138 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.207943 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b4dd8884-1d6e-4abe-ad6c-3c947623bb60" volumeName="kubernetes.io/projected/b4dd8884-1d6e-4abe-ad6c-3c947623bb60-kube-api-access-9k9q8" seLinuxMountContext="" Oct 31 07:30:26.208138 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.207956 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c734ebf9-eba8-4e14-b20d-2d78dd65b10d" volumeName="kubernetes.io/empty-dir/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-ca-trust-extracted" seLinuxMountContext="" Oct 31 07:30:26.208138 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.207968 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c734ebf9-eba8-4e14-b20d-2d78dd65b10d" volumeName="kubernetes.io/projected/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-kube-api-access-z5s89" seLinuxMountContext="" Oct 31 07:30:26.208138 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.207981 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f3bf19fd-bc58-4dff-a1a8-4f61381e1fde" volumeName="kubernetes.io/projected/f3bf19fd-bc58-4dff-a1a8-4f61381e1fde-kube-api-access-cb5dx" seLinuxMountContext="" Oct 31 07:30:26.208138 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.207994 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="58bc3ea1-f58c-4891-a049-1289a99be055" volumeName="kubernetes.io/configmap/58bc3ea1-f58c-4891-a049-1289a99be055-config-volume" seLinuxMountContext="" Oct 31 07:30:26.208138 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.208005 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ac4bf75a-377c-49b2-818a-7d2436c21e9c" volumeName="kubernetes.io/configmap/ac4bf75a-377c-49b2-818a-7d2436c21e9c-iptables-alerter-script" seLinuxMountContext="" Oct 31 07:30:26.208138 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.208017 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6ae10413-8f49-4dfe-8aad-00dcb9d390c8" volumeName="kubernetes.io/configmap/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-multus-daemon-config" seLinuxMountContext="" Oct 31 07:30:26.208138 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.208028 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f3bf19fd-bc58-4dff-a1a8-4f61381e1fde" volumeName="kubernetes.io/configmap/f3bf19fd-bc58-4dff-a1a8-4f61381e1fde-config" seLinuxMountContext="" Oct 31 07:30:26.208138 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.208041 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b4dd8884-1d6e-4abe-ad6c-3c947623bb60" volumeName="kubernetes.io/empty-dir/b4dd8884-1d6e-4abe-ad6c-3c947623bb60-tmp" seLinuxMountContext="" Oct 31 07:30:26.208138 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.208052 4935 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c734ebf9-eba8-4e14-b20d-2d78dd65b10d" volumeName="kubernetes.io/configmap/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-registry-certificates" seLinuxMountContext="" Oct 31 07:30:26.208138 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.208065 4935 reconstruct.go:97] "Volume reconstruction finished" Oct 31 07:30:26.208138 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.208073 4935 reconciler.go:29] "Reconciler: start to sync state" Oct 31 07:30:26.210578 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.208278 4935 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Oct 31 07:30:26.215310 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.214533 4935 reconstruct.go:205] "DevicePaths of reconstructed volumes updated" Oct 31 07:30:26.220501 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.220481 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jztcz_396e1c5e-3700-4fe6-b367-54ad2a244225/console-operator/1.log" Oct 31 07:30:26.220577 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.220514 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jztcz_396e1c5e-3700-4fe6-b367-54ad2a244225/console-operator/1.log" Oct 31 07:30:26.232547 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.232503 4935 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Oct 31 07:30:26.236375 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.236235 4935 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Oct 31 07:30:26.236451 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.236381 4935 status_manager.go:249] "Starting to sync pod status with apiserver" Oct 31 07:30:26.236451 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.236404 4935 kubelet.go:2521] "Starting kubelet main sync loop" Oct 31 07:30:26.236549 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:26.236446 4935 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Oct 31 07:30:26.240034 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.240014 4935 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Oct 31 07:30:26.246224 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.246203 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jztcz_396e1c5e-3700-4fe6-b367-54ad2a244225/console-operator/1.log" Oct 31 07:30:26.246308 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.246228 4935 generic.go:356] "Generic (PLEG): container finished" podID="396e1c5e-3700-4fe6-b367-54ad2a244225" containerID="25aec100393d03aba086bd02f13db3038b24b6ae07422807f4c23edbb520f3f5" exitCode=255 Oct 31 07:30:26.260136 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.260109 4935 generic.go:356] "Generic (PLEG): container finished" podID="11f3ad8c54222373dd06005938b97a96" containerID="f1629f237dd7db0cfceeb4f1731b6ccb4f8c325b5187830e81989b689220cb36" exitCode=0 Oct 31 07:30:26.279089 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.279069 4935 generic.go:356] "Generic (PLEG): container finished" podID="ed522b38-611a-452a-b201-3ca416428d42" containerID="ec55eab7889a3e6a63fcd29bdbc907e6fae3e3d265b5fb4b4f6ca295ebd8023e" exitCode=0 Oct 31 07:30:26.279189 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.279090 4935 generic.go:356] "Generic (PLEG): container finished" podID="ed522b38-611a-452a-b201-3ca416428d42" containerID="0857706c3a24d3436bc55abbea6cc7e47995262128b08cec8803612caac184d0" exitCode=0 Oct 31 07:30:26.279189 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.279102 4935 generic.go:356] "Generic (PLEG): container finished" podID="ed522b38-611a-452a-b201-3ca416428d42" containerID="050e01d8523ffaf42d5f2babfd83abee1dc9af7a5dabef0ace9cbce09bbf1f82" exitCode=0 Oct 31 07:30:26.279189 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.279111 4935 generic.go:356] "Generic (PLEG): container finished" podID="ed522b38-611a-452a-b201-3ca416428d42" containerID="786e5a0a40cb4f1ba47f7ed2e5bb820c8bfdac16ac667cd7f63d40dfb804f043" exitCode=0 Oct 31 07:30:26.279189 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.279118 4935 generic.go:356] "Generic (PLEG): container finished" podID="ed522b38-611a-452a-b201-3ca416428d42" containerID="d255068b661bc489dc9facfded815b3b2fdcbc435885f24ac5fa7c86bfe68dce" exitCode=0 Oct 31 07:30:26.279189 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.279125 4935 generic.go:356] "Generic (PLEG): container finished" podID="ed522b38-611a-452a-b201-3ca416428d42" containerID="81b3078f11cd9615e6f1318d1b692b1f4c5ae4de91dc4671bd11c0374748a756" exitCode=0 Oct 31 07:30:26.297925 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.297910 4935 manager.go:324] Recovery completed Oct 31 07:30:26.317842 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.317795 4935 cpu_manager.go:226] "Starting" policy="none" Oct 31 07:30:26.317842 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.317809 4935 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s" Oct 31 07:30:26.317842 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.317824 4935 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory" Oct 31 07:30:26.317985 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.317940 4935 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet="" Oct 31 07:30:26.317985 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.317950 4935 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={} Oct 31 07:30:26.317985 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.317964 4935 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint" Oct 31 07:30:26.317985 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.317969 4935 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet="" Oct 31 07:30:26.317985 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.317974 4935 policy_none.go:50] "Start" Oct 31 07:30:26.317985 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.317982 4935 memory_manager.go:187] "Starting memorymanager" policy="None" Oct 31 07:30:26.318148 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.317991 4935 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Oct 31 07:30:26.318148 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.318102 4935 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint" Oct 31 07:30:26.318148 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.318109 4935 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint" Oct 31 07:30:26.318148 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.318114 4935 policy_none.go:44] "Start" Oct 31 07:30:26.322325 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.322301 4935 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"} Oct 31 07:30:26.324846 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.324833 4935 manager.go:341] "Starting Device Plugin manager" Oct 31 07:30:26.325344 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:26.324857 4935 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Oct 31 07:30:26.325344 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.325363 4935 server.go:87] "Starting device plugin registration server" Oct 31 07:30:26.325604 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.325595 4935 eviction_manager.go:194] "Eviction manager: starting control loop" Oct 31 07:30:26.325638 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.325608 4935 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Oct 31 07:30:26.325729 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.325714 4935 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Oct 31 07:30:26.325833 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.325804 4935 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Oct 31 07:30:26.325886 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.325843 4935 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts" Oct 31 07:30:26.325886 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.325847 4935 plugin_manager.go:121] "Starting Kubelet Plugin Manager" Oct 31 07:30:26.325982 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.325883 4935 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-31T07:30:26.325830174Z","UUID":"7b5e6e15-f8cf-482a-9450-9b61dde4246f","Handler":null,"Name":"","Endpoint":""} Oct 31 07:30:26.327294 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:26.327275 4935 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Oct 31 07:30:26.327405 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.327309 4935 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Oct 31 07:30:26.327405 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.327338 4935 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Oct 31 07:30:26.336832 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.336797 4935 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-48.us-east-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-48.us-east-2.compute.internal"] Oct 31 07:30:26.337485 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.337443 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-48.us-east-2.compute.internal" event={"ID":"11f3ad8c54222373dd06005938b97a96","Type":"ContainerStarted","Data":"73675a98041d2bd87e79f8e8424f260d98e196b1a934e51b37625e26c91fdad8"} Oct 31 07:30:26.337570 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.337492 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-48.us-east-2.compute.internal" event={"ID":"11f3ad8c54222373dd06005938b97a96","Type":"ContainerDied","Data":"f1629f237dd7db0cfceeb4f1731b6ccb4f8c325b5187830e81989b689220cb36"} Oct 31 07:30:26.337570 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.337511 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-48.us-east-2.compute.internal" event={"ID":"11f3ad8c54222373dd06005938b97a96","Type":"ContainerStarted","Data":"c4b192911d11550f7de0ff5884848f30c9f162951df39fa0707f7b8b1f85bc45"} Oct 31 07:30:26.337684 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.337596 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-48.us-east-2.compute.internal" event={"ID":"8927b882957b5b07fb0714896c6f1a7f","Type":"ContainerStarted","Data":"067c9d6db4729beb6980f4858378e6c88f4a10834ca4d7804f322dceae6f7f0f"} Oct 31 07:30:26.337684 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.337601 4935 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-48.us-east-2.compute.internal" Oct 31 07:30:26.337684 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.337616 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-48.us-east-2.compute.internal" event={"ID":"8927b882957b5b07fb0714896c6f1a7f","Type":"ContainerStarted","Data":"c8e97bb91fa7a4579a6eb8f24f665103772028210f2a68df470a03c9f5841638"} Oct 31 07:30:26.337684 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.337598 4935 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-48.us-east-2.compute.internal" Oct 31 07:30:26.351166 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.351144 4935 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 31 07:30:26.351242 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:26.351199 4935 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-130-48.us-east-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-48.us-east-2.compute.internal" Oct 31 07:30:26.351615 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.351596 4935 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 31 07:30:26.351701 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:26.351634 4935 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-130-48.us-east-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-48.us-east-2.compute.internal" Oct 31 07:30:26.415475 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.415444 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/11f3ad8c54222373dd06005938b97a96-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-48.us-east-2.compute.internal\" (UID: \"11f3ad8c54222373dd06005938b97a96\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-48.us-east-2.compute.internal" Oct 31 07:30:26.415475 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.415477 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/11f3ad8c54222373dd06005938b97a96-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-48.us-east-2.compute.internal\" (UID: \"11f3ad8c54222373dd06005938b97a96\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-48.us-east-2.compute.internal" Oct 31 07:30:26.415647 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.415496 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8927b882957b5b07fb0714896c6f1a7f-config\") pod \"kube-apiserver-proxy-ip-10-0-130-48.us-east-2.compute.internal\" (UID: \"8927b882957b5b07fb0714896c6f1a7f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-48.us-east-2.compute.internal" Oct 31 07:30:26.426537 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.426514 4935 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 31 07:30:26.427775 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.427758 4935 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-48.us-east-2.compute.internal" event="NodeHasSufficientMemory" Oct 31 07:30:26.427844 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.427798 4935 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-48.us-east-2.compute.internal" event="NodeHasNoDiskPressure" Oct 31 07:30:26.427844 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.427813 4935 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-48.us-east-2.compute.internal" event="NodeHasSufficientPID" Oct 31 07:30:26.427914 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.427875 4935 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-130-48.us-east-2.compute.internal" Oct 31 07:30:26.441360 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.441330 4935 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-130-48.us-east-2.compute.internal" Oct 31 07:30:26.441436 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.441403 4935 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-130-48.us-east-2.compute.internal" Oct 31 07:30:26.515855 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.515828 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/11f3ad8c54222373dd06005938b97a96-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-48.us-east-2.compute.internal\" (UID: \"11f3ad8c54222373dd06005938b97a96\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-48.us-east-2.compute.internal" Oct 31 07:30:26.515976 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.515879 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/11f3ad8c54222373dd06005938b97a96-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-48.us-east-2.compute.internal\" (UID: \"11f3ad8c54222373dd06005938b97a96\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-48.us-east-2.compute.internal" Oct 31 07:30:26.515976 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.515905 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8927b882957b5b07fb0714896c6f1a7f-config\") pod \"kube-apiserver-proxy-ip-10-0-130-48.us-east-2.compute.internal\" (UID: \"8927b882957b5b07fb0714896c6f1a7f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-48.us-east-2.compute.internal" Oct 31 07:30:26.515976 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.515836 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/11f3ad8c54222373dd06005938b97a96-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-48.us-east-2.compute.internal\" (UID: \"11f3ad8c54222373dd06005938b97a96\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-48.us-east-2.compute.internal" Oct 31 07:30:26.515976 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.515953 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8927b882957b5b07fb0714896c6f1a7f-config\") pod \"kube-apiserver-proxy-ip-10-0-130-48.us-east-2.compute.internal\" (UID: \"8927b882957b5b07fb0714896c6f1a7f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-48.us-east-2.compute.internal" Oct 31 07:30:26.515976 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:26.515970 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/11f3ad8c54222373dd06005938b97a96-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-48.us-east-2.compute.internal\" (UID: \"11f3ad8c54222373dd06005938b97a96\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-48.us-east-2.compute.internal" Oct 31 07:30:27.188762 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.188696 4935 apiserver.go:52] "Watching apiserver" Oct 31 07:30:27.199807 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.199786 4935 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Oct 31 07:30:27.200586 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.200564 4935 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-282kr","openshift-network-diagnostics/network-check-target-ncnd7","openshift-cluster-node-tuning-operator/tuned-sqxmw","openshift-image-registry/image-registry-69fbc98d44-fn7fh","openshift-ingress/router-default-5549647ff-cxcdt","openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-46qvv","kube-system/kas-connection-checker-685895c7c9-vhcq9","openshift-ingress-canary/ingress-canary-22cmd","openshift-multus/network-metrics-daemon-7thxr","openshift-console-operator/console-operator-7cd56b8dbc-jztcz","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-48.us-east-2.compute.internal","kube-system/global-pull-secret-syncer-8hpcb","openshift-multus/multus-additional-cni-plugins-ktzg4","openshift-insights/insights-operator-7c4986d449-hz58r","openshift-insights/insights-runtime-extractor-mvmr5","openshift-kube-storage-version-migrator/migrator-66df5d496b-dwlls","openshift-image-registry/node-ca-rhg7w","openshift-network-operator/iptables-alerter-lmsk9","openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-86n8d","openshift-ovn-kubernetes/ovnkube-node-98whn","openshift-network-diagnostics/network-check-source-6f78d59f99-4xtdb","openshift-multus/multus-w6vnh","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r78n8","openshift-dns/node-resolver-scrms","kube-system/kas-connection-checker-685895c7c9-xkmbn","openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-m6sns","openshift-dns/dns-default-z5sfp","openshift-network-console/networking-console-plugin-86c478bb88-s75pm","openshift-image-registry/image-registry-54854c7897-pc7km","openshift-service-ca-operator/service-ca-operator-544d749c8c-62tn2","kube-system/kube-apiserver-proxy-ip-10-0-130-48.us-east-2.compute.internal","kube-system/kas-connection-checker-685895c7c9-cpkfr","kube-system/konnectivity-agent-pl5cl"] Oct 31 07:30:27.202099 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.202081 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5549647ff-cxcdt" Oct 31 07:30:27.202465 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.202445 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-282kr" Oct 31 07:30:27.203437 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.203415 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-86n8d" Oct 31 07:30:27.205066 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.205015 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\"" Oct 31 07:30:27.205187 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.205169 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\"" Oct 31 07:30:27.205339 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.205316 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-54854c7897-pc7km" Oct 31 07:30:27.206828 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.206807 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Oct 31 07:30:27.208093 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.208076 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-s75pm" Oct 31 07:30:27.208373 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.208327 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7thxr" Oct 31 07:30:27.209056 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.208999 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Oct 31 07:30:27.209532 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.209502 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Oct 31 07:30:27.209611 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.209590 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Oct 31 07:30:27.209674 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.209590 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Oct 31 07:30:27.209721 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.209706 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Oct 31 07:30:27.209794 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.209779 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Oct 31 07:30:27.209838 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.209797 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Oct 31 07:30:27.209880 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.209835 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Oct 31 07:30:27.209880 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.209860 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Oct 31 07:30:27.211463 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.211444 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Oct 31 07:30:27.211568 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.211552 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Oct 31 07:30:27.212522 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.212501 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Oct 31 07:30:27.212522 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.212518 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Oct 31 07:30:27.212659 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.212599 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\"" Oct 31 07:30:27.212659 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.212641 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Oct 31 07:30:27.212780 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.212733 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Oct 31 07:30:27.212871 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.212852 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Oct 31 07:30:27.212936 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.212865 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Oct 31 07:30:27.212936 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.212930 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Oct 31 07:30:27.213029 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.212944 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Oct 31 07:30:27.213029 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.212962 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Oct 31 07:30:27.213029 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.212995 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Oct 31 07:30:27.213029 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.213022 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Oct 31 07:30:27.213167 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.213028 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Oct 31 07:30:27.213167 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.213058 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Oct 31 07:30:27.213329 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.213312 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Oct 31 07:30:27.215662 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.215649 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" Oct 31 07:30:27.215959 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.215938 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Oct 31 07:30:27.216051 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.215947 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Oct 31 07:30:27.216051 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.215997 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Oct 31 07:30:27.216051 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.216001 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Oct 31 07:30:27.216051 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.216014 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Oct 31 07:30:27.216233 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.216054 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Oct 31 07:30:27.216233 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.216081 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Oct 31 07:30:27.216233 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.216091 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Oct 31 07:30:27.216233 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.216080 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Oct 31 07:30:27.216233 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.216123 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Oct 31 07:30:27.216233 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.216133 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Oct 31 07:30:27.216233 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.216149 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Oct 31 07:30:27.216233 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.216155 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-2schs\"" Oct 31 07:30:27.216233 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.216159 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Oct 31 07:30:27.216233 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.216164 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Oct 31 07:30:27.216233 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.216189 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-25rsd\"" Oct 31 07:30:27.216233 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.216195 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Oct 31 07:30:27.216233 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.216212 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Oct 31 07:30:27.216233 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.216224 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Oct 31 07:30:27.216745 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.216251 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Oct 31 07:30:27.216745 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.216404 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-z5sfp" Oct 31 07:30:27.216834 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.215951 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Oct 31 07:30:27.216917 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.216845 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xb7gt\"" Oct 31 07:30:27.216917 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.216866 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Oct 31 07:30:27.217583 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.217569 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mvmr5" Oct 31 07:30:27.219767 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.219742 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsvzt\" (UniqueName: \"kubernetes.io/projected/6fae9634-7abe-40c6-a892-9eb58166e9ad-kube-api-access-wsvzt\") pod \"kas-connection-checker-685895c7c9-xkmbn\" (UID: \"6fae9634-7abe-40c6-a892-9eb58166e9ad\") " pod="kube-system/kas-connection-checker-685895c7c9-xkmbn" Oct 31 07:30:27.219767 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.219782 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/b4dd8884-1d6e-4abe-ad6c-3c947623bb60-snapshots\") pod \"insights-operator-7c4986d449-hz58r\" (UID: \"b4dd8884-1d6e-4abe-ad6c-3c947623bb60\") " pod="openshift-insights/insights-operator-7c4986d449-hz58r" Oct 31 07:30:27.219954 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.219819 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28be803a-a932-4009-ad61-b356366821ef-service-ca-bundle\") pod \"router-default-5549647ff-cxcdt\" (UID: \"28be803a-a932-4009-ad61-b356366821ef\") " pod="openshift-ingress/router-default-5549647ff-cxcdt" Oct 31 07:30:27.219954 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.219852 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cf9zh\" (UniqueName: \"kubernetes.io/projected/28be803a-a932-4009-ad61-b356366821ef-kube-api-access-cf9zh\") pod \"router-default-5549647ff-cxcdt\" (UID: \"28be803a-a932-4009-ad61-b356366821ef\") " pod="openshift-ingress/router-default-5549647ff-cxcdt" Oct 31 07:30:27.219954 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.219887 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djqrt\" (UniqueName: \"kubernetes.io/projected/5e743234-2b5f-4c91-9b06-189f546cbce4-kube-api-access-djqrt\") pod \"cluster-monitoring-operator-884576c57-282kr\" (UID: \"5e743234-2b5f-4c91-9b06-189f546cbce4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-282kr" Oct 31 07:30:27.219954 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.219919 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-86n8d\" (UID: \"d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-86n8d" Oct 31 07:30:27.219954 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.219945 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cb5dx\" (UniqueName: \"kubernetes.io/projected/f3bf19fd-bc58-4dff-a1a8-4f61381e1fde-kube-api-access-cb5dx\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-46qvv\" (UID: \"f3bf19fd-bc58-4dff-a1a8-4f61381e1fde\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-46qvv" Oct 31 07:30:27.220196 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.219976 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g76kv\" (UniqueName: \"kubernetes.io/projected/396e1c5e-3700-4fe6-b367-54ad2a244225-kube-api-access-g76kv\") pod \"console-operator-7cd56b8dbc-jztcz\" (UID: \"396e1c5e-3700-4fe6-b367-54ad2a244225\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jztcz" Oct 31 07:30:27.220196 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.220008 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28be803a-a932-4009-ad61-b356366821ef-metrics-certs\") pod \"router-default-5549647ff-cxcdt\" (UID: \"28be803a-a932-4009-ad61-b356366821ef\") " pod="openshift-ingress/router-default-5549647ff-cxcdt" Oct 31 07:30:27.220196 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.220039 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4dd8884-1d6e-4abe-ad6c-3c947623bb60-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-hz58r\" (UID: \"b4dd8884-1d6e-4abe-ad6c-3c947623bb60\") " pod="openshift-insights/insights-operator-7c4986d449-hz58r" Oct 31 07:30:27.220196 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.220071 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9k9q8\" (UniqueName: \"kubernetes.io/projected/b4dd8884-1d6e-4abe-ad6c-3c947623bb60-kube-api-access-9k9q8\") pod \"insights-operator-7c4986d449-hz58r\" (UID: \"b4dd8884-1d6e-4abe-ad6c-3c947623bb60\") " pod="openshift-insights/insights-operator-7c4986d449-hz58r" Oct 31 07:30:27.220196 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.220099 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b6997c76-e103-4c3f-b39d-a6a17fb98270-serving-cert\") pod \"service-ca-operator-544d749c8c-62tn2\" (UID: \"b6997c76-e103-4c3f-b39d-a6a17fb98270\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-62tn2" Oct 31 07:30:27.220196 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.220127 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b6997c76-e103-4c3f-b39d-a6a17fb98270-config\") pod \"service-ca-operator-544d749c8c-62tn2\" (UID: \"b6997c76-e103-4c3f-b39d-a6a17fb98270\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-62tn2" Oct 31 07:30:27.220196 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.220157 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/396e1c5e-3700-4fe6-b367-54ad2a244225-config\") pod \"console-operator-7cd56b8dbc-jztcz\" (UID: \"396e1c5e-3700-4fe6-b367-54ad2a244225\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jztcz" Oct 31 07:30:27.220196 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.220187 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lx427\" (UniqueName: \"kubernetes.io/projected/54296c9d-1c43-44cd-bfb6-d81fe892484c-kube-api-access-lx427\") pod \"volume-data-source-validator-58c9f5f46d-m6sns\" (UID: \"54296c9d-1c43-44cd-bfb6-d81fe892484c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-m6sns" Oct 31 07:30:27.220564 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.220219 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4dd8884-1d6e-4abe-ad6c-3c947623bb60-service-ca-bundle\") pod \"insights-operator-7c4986d449-hz58r\" (UID: \"b4dd8884-1d6e-4abe-ad6c-3c947623bb60\") " pod="openshift-insights/insights-operator-7c4986d449-hz58r" Oct 31 07:30:27.220564 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.220247 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/5e743234-2b5f-4c91-9b06-189f546cbce4-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-282kr\" (UID: \"5e743234-2b5f-4c91-9b06-189f546cbce4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-282kr" Oct 31 07:30:27.220564 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.220282 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5e743234-2b5f-4c91-9b06-189f546cbce4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-282kr\" (UID: \"5e743234-2b5f-4c91-9b06-189f546cbce4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-282kr" Oct 31 07:30:27.220564 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.220313 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p76jj\" (UniqueName: \"kubernetes.io/projected/b6997c76-e103-4c3f-b39d-a6a17fb98270-kube-api-access-p76jj\") pod \"service-ca-operator-544d749c8c-62tn2\" (UID: \"b6997c76-e103-4c3f-b39d-a6a17fb98270\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-62tn2" Oct 31 07:30:27.220564 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.220367 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xksf5\" (UniqueName: \"kubernetes.io/projected/e84bef13-a137-49f7-931d-9f94952dacfc-kube-api-access-xksf5\") pod \"kas-connection-checker-685895c7c9-cpkfr\" (UID: \"e84bef13-a137-49f7-931d-9f94952dacfc\") " pod="kube-system/kas-connection-checker-685895c7c9-cpkfr" Oct 31 07:30:27.220564 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.220398 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/396e1c5e-3700-4fe6-b367-54ad2a244225-serving-cert\") pod \"console-operator-7cd56b8dbc-jztcz\" (UID: \"396e1c5e-3700-4fe6-b367-54ad2a244225\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jztcz" Oct 31 07:30:27.220564 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.220423 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/396e1c5e-3700-4fe6-b367-54ad2a244225-trusted-ca\") pod \"console-operator-7cd56b8dbc-jztcz\" (UID: \"396e1c5e-3700-4fe6-b367-54ad2a244225\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jztcz" Oct 31 07:30:27.220564 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.220453 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b4dd8884-1d6e-4abe-ad6c-3c947623bb60-tmp\") pod \"insights-operator-7c4986d449-hz58r\" (UID: \"b4dd8884-1d6e-4abe-ad6c-3c947623bb60\") " pod="openshift-insights/insights-operator-7c4986d449-hz58r" Oct 31 07:30:27.220564 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.220482 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b4dd8884-1d6e-4abe-ad6c-3c947623bb60-serving-cert\") pod \"insights-operator-7c4986d449-hz58r\" (UID: \"b4dd8884-1d6e-4abe-ad6c-3c947623bb60\") " pod="openshift-insights/insights-operator-7c4986d449-hz58r" Oct 31 07:30:27.220564 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.220512 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f3bf19fd-bc58-4dff-a1a8-4f61381e1fde-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-46qvv\" (UID: \"f3bf19fd-bc58-4dff-a1a8-4f61381e1fde\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-46qvv" Oct 31 07:30:27.220564 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.220541 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/28be803a-a932-4009-ad61-b356366821ef-stats-auth\") pod \"router-default-5549647ff-cxcdt\" (UID: \"28be803a-a932-4009-ad61-b356366821ef\") " pod="openshift-ingress/router-default-5549647ff-cxcdt" Oct 31 07:30:27.220564 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.220573 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlm6c\" (UniqueName: \"kubernetes.io/projected/d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e-kube-api-access-nlm6c\") pod \"cluster-samples-operator-6ddb6c8b94-86n8d\" (UID: \"d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-86n8d" Oct 31 07:30:27.221104 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.220603 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4zgz\" (UniqueName: \"kubernetes.io/projected/3cbf4695-0536-45d5-bc48-be56861b87b8-kube-api-access-v4zgz\") pod \"kas-connection-checker-685895c7c9-vhcq9\" (UID: \"3cbf4695-0536-45d5-bc48-be56861b87b8\") " pod="kube-system/kas-connection-checker-685895c7c9-vhcq9" Oct 31 07:30:27.221104 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.220633 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f3bf19fd-bc58-4dff-a1a8-4f61381e1fde-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-46qvv\" (UID: \"f3bf19fd-bc58-4dff-a1a8-4f61381e1fde\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-46qvv" Oct 31 07:30:27.221104 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.220664 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/28be803a-a932-4009-ad61-b356366821ef-default-certificate\") pod \"router-default-5549647ff-cxcdt\" (UID: \"28be803a-a932-4009-ad61-b356366821ef\") " pod="openshift-ingress/router-default-5549647ff-cxcdt" Oct 31 07:30:27.221104 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.220699 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-86n8d\" (UID: \"d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-86n8d" Oct 31 07:30:27.221382 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.221339 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-86n8d\" (UID: \"d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-86n8d" Oct 31 07:30:27.221445 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.221364 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/b4dd8884-1d6e-4abe-ad6c-3c947623bb60-snapshots\") pod \"insights-operator-7c4986d449-hz58r\" (UID: \"b4dd8884-1d6e-4abe-ad6c-3c947623bb60\") " pod="openshift-insights/insights-operator-7c4986d449-hz58r" Oct 31 07:30:27.221711 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.221685 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/396e1c5e-3700-4fe6-b367-54ad2a244225-serving-cert\") pod \"console-operator-7cd56b8dbc-jztcz\" (UID: \"396e1c5e-3700-4fe6-b367-54ad2a244225\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jztcz" Oct 31 07:30:27.221845 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.221824 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/396e1c5e-3700-4fe6-b367-54ad2a244225-config\") pod \"console-operator-7cd56b8dbc-jztcz\" (UID: \"396e1c5e-3700-4fe6-b367-54ad2a244225\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jztcz" Oct 31 07:30:27.221995 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.221973 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b6997c76-e103-4c3f-b39d-a6a17fb98270-serving-cert\") pod \"service-ca-operator-544d749c8c-62tn2\" (UID: \"b6997c76-e103-4c3f-b39d-a6a17fb98270\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-62tn2" Oct 31 07:30:27.222079 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.222057 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b4dd8884-1d6e-4abe-ad6c-3c947623bb60-tmp\") pod \"insights-operator-7c4986d449-hz58r\" (UID: \"b4dd8884-1d6e-4abe-ad6c-3c947623bb60\") " pod="openshift-insights/insights-operator-7c4986d449-hz58r" Oct 31 07:30:27.222152 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.222133 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4dd8884-1d6e-4abe-ad6c-3c947623bb60-service-ca-bundle\") pod \"insights-operator-7c4986d449-hz58r\" (UID: \"b4dd8884-1d6e-4abe-ad6c-3c947623bb60\") " pod="openshift-insights/insights-operator-7c4986d449-hz58r" Oct 31 07:30:27.222294 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.222259 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f3bf19fd-bc58-4dff-a1a8-4f61381e1fde-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-46qvv\" (UID: \"f3bf19fd-bc58-4dff-a1a8-4f61381e1fde\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-46qvv" Oct 31 07:30:27.222395 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.222323 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b6997c76-e103-4c3f-b39d-a6a17fb98270-config\") pod \"service-ca-operator-544d749c8c-62tn2\" (UID: \"b6997c76-e103-4c3f-b39d-a6a17fb98270\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-62tn2" Oct 31 07:30:27.222395 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.222339 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/396e1c5e-3700-4fe6-b367-54ad2a244225-trusted-ca\") pod \"console-operator-7cd56b8dbc-jztcz\" (UID: \"396e1c5e-3700-4fe6-b367-54ad2a244225\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jztcz" Oct 31 07:30:27.222512 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.222423 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/5e743234-2b5f-4c91-9b06-189f546cbce4-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-282kr\" (UID: \"5e743234-2b5f-4c91-9b06-189f546cbce4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-282kr" Oct 31 07:30:27.222512 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.222458 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f3bf19fd-bc58-4dff-a1a8-4f61381e1fde-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-46qvv\" (UID: \"f3bf19fd-bc58-4dff-a1a8-4f61381e1fde\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-46qvv" Oct 31 07:30:27.222512 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.222481 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b4dd8884-1d6e-4abe-ad6c-3c947623bb60-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-hz58r\" (UID: \"b4dd8884-1d6e-4abe-ad6c-3c947623bb60\") " pod="openshift-insights/insights-operator-7c4986d449-hz58r" Oct 31 07:30:27.222661 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.222517 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b4dd8884-1d6e-4abe-ad6c-3c947623bb60-serving-cert\") pod \"insights-operator-7c4986d449-hz58r\" (UID: \"b4dd8884-1d6e-4abe-ad6c-3c947623bb60\") " pod="openshift-insights/insights-operator-7c4986d449-hz58r" Oct 31 07:30:27.222661 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.222532 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/28be803a-a932-4009-ad61-b356366821ef-stats-auth\") pod \"router-default-5549647ff-cxcdt\" (UID: \"28be803a-a932-4009-ad61-b356366821ef\") " pod="openshift-ingress/router-default-5549647ff-cxcdt" Oct 31 07:30:27.222904 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.222876 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/28be803a-a932-4009-ad61-b356366821ef-default-certificate\") pod \"router-default-5549647ff-cxcdt\" (UID: \"28be803a-a932-4009-ad61-b356366821ef\") " pod="openshift-ingress/router-default-5549647ff-cxcdt" Oct 31 07:30:27.230913 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.230890 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cb5dx\" (UniqueName: \"kubernetes.io/projected/f3bf19fd-bc58-4dff-a1a8-4f61381e1fde-kube-api-access-cb5dx\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-46qvv\" (UID: \"f3bf19fd-bc58-4dff-a1a8-4f61381e1fde\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-46qvv" Oct 31 07:30:27.230913 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.230909 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lx427\" (UniqueName: \"kubernetes.io/projected/54296c9d-1c43-44cd-bfb6-d81fe892484c-kube-api-access-lx427\") pod \"volume-data-source-validator-58c9f5f46d-m6sns\" (UID: \"54296c9d-1c43-44cd-bfb6-d81fe892484c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-m6sns" Oct 31 07:30:27.231157 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.231115 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsvzt\" (UniqueName: \"kubernetes.io/projected/6fae9634-7abe-40c6-a892-9eb58166e9ad-kube-api-access-wsvzt\") pod \"kas-connection-checker-685895c7c9-xkmbn\" (UID: \"6fae9634-7abe-40c6-a892-9eb58166e9ad\") " pod="kube-system/kas-connection-checker-685895c7c9-xkmbn" Oct 31 07:30:27.231675 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.231632 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g76kv\" (UniqueName: \"kubernetes.io/projected/396e1c5e-3700-4fe6-b367-54ad2a244225-kube-api-access-g76kv\") pod \"console-operator-7cd56b8dbc-jztcz\" (UID: \"396e1c5e-3700-4fe6-b367-54ad2a244225\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-jztcz" Oct 31 07:30:27.231913 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.231894 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-djqrt\" (UniqueName: \"kubernetes.io/projected/5e743234-2b5f-4c91-9b06-189f546cbce4-kube-api-access-djqrt\") pod \"cluster-monitoring-operator-884576c57-282kr\" (UID: \"5e743234-2b5f-4c91-9b06-189f546cbce4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-282kr" Oct 31 07:30:27.232026 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.232005 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cf9zh\" (UniqueName: \"kubernetes.io/projected/28be803a-a932-4009-ad61-b356366821ef-kube-api-access-cf9zh\") pod \"router-default-5549647ff-cxcdt\" (UID: \"28be803a-a932-4009-ad61-b356366821ef\") " pod="openshift-ingress/router-default-5549647ff-cxcdt" Oct 31 07:30:27.232100 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.232079 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9k9q8\" (UniqueName: \"kubernetes.io/projected/b4dd8884-1d6e-4abe-ad6c-3c947623bb60-kube-api-access-9k9q8\") pod \"insights-operator-7c4986d449-hz58r\" (UID: \"b4dd8884-1d6e-4abe-ad6c-3c947623bb60\") " pod="openshift-insights/insights-operator-7c4986d449-hz58r" Oct 31 07:30:27.232140 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.232087 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p76jj\" (UniqueName: \"kubernetes.io/projected/b6997c76-e103-4c3f-b39d-a6a17fb98270-kube-api-access-p76jj\") pod \"service-ca-operator-544d749c8c-62tn2\" (UID: \"b6997c76-e103-4c3f-b39d-a6a17fb98270\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-62tn2" Oct 31 07:30:27.232170 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.232136 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xksf5\" (UniqueName: \"kubernetes.io/projected/e84bef13-a137-49f7-931d-9f94952dacfc-kube-api-access-xksf5\") pod \"kas-connection-checker-685895c7c9-cpkfr\" (UID: \"e84bef13-a137-49f7-931d-9f94952dacfc\") " pod="kube-system/kas-connection-checker-685895c7c9-cpkfr" Oct 31 07:30:27.232206 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.232167 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4zgz\" (UniqueName: \"kubernetes.io/projected/3cbf4695-0536-45d5-bc48-be56861b87b8-kube-api-access-v4zgz\") pod \"kas-connection-checker-685895c7c9-vhcq9\" (UID: \"3cbf4695-0536-45d5-bc48-be56861b87b8\") " pod="kube-system/kas-connection-checker-685895c7c9-vhcq9" Oct 31 07:30:27.232435 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.232418 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlm6c\" (UniqueName: \"kubernetes.io/projected/d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e-kube-api-access-nlm6c\") pod \"cluster-samples-operator-6ddb6c8b94-86n8d\" (UID: \"d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-86n8d" Oct 31 07:30:27.266959 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.266937 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Oct 31 07:30:27.267146 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.267126 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Oct 31 07:30:27.267241 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.267145 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Oct 31 07:30:27.267324 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.267308 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Oct 31 07:30:27.267395 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.267311 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5t598\"" Oct 31 07:30:27.267553 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.267538 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-sl77n\"" Oct 31 07:30:27.267811 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.267796 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ssdfd\"" Oct 31 07:30:27.267861 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.267829 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-x6jvt\"" Oct 31 07:30:27.268095 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.268080 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-65sf9\"" Oct 31 07:30:27.268146 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.268079 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Oct 31 07:30:27.268146 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.268103 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-kqb4j\"" Oct 31 07:30:27.268146 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.268083 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Oct 31 07:30:27.268146 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.268133 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Oct 31 07:30:27.268261 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.268199 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-x8m6s\"" Oct 31 07:30:27.268868 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.268853 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Oct 31 07:30:27.268917 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.268871 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-78wjc\"" Oct 31 07:30:27.268917 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.268901 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-jshc6\"" Oct 31 07:30:27.269016 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.268982 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-hztdx\"" Oct 31 07:30:27.269016 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.268987 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-tghnm\"" Oct 31 07:30:27.269016 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.268984 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9wbww\"" Oct 31 07:30:27.269016 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.269008 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-bcvgm\"" Oct 31 07:30:27.269016 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.269012 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-lhm78\"" Oct 31 07:30:27.269173 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.269022 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-jjzj5\"" Oct 31 07:30:27.269173 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.269023 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-p6nqs\"" Oct 31 07:30:27.288588 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.288570 4935 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-48.us-east-2.compute.internal" Oct 31 07:30:27.288690 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.288679 4935 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-48.us-east-2.compute.internal" Oct 31 07:30:27.298552 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.298532 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Oct 31 07:30:27.298616 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.298579 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Oct 31 07:30:27.298764 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.298750 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Oct 31 07:30:27.298812 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.298793 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Oct 31 07:30:27.298900 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.298883 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Oct 31 07:30:27.298900 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.298896 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Oct 31 07:30:27.298977 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.298888 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Oct 31 07:30:27.298977 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.298906 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Oct 31 07:30:27.298977 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.298919 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Oct 31 07:30:27.298977 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.298935 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Oct 31 07:30:27.298977 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.298954 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Oct 31 07:30:27.298977 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.298964 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Oct 31 07:30:27.299275 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.299257 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Oct 31 07:30:27.299408 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.299276 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Oct 31 07:30:27.299408 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.299298 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Oct 31 07:30:27.299408 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.299329 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Oct 31 07:30:27.299408 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.299333 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Oct 31 07:30:27.299408 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.299340 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Oct 31 07:30:27.299408 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.299322 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Oct 31 07:30:27.299408 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.299312 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Oct 31 07:30:27.299408 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.299398 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Oct 31 07:30:27.299654 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.299316 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Oct 31 07:30:27.299654 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.299344 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Oct 31 07:30:27.299654 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.299382 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Oct 31 07:30:27.299654 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.299306 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Oct 31 07:30:27.300302 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.300289 4935 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 31 07:30:27.300337 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:27.300326 4935 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-130-48.us-east-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-48.us-east-2.compute.internal" Oct 31 07:30:27.300447 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.300434 4935 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 31 07:30:27.300498 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:27.300476 4935 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-130-48.us-east-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-48.us-east-2.compute.internal" Oct 31 07:30:27.305749 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.305733 4935 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Oct 31 07:30:27.320884 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.320862 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/58bc3ea1-f58c-4891-a049-1289a99be055-config-volume\") pod \"dns-default-z5sfp\" (UID: \"58bc3ea1-f58c-4891-a049-1289a99be055\") " pod="openshift-dns/dns-default-z5sfp" Oct 31 07:30:27.320963 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.320886 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-lib-modules\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:30:27.320963 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.320903 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-image-registry-private-configuration\") pod \"image-registry-54854c7897-pc7km\" (UID: \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\") " pod="openshift-image-registry/image-registry-54854c7897-pc7km" Oct 31 07:30:27.320963 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.320918 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrvrg\" (UniqueName: \"kubernetes.io/projected/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-kube-api-access-mrvrg\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:27.320963 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.320939 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28be803a-a932-4009-ad61-b356366821ef-service-ca-bundle\") pod \"router-default-5549647ff-cxcdt\" (UID: \"28be803a-a932-4009-ad61-b356366821ef\") " pod="openshift-ingress/router-default-5549647ff-cxcdt" Oct 31 07:30:27.321110 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.320976 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f13bc62a-1894-472c-95de-262afa399b38-data-volume\") pod \"insights-runtime-extractor-mvmr5\" (UID: \"f13bc62a-1894-472c-95de-262afa399b38\") " pod="openshift-insights/insights-runtime-extractor-mvmr5" Oct 31 07:30:27.321110 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.321013 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-multus-socket-dir-parent\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:30:27.321110 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.321038 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ac4bf75a-377c-49b2-818a-7d2436c21e9c-iptables-alerter-script\") pod \"iptables-alerter-lmsk9\" (UID: \"ac4bf75a-377c-49b2-818a-7d2436c21e9c\") " pod="openshift-network-operator/iptables-alerter-lmsk9" Oct 31 07:30:27.321110 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.321103 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-var-lib-kubelet\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:30:27.321280 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.321106 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f13bc62a-1894-472c-95de-262afa399b38-data-volume\") pod \"insights-runtime-extractor-mvmr5\" (UID: \"f13bc62a-1894-472c-95de-262afa399b38\") " pod="openshift-insights/insights-runtime-extractor-mvmr5" Oct 31 07:30:27.321280 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.321156 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/58bc3ea1-f58c-4891-a049-1289a99be055-config-volume\") pod \"dns-default-z5sfp\" (UID: \"58bc3ea1-f58c-4891-a049-1289a99be055\") " pod="openshift-dns/dns-default-z5sfp" Oct 31 07:30:27.321280 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.321180 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-image-registry-private-configuration\") pod \"image-registry-54854c7897-pc7km\" (UID: \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\") " pod="openshift-image-registry/image-registry-54854c7897-pc7km" Oct 31 07:30:27.321280 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:27.321202 4935 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/28be803a-a932-4009-ad61-b356366821ef-service-ca-bundle podName:28be803a-a932-4009-ad61-b356366821ef nodeName:}" failed. No retries permitted until 2026-10-31 07:30:27.821183933 +0000 UTC m=+1.691715463 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/28be803a-a932-4009-ad61-b356366821ef-service-ca-bundle") pod "router-default-5549647ff-cxcdt" (UID: "28be803a-a932-4009-ad61-b356366821ef") : configmap references non-existent config key: service-ca.crt Oct 31 07:30:27.321280 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.321221 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-run-systemd\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:27.321280 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.321266 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-env-overrides\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:27.321566 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.321325 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ea9beb38-1491-4cf0-83ca-fd79935fa6a9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r78n8\" (UID: \"ea9beb38-1491-4cf0-83ca-fd79935fa6a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r78n8" Oct 31 07:30:27.321566 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.321380 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/297b35d7-aa65-49c7-8460-ac3f9860248c-dbus\") pod \"global-pull-secret-syncer-8hpcb\" (UID: \"297b35d7-aa65-49c7-8460-ac3f9860248c\") " pod="kube-system/global-pull-secret-syncer-8hpcb" Oct 31 07:30:27.321566 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.321396 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ac4bf75a-377c-49b2-818a-7d2436c21e9c-iptables-alerter-script\") pod \"iptables-alerter-lmsk9\" (UID: \"ac4bf75a-377c-49b2-818a-7d2436c21e9c\") " pod="openshift-network-operator/iptables-alerter-lmsk9" Oct 31 07:30:27.321566 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.321418 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d25f2854-fd38-44e7-ad7e-da744d57e671-konnectivity-ca\") pod \"konnectivity-agent-pl5cl\" (UID: \"d25f2854-fd38-44e7-ad7e-da744d57e671\") " pod="kube-system/konnectivity-agent-pl5cl" Oct 31 07:30:27.321566 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.321460 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4104ba06-fba8-4c3c-b876-cca5087a7d0a-registry-certificates\") pod \"image-registry-69fbc98d44-fn7fh\" (UID: \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\") " pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" Oct 31 07:30:27.321566 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.321529 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ed522b38-611a-452a-b201-3ca416428d42-cni-binary-copy\") pod \"multus-additional-cni-plugins-ktzg4\" (UID: \"ed522b38-611a-452a-b201-3ca416428d42\") " pod="openshift-multus/multus-additional-cni-plugins-ktzg4" Oct 31 07:30:27.321829 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.321572 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/53c36e19-9ef6-47a2-9bc2-387115539850-hosts-file\") pod \"node-resolver-scrms\" (UID: \"53c36e19-9ef6-47a2-9bc2-387115539850\") " pod="openshift-dns/node-resolver-scrms" Oct 31 07:30:27.321829 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.321597 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-etc-kubernetes\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:30:27.321829 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.321610 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-env-overrides\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:27.321829 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.321656 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d25f2854-fd38-44e7-ad7e-da744d57e671-konnectivity-ca\") pod \"konnectivity-agent-pl5cl\" (UID: \"d25f2854-fd38-44e7-ad7e-da744d57e671\") " pod="kube-system/konnectivity-agent-pl5cl" Oct 31 07:30:27.321829 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.321650 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-host-cni-netd\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:27.321829 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.321667 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4104ba06-fba8-4c3c-b876-cca5087a7d0a-registry-certificates\") pod \"image-registry-69fbc98d44-fn7fh\" (UID: \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\") " pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" Oct 31 07:30:27.321829 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.321714 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58bc3ea1-f58c-4891-a049-1289a99be055-tmp-dir\") pod \"dns-default-z5sfp\" (UID: \"58bc3ea1-f58c-4891-a049-1289a99be055\") " pod="openshift-dns/dns-default-z5sfp" Oct 31 07:30:27.321829 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.321732 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f13bc62a-1894-472c-95de-262afa399b38-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mvmr5\" (UID: \"f13bc62a-1894-472c-95de-262afa399b38\") " pod="openshift-insights/insights-runtime-extractor-mvmr5" Oct 31 07:30:27.321829 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.321747 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-hostroot\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:30:27.321829 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.321761 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-multus-conf-dir\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:30:27.321829 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.321778 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ed522b38-611a-452a-b201-3ca416428d42-cni-binary-copy\") pod \"multus-additional-cni-plugins-ktzg4\" (UID: \"ed522b38-611a-452a-b201-3ca416428d42\") " pod="openshift-multus/multus-additional-cni-plugins-ktzg4" Oct 31 07:30:27.321829 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.321807 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4104ba06-fba8-4c3c-b876-cca5087a7d0a-bound-sa-token\") pod \"image-registry-69fbc98d44-fn7fh\" (UID: \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\") " pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" Oct 31 07:30:27.321829 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.321810 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58bc3ea1-f58c-4891-a049-1289a99be055-tmp-dir\") pod \"dns-default-z5sfp\" (UID: \"58bc3ea1-f58c-4891-a049-1289a99be055\") " pod="openshift-dns/dns-default-z5sfp" Oct 31 07:30:27.322197 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.321835 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-host-var-lib-kubelet\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:30:27.322197 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.322006 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ed522b38-611a-452a-b201-3ca416428d42-cnibin\") pod \"multus-additional-cni-plugins-ktzg4\" (UID: \"ed522b38-611a-452a-b201-3ca416428d42\") " pod="openshift-multus/multus-additional-cni-plugins-ktzg4" Oct 31 07:30:27.322197 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.322029 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-host-slash\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:27.322197 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.322046 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/00a0d978-8020-46df-932d-b0dd22f946f9-cert\") pod \"ingress-canary-22cmd\" (UID: \"00a0d978-8020-46df-932d-b0dd22f946f9\") " pod="openshift-ingress-canary/ingress-canary-22cmd" Oct 31 07:30:27.322197 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.322092 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ac4bf75a-377c-49b2-818a-7d2436c21e9c-host-slash\") pod \"iptables-alerter-lmsk9\" (UID: \"ac4bf75a-377c-49b2-818a-7d2436c21e9c\") " pod="openshift-network-operator/iptables-alerter-lmsk9" Oct 31 07:30:27.322197 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.322121 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/236238a7-255d-40f7-a99b-d65903b3bcb7-serviceca\") pod \"node-ca-rhg7w\" (UID: \"236238a7-255d-40f7-a99b-d65903b3bcb7\") " pod="openshift-image-registry/node-ca-rhg7w" Oct 31 07:30:27.322197 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.322143 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4104ba06-fba8-4c3c-b876-cca5087a7d0a-ca-trust-extracted\") pod \"image-registry-69fbc98d44-fn7fh\" (UID: \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\") " pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" Oct 31 07:30:27.322197 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.322168 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-host-cni-bin\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:27.322197 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.322186 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ea9beb38-1491-4cf0-83ca-fd79935fa6a9-device-dir\") pod \"aws-ebs-csi-driver-node-r78n8\" (UID: \"ea9beb38-1491-4cf0-83ca-fd79935fa6a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r78n8" Oct 31 07:30:27.322584 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.322200 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-etc-sysconfig\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:30:27.322584 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.322212 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4104ba06-fba8-4c3c-b876-cca5087a7d0a-ca-trust-extracted\") pod \"image-registry-69fbc98d44-fn7fh\" (UID: \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\") " pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" Oct 31 07:30:27.322584 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.322217 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4104ba06-fba8-4c3c-b876-cca5087a7d0a-image-registry-private-configuration\") pod \"image-registry-69fbc98d44-fn7fh\" (UID: \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\") " pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" Oct 31 07:30:27.322584 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.322255 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-host-var-lib-cni-bin\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:30:27.322584 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.322277 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/00a0d978-8020-46df-932d-b0dd22f946f9-cert\") pod \"ingress-canary-22cmd\" (UID: \"00a0d978-8020-46df-932d-b0dd22f946f9\") " pod="openshift-ingress-canary/ingress-canary-22cmd" Oct 31 07:30:27.322584 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.322294 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/236238a7-255d-40f7-a99b-d65903b3bcb7-serviceca\") pod \"node-ca-rhg7w\" (UID: \"236238a7-255d-40f7-a99b-d65903b3bcb7\") " pod="openshift-image-registry/node-ca-rhg7w" Oct 31 07:30:27.322584 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.322368 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/563d3b8b-0040-439e-b0b2-e858eed5abf6-nginx-conf\") pod \"networking-console-plugin-86c478bb88-s75pm\" (UID: \"563d3b8b-0040-439e-b0b2-e858eed5abf6\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-s75pm" Oct 31 07:30:27.322584 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.322405 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/563d3b8b-0040-439e-b0b2-e858eed5abf6-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-s75pm\" (UID: \"563d3b8b-0040-439e-b0b2-e858eed5abf6\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-s75pm" Oct 31 07:30:27.322584 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.322422 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4104ba06-fba8-4c3c-b876-cca5087a7d0a-image-registry-private-configuration\") pod \"image-registry-69fbc98d44-fn7fh\" (UID: \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\") " pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" Oct 31 07:30:27.322584 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.322433 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ea9beb38-1491-4cf0-83ca-fd79935fa6a9-sys-fs\") pod \"aws-ebs-csi-driver-node-r78n8\" (UID: \"ea9beb38-1491-4cf0-83ca-fd79935fa6a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r78n8" Oct 31 07:30:27.322584 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.322470 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ed522b38-611a-452a-b201-3ca416428d42-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ktzg4\" (UID: \"ed522b38-611a-452a-b201-3ca416428d42\") " pod="openshift-multus/multus-additional-cni-plugins-ktzg4" Oct 31 07:30:27.322584 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.322511 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ed522b38-611a-452a-b201-3ca416428d42-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ktzg4\" (UID: \"ed522b38-611a-452a-b201-3ca416428d42\") " pod="openshift-multus/multus-additional-cni-plugins-ktzg4" Oct 31 07:30:27.322584 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.322519 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/563d3b8b-0040-439e-b0b2-e858eed5abf6-nginx-conf\") pod \"networking-console-plugin-86c478bb88-s75pm\" (UID: \"563d3b8b-0040-439e-b0b2-e858eed5abf6\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-s75pm" Oct 31 07:30:27.322584 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.322543 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-etc-sysctl-d\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:30:27.322584 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.322559 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/236238a7-255d-40f7-a99b-d65903b3bcb7-host\") pod \"node-ca-rhg7w\" (UID: \"236238a7-255d-40f7-a99b-d65903b3bcb7\") " pod="openshift-image-registry/node-ca-rhg7w" Oct 31 07:30:27.322584 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.322576 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftf9v\" (UniqueName: \"kubernetes.io/projected/236238a7-255d-40f7-a99b-d65903b3bcb7-kube-api-access-ftf9v\") pod \"node-ca-rhg7w\" (UID: \"236238a7-255d-40f7-a99b-d65903b3bcb7\") " pod="openshift-image-registry/node-ca-rhg7w" Oct 31 07:30:27.322584 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.322591 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28be803a-a932-4009-ad61-b356366821ef-metrics-certs\") pod \"router-default-5549647ff-cxcdt\" (UID: \"28be803a-a932-4009-ad61-b356366821ef\") " pod="openshift-ingress/router-default-5549647ff-cxcdt" Oct 31 07:30:27.323335 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.322621 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4q9t4\" (UniqueName: \"kubernetes.io/projected/58bc3ea1-f58c-4891-a049-1289a99be055-kube-api-access-4q9t4\") pod \"dns-default-z5sfp\" (UID: \"58bc3ea1-f58c-4891-a049-1289a99be055\") " pod="openshift-dns/dns-default-z5sfp" Oct 31 07:30:27.323335 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.322695 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/046b1de8-d709-4729-9aec-391a2b3294d5-tmp\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:30:27.323335 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.322697 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ed522b38-611a-452a-b201-3ca416428d42-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ktzg4\" (UID: \"ed522b38-611a-452a-b201-3ca416428d42\") " pod="openshift-multus/multus-additional-cni-plugins-ktzg4" Oct 31 07:30:27.323335 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:27.322725 4935 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Oct 31 07:30:27.323335 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.322725 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-host-run-netns\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:27.323335 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.322753 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/046b1de8-d709-4729-9aec-391a2b3294d5-tmp\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:30:27.323335 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.322764 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-node-log\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:27.323335 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:27.322819 4935 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28be803a-a932-4009-ad61-b356366821ef-metrics-certs podName:28be803a-a932-4009-ad61-b356366821ef nodeName:}" failed. No retries permitted until 2026-10-31 07:30:27.82280276 +0000 UTC m=+1.693334294 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/28be803a-a932-4009-ad61-b356366821ef-metrics-certs") pod "router-default-5549647ff-cxcdt" (UID: "28be803a-a932-4009-ad61-b356366821ef") : secret "router-metrics-certs-default" not found Oct 31 07:30:27.323335 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.322833 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ed522b38-611a-452a-b201-3ca416428d42-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ktzg4\" (UID: \"ed522b38-611a-452a-b201-3ca416428d42\") " pod="openshift-multus/multus-additional-cni-plugins-ktzg4" Oct 31 07:30:27.323335 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.322852 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xv5b5\" (UniqueName: \"kubernetes.io/projected/ac4bf75a-377c-49b2-818a-7d2436c21e9c-kube-api-access-xv5b5\") pod \"iptables-alerter-lmsk9\" (UID: \"ac4bf75a-377c-49b2-818a-7d2436c21e9c\") " pod="openshift-network-operator/iptables-alerter-lmsk9" Oct 31 07:30:27.323335 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.322867 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-etc-modprobe-d\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:30:27.323335 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.322886 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/046b1de8-d709-4729-9aec-391a2b3294d5-etc-tuned\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:30:27.323335 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.322930 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-host-run-ovn-kubernetes\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:27.323335 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.322972 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mqks\" (UniqueName: \"kubernetes.io/projected/28ec9621-7ec5-421c-a88b-51bb20d3a7bf-kube-api-access-8mqks\") pod \"network-metrics-daemon-7thxr\" (UID: \"28ec9621-7ec5-421c-a88b-51bb20d3a7bf\") " pod="openshift-multus/network-metrics-daemon-7thxr" Oct 31 07:30:27.323335 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.322988 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ed522b38-611a-452a-b201-3ca416428d42-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ktzg4\" (UID: \"ed522b38-611a-452a-b201-3ca416428d42\") " pod="openshift-multus/multus-additional-cni-plugins-ktzg4" Oct 31 07:30:27.323335 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.323014 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-bound-sa-token\") pod \"image-registry-54854c7897-pc7km\" (UID: \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\") " pod="openshift-image-registry/image-registry-54854c7897-pc7km" Oct 31 07:30:27.323335 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.323109 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/046b1de8-d709-4729-9aec-391a2b3294d5-etc-tuned\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:30:27.323335 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.323117 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-multus-daemon-config\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:30:27.323335 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.323187 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:27.323335 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.323309 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ea9beb38-1491-4cf0-83ca-fd79935fa6a9-registration-dir\") pod \"aws-ebs-csi-driver-node-r78n8\" (UID: \"ea9beb38-1491-4cf0-83ca-fd79935fa6a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r78n8" Oct 31 07:30:27.324205 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.323367 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-multus-daemon-config\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:30:27.324205 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.323370 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhvbn\" (UniqueName: \"kubernetes.io/projected/f13bc62a-1894-472c-95de-262afa399b38-kube-api-access-qhvbn\") pod \"insights-runtime-extractor-mvmr5\" (UID: \"f13bc62a-1894-472c-95de-262afa399b38\") " pod="openshift-insights/insights-runtime-extractor-mvmr5" Oct 31 07:30:27.324205 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.323406 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-registry-tls\") pod \"image-registry-54854c7897-pc7km\" (UID: \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\") " pod="openshift-image-registry/image-registry-54854c7897-pc7km" Oct 31 07:30:27.324205 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.323421 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-host-run-k8s-cni-cncf-io\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:30:27.324205 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.323445 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/297b35d7-aa65-49c7-8460-ac3f9860248c-original-pull-secret\") pod \"global-pull-secret-syncer-8hpcb\" (UID: \"297b35d7-aa65-49c7-8460-ac3f9860248c\") " pod="kube-system/global-pull-secret-syncer-8hpcb" Oct 31 07:30:27.324205 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.323488 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-86n8d\" (UID: \"d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-86n8d" Oct 31 07:30:27.324205 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.323522 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-cnibin\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:30:27.324205 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.323547 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-etc-kubernetes\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:30:27.324205 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.323573 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-run-ovn\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:27.324205 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:27.323612 4935 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Oct 31 07:30:27.324205 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.323617 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ed522b38-611a-452a-b201-3ca416428d42-system-cni-dir\") pod \"multus-additional-cni-plugins-ktzg4\" (UID: \"ed522b38-611a-452a-b201-3ca416428d42\") " pod="openshift-multus/multus-additional-cni-plugins-ktzg4" Oct 31 07:30:27.324205 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.323652 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53c36e19-9ef6-47a2-9bc2-387115539850-tmp-dir\") pod \"node-resolver-scrms\" (UID: \"53c36e19-9ef6-47a2-9bc2-387115539850\") " pod="openshift-dns/node-resolver-scrms" Oct 31 07:30:27.324205 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:27.323662 4935 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e-samples-operator-tls podName:d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e nodeName:}" failed. No retries permitted until 2026-10-31 07:30:27.823645048 +0000 UTC m=+1.694176586 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-86n8d" (UID: "d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e") : secret "samples-operator-tls" not found Oct 31 07:30:27.324205 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.323666 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/297b35d7-aa65-49c7-8460-ac3f9860248c-original-pull-secret\") pod \"global-pull-secret-syncer-8hpcb\" (UID: \"297b35d7-aa65-49c7-8460-ac3f9860248c\") " pod="kube-system/global-pull-secret-syncer-8hpcb" Oct 31 07:30:27.324205 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.323693 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d25f2854-fd38-44e7-ad7e-da744d57e671-agent-certs\") pod \"konnectivity-agent-pl5cl\" (UID: \"d25f2854-fd38-44e7-ad7e-da744d57e671\") " pod="kube-system/konnectivity-agent-pl5cl" Oct 31 07:30:27.324205 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.323708 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53c36e19-9ef6-47a2-9bc2-387115539850-tmp-dir\") pod \"node-resolver-scrms\" (UID: \"53c36e19-9ef6-47a2-9bc2-387115539850\") " pod="openshift-dns/node-resolver-scrms" Oct 31 07:30:27.324205 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.323720 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-multus-cni-dir\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:30:27.324205 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.323747 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-cni-binary-copy\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:30:27.324205 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.323783 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-registry-certificates\") pod \"image-registry-54854c7897-pc7km\" (UID: \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\") " pod="openshift-image-registry/image-registry-54854c7897-pc7km" Oct 31 07:30:27.324205 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.323808 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76x7d\" (UniqueName: \"kubernetes.io/projected/046b1de8-d709-4729-9aec-391a2b3294d5-kube-api-access-76x7d\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:30:27.324205 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.323823 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4104ba06-fba8-4c3c-b876-cca5087a7d0a-installation-pull-secrets\") pod \"image-registry-69fbc98d44-fn7fh\" (UID: \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\") " pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" Oct 31 07:30:27.324205 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.323841 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-run-openvswitch\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:27.324205 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.323866 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5s89\" (UniqueName: \"kubernetes.io/projected/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-kube-api-access-z5s89\") pod \"image-registry-54854c7897-pc7km\" (UID: \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\") " pod="openshift-image-registry/image-registry-54854c7897-pc7km" Oct 31 07:30:27.324205 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.323877 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d25f2854-fd38-44e7-ad7e-da744d57e671-agent-certs\") pod \"konnectivity-agent-pl5cl\" (UID: \"d25f2854-fd38-44e7-ad7e-da744d57e671\") " pod="kube-system/konnectivity-agent-pl5cl" Oct 31 07:30:27.324205 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.323930 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-cni-binary-copy\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:30:27.324205 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.323946 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-registry-certificates\") pod \"image-registry-54854c7897-pc7km\" (UID: \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\") " pod="openshift-image-registry/image-registry-54854c7897-pc7km" Oct 31 07:30:27.324205 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.323976 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ea9beb38-1491-4cf0-83ca-fd79935fa6a9-etc-selinux\") pod \"aws-ebs-csi-driver-node-r78n8\" (UID: \"ea9beb38-1491-4cf0-83ca-fd79935fa6a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r78n8" Oct 31 07:30:27.324205 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.323997 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ea9beb38-1491-4cf0-83ca-fd79935fa6a9-socket-dir\") pod \"aws-ebs-csi-driver-node-r78n8\" (UID: \"ea9beb38-1491-4cf0-83ca-fd79935fa6a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r78n8" Oct 31 07:30:27.324205 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.324016 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-trusted-ca\") pod \"image-registry-54854c7897-pc7km\" (UID: \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\") " pod="openshift-image-registry/image-registry-54854c7897-pc7km" Oct 31 07:30:27.324205 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.324046 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4104ba06-fba8-4c3c-b876-cca5087a7d0a-installation-pull-secrets\") pod \"image-registry-69fbc98d44-fn7fh\" (UID: \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\") " pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" Oct 31 07:30:27.324205 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.324054 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-host-run-multus-certs\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:30:27.324205 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.324134 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5e743234-2b5f-4c91-9b06-189f546cbce4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-282kr\" (UID: \"5e743234-2b5f-4c91-9b06-189f546cbce4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-282kr" Oct 31 07:30:27.324205 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.324167 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/58bc3ea1-f58c-4891-a049-1289a99be055-metrics-tls\") pod \"dns-default-z5sfp\" (UID: \"58bc3ea1-f58c-4891-a049-1289a99be055\") " pod="openshift-dns/dns-default-z5sfp" Oct 31 07:30:27.324205 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.324191 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-host\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:30:27.325201 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:27.324228 4935 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 31 07:30:27.325201 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.324264 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-var-lib-openvswitch\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:27.325201 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:27.324281 4935 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e743234-2b5f-4c91-9b06-189f546cbce4-cluster-monitoring-operator-tls podName:5e743234-2b5f-4c91-9b06-189f546cbce4 nodeName:}" failed. No retries permitted until 2026-10-31 07:30:27.824267819 +0000 UTC m=+1.694799348 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5e743234-2b5f-4c91-9b06-189f546cbce4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-282kr" (UID: "5e743234-2b5f-4c91-9b06-189f546cbce4") : secret "cluster-monitoring-operator-tls" not found Oct 31 07:30:27.325201 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.324318 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-ovnkube-script-lib\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:27.325201 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.324342 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-trusted-ca\") pod \"image-registry-54854c7897-pc7km\" (UID: \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\") " pod="openshift-image-registry/image-registry-54854c7897-pc7km" Oct 31 07:30:27.325201 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.324439 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-log-socket\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:27.325201 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.324515 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-etc-systemd\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:30:27.325201 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.324543 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4104ba06-fba8-4c3c-b876-cca5087a7d0a-registry-tls\") pod \"image-registry-69fbc98d44-fn7fh\" (UID: \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\") " pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" Oct 31 07:30:27.325201 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.324563 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4104ba06-fba8-4c3c-b876-cca5087a7d0a-trusted-ca\") pod \"image-registry-69fbc98d44-fn7fh\" (UID: \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\") " pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" Oct 31 07:30:27.325201 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.324565 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-ovnkube-script-lib\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:27.325201 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.324592 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-host-var-lib-cni-multus\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:30:27.325201 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.324685 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ckzf\" (UniqueName: \"kubernetes.io/projected/d8b28b9f-37a2-4859-87fd-d27bb37d832b-kube-api-access-2ckzf\") pod \"network-check-target-ncnd7\" (UID: \"d8b28b9f-37a2-4859-87fd-d27bb37d832b\") " pod="openshift-network-diagnostics/network-check-target-ncnd7" Oct 31 07:30:27.325201 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.324722 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4tdk\" (UniqueName: \"kubernetes.io/projected/ed522b38-611a-452a-b201-3ca416428d42-kube-api-access-w4tdk\") pod \"multus-additional-cni-plugins-ktzg4\" (UID: \"ed522b38-611a-452a-b201-3ca416428d42\") " pod="openshift-multus/multus-additional-cni-plugins-ktzg4" Oct 31 07:30:27.325201 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.324803 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jhpf\" (UniqueName: \"kubernetes.io/projected/00a0d978-8020-46df-932d-b0dd22f946f9-kube-api-access-2jhpf\") pod \"ingress-canary-22cmd\" (UID: \"00a0d978-8020-46df-932d-b0dd22f946f9\") " pod="openshift-ingress-canary/ingress-canary-22cmd" Oct 31 07:30:27.325201 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.324822 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f13bc62a-1894-472c-95de-262afa399b38-crio-socket\") pod \"insights-runtime-extractor-mvmr5\" (UID: \"f13bc62a-1894-472c-95de-262afa399b38\") " pod="openshift-insights/insights-runtime-extractor-mvmr5" Oct 31 07:30:27.325201 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.324844 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f13bc62a-1894-472c-95de-262afa399b38-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mvmr5\" (UID: \"f13bc62a-1894-472c-95de-262afa399b38\") " pod="openshift-insights/insights-runtime-extractor-mvmr5" Oct 31 07:30:27.325201 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.324852 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4104ba06-fba8-4c3c-b876-cca5087a7d0a-trusted-ca\") pod \"image-registry-69fbc98d44-fn7fh\" (UID: \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\") " pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" Oct 31 07:30:27.325201 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.324859 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-host-kubelet\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:27.325201 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.324896 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrj54\" (UniqueName: \"kubernetes.io/projected/ea9beb38-1491-4cf0-83ca-fd79935fa6a9-kube-api-access-wrj54\") pod \"aws-ebs-csi-driver-node-r78n8\" (UID: \"ea9beb38-1491-4cf0-83ca-fd79935fa6a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r78n8" Oct 31 07:30:27.325201 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.324914 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-installation-pull-secrets\") pod \"image-registry-54854c7897-pc7km\" (UID: \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\") " pod="openshift-image-registry/image-registry-54854c7897-pc7km" Oct 31 07:30:27.325201 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.325023 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-systemd-units\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:27.325201 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.325052 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28ec9621-7ec5-421c-a88b-51bb20d3a7bf-metrics-certs\") pod \"network-metrics-daemon-7thxr\" (UID: \"28ec9621-7ec5-421c-a88b-51bb20d3a7bf\") " pod="openshift-multus/network-metrics-daemon-7thxr" Oct 31 07:30:27.325201 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.325063 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-installation-pull-secrets\") pod \"image-registry-54854c7897-pc7km\" (UID: \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\") " pod="openshift-image-registry/image-registry-54854c7897-pc7km" Oct 31 07:30:27.325201 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.325106 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-sys\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:30:27.325201 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.325133 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82jqk\" (UniqueName: \"kubernetes.io/projected/43c59587-4ff8-4546-9c0a-6161ece201d0-kube-api-access-82jqk\") pod \"network-check-source-6f78d59f99-4xtdb\" (UID: \"43c59587-4ff8-4546-9c0a-6161ece201d0\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-4xtdb" Oct 31 07:30:27.325201 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.325157 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzvgd\" (UniqueName: \"kubernetes.io/projected/53c36e19-9ef6-47a2-9bc2-387115539850-kube-api-access-zzvgd\") pod \"node-resolver-scrms\" (UID: \"53c36e19-9ef6-47a2-9bc2-387115539850\") " pod="openshift-dns/node-resolver-scrms" Oct 31 07:30:27.325201 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.325181 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-ca-trust-extracted\") pod \"image-registry-54854c7897-pc7km\" (UID: \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\") " pod="openshift-image-registry/image-registry-54854c7897-pc7km" Oct 31 07:30:27.325201 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.325206 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-997f5\" (UniqueName: \"kubernetes.io/projected/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-kube-api-access-997f5\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:30:27.326146 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.325229 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-ovnkube-config\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:27.326146 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.325254 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvpkt\" (UniqueName: \"kubernetes.io/projected/acda6b08-2faa-4836-888e-d4ecf324cb1d-kube-api-access-tvpkt\") pod \"migrator-66df5d496b-dwlls\" (UID: \"acda6b08-2faa-4836-888e-d4ecf324cb1d\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-dwlls" Oct 31 07:30:27.326146 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.325291 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-ca-trust-extracted\") pod \"image-registry-54854c7897-pc7km\" (UID: \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\") " pod="openshift-image-registry/image-registry-54854c7897-pc7km" Oct 31 07:30:27.326146 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.325378 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-run\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:30:27.326146 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.325409 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/297b35d7-aa65-49c7-8460-ac3f9860248c-kubelet-config\") pod \"global-pull-secret-syncer-8hpcb\" (UID: \"297b35d7-aa65-49c7-8460-ac3f9860248c\") " pod="kube-system/global-pull-secret-syncer-8hpcb" Oct 31 07:30:27.326146 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.325460 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-host-run-netns\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:30:27.326146 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.325473 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-ovnkube-config\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:27.326146 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.325486 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-etc-sysctl-conf\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:30:27.326146 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.325573 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hldt6\" (UniqueName: \"kubernetes.io/projected/4104ba06-fba8-4c3c-b876-cca5087a7d0a-kube-api-access-hldt6\") pod \"image-registry-69fbc98d44-fn7fh\" (UID: \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\") " pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" Oct 31 07:30:27.326146 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.325619 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-system-cni-dir\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:30:27.326146 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.325652 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-etc-openvswitch\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:27.326146 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.325676 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-ovn-node-metrics-cert\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:27.326146 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.325829 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-ovn-node-metrics-cert\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:27.331544 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.331525 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrvrg\" (UniqueName: \"kubernetes.io/projected/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-kube-api-access-mrvrg\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:27.333896 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.333877 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4q9t4\" (UniqueName: \"kubernetes.io/projected/58bc3ea1-f58c-4891-a049-1289a99be055-kube-api-access-4q9t4\") pod \"dns-default-z5sfp\" (UID: \"58bc3ea1-f58c-4891-a049-1289a99be055\") " pod="openshift-dns/dns-default-z5sfp" Oct 31 07:30:27.333896 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.333886 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4104ba06-fba8-4c3c-b876-cca5087a7d0a-bound-sa-token\") pod \"image-registry-69fbc98d44-fn7fh\" (UID: \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\") " pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" Oct 31 07:30:27.334032 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.333985 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftf9v\" (UniqueName: \"kubernetes.io/projected/236238a7-255d-40f7-a99b-d65903b3bcb7-kube-api-access-ftf9v\") pod \"node-ca-rhg7w\" (UID: \"236238a7-255d-40f7-a99b-d65903b3bcb7\") " pod="openshift-image-registry/node-ca-rhg7w" Oct 31 07:30:27.339546 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.339527 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xv5b5\" (UniqueName: \"kubernetes.io/projected/ac4bf75a-377c-49b2-818a-7d2436c21e9c-kube-api-access-xv5b5\") pod \"iptables-alerter-lmsk9\" (UID: \"ac4bf75a-377c-49b2-818a-7d2436c21e9c\") " pod="openshift-network-operator/iptables-alerter-lmsk9" Oct 31 07:30:27.339546 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.339540 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-bound-sa-token\") pod \"image-registry-54854c7897-pc7km\" (UID: \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\") " pod="openshift-image-registry/image-registry-54854c7897-pc7km" Oct 31 07:30:27.339793 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.339776 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mqks\" (UniqueName: \"kubernetes.io/projected/28ec9621-7ec5-421c-a88b-51bb20d3a7bf-kube-api-access-8mqks\") pod \"network-metrics-daemon-7thxr\" (UID: \"28ec9621-7ec5-421c-a88b-51bb20d3a7bf\") " pod="openshift-multus/network-metrics-daemon-7thxr" Oct 31 07:30:27.340077 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.340062 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhvbn\" (UniqueName: \"kubernetes.io/projected/f13bc62a-1894-472c-95de-262afa399b38-kube-api-access-qhvbn\") pod \"insights-runtime-extractor-mvmr5\" (UID: \"f13bc62a-1894-472c-95de-262afa399b38\") " pod="openshift-insights/insights-runtime-extractor-mvmr5" Oct 31 07:30:27.340665 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.340647 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5s89\" (UniqueName: \"kubernetes.io/projected/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-kube-api-access-z5s89\") pod \"image-registry-54854c7897-pc7km\" (UID: \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\") " pod="openshift-image-registry/image-registry-54854c7897-pc7km" Oct 31 07:30:27.340949 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.340932 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-76x7d\" (UniqueName: \"kubernetes.io/projected/046b1de8-d709-4729-9aec-391a2b3294d5-kube-api-access-76x7d\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:30:27.345304 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.342530 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ckzf\" (UniqueName: \"kubernetes.io/projected/d8b28b9f-37a2-4859-87fd-d27bb37d832b-kube-api-access-2ckzf\") pod \"network-check-target-ncnd7\" (UID: \"d8b28b9f-37a2-4859-87fd-d27bb37d832b\") " pod="openshift-network-diagnostics/network-check-target-ncnd7" Oct 31 07:30:27.346606 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.346579 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrj54\" (UniqueName: \"kubernetes.io/projected/ea9beb38-1491-4cf0-83ca-fd79935fa6a9-kube-api-access-wrj54\") pod \"aws-ebs-csi-driver-node-r78n8\" (UID: \"ea9beb38-1491-4cf0-83ca-fd79935fa6a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r78n8" Oct 31 07:30:27.346688 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.346617 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-997f5\" (UniqueName: \"kubernetes.io/projected/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-kube-api-access-997f5\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:30:27.346688 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.346671 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-82jqk\" (UniqueName: \"kubernetes.io/projected/43c59587-4ff8-4546-9c0a-6161ece201d0-kube-api-access-82jqk\") pod \"network-check-source-6f78d59f99-4xtdb\" (UID: \"43c59587-4ff8-4546-9c0a-6161ece201d0\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-4xtdb" Oct 31 07:30:27.346813 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.346798 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jhpf\" (UniqueName: \"kubernetes.io/projected/00a0d978-8020-46df-932d-b0dd22f946f9-kube-api-access-2jhpf\") pod \"ingress-canary-22cmd\" (UID: \"00a0d978-8020-46df-932d-b0dd22f946f9\") " pod="openshift-ingress-canary/ingress-canary-22cmd" Oct 31 07:30:27.346854 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.346838 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvpkt\" (UniqueName: \"kubernetes.io/projected/acda6b08-2faa-4836-888e-d4ecf324cb1d-kube-api-access-tvpkt\") pod \"migrator-66df5d496b-dwlls\" (UID: \"acda6b08-2faa-4836-888e-d4ecf324cb1d\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-dwlls" Oct 31 07:30:27.346884 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.346845 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4tdk\" (UniqueName: \"kubernetes.io/projected/ed522b38-611a-452a-b201-3ca416428d42-kube-api-access-w4tdk\") pod \"multus-additional-cni-plugins-ktzg4\" (UID: \"ed522b38-611a-452a-b201-3ca416428d42\") " pod="openshift-multus/multus-additional-cni-plugins-ktzg4" Oct 31 07:30:27.347043 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.347027 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hldt6\" (UniqueName: \"kubernetes.io/projected/4104ba06-fba8-4c3c-b876-cca5087a7d0a-kube-api-access-hldt6\") pod \"image-registry-69fbc98d44-fn7fh\" (UID: \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\") " pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" Oct 31 07:30:27.347388 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.347372 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzvgd\" (UniqueName: \"kubernetes.io/projected/53c36e19-9ef6-47a2-9bc2-387115539850-kube-api-access-zzvgd\") pod \"node-resolver-scrms\" (UID: \"53c36e19-9ef6-47a2-9bc2-387115539850\") " pod="openshift-dns/node-resolver-scrms" Oct 31 07:30:27.390220 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.390190 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:27.426596 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.426574 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-host-run-multus-certs\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:30:27.426694 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.426608 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/58bc3ea1-f58c-4891-a049-1289a99be055-metrics-tls\") pod \"dns-default-z5sfp\" (UID: \"58bc3ea1-f58c-4891-a049-1289a99be055\") " pod="openshift-dns/dns-default-z5sfp" Oct 31 07:30:27.426694 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.426623 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-host\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:30:27.426694 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.426639 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-var-lib-openvswitch\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:27.426694 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.426655 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-log-socket\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:27.426694 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.426668 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-etc-systemd\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:30:27.426694 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.426683 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4104ba06-fba8-4c3c-b876-cca5087a7d0a-registry-tls\") pod \"image-registry-69fbc98d44-fn7fh\" (UID: \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\") " pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" Oct 31 07:30:27.427002 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.426706 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-host-var-lib-cni-multus\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:30:27.427002 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.426705 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-host\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:30:27.427002 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:27.426719 4935 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 31 07:30:27.427002 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.426723 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-var-lib-openvswitch\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:27.427002 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.426744 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-log-socket\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:27.427002 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.426757 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-host-run-multus-certs\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:30:27.427002 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.426767 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-etc-systemd\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:30:27.427002 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:27.426784 4935 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58bc3ea1-f58c-4891-a049-1289a99be055-metrics-tls podName:58bc3ea1-f58c-4891-a049-1289a99be055 nodeName:}" failed. No retries permitted until 2026-10-31 07:30:27.926767089 +0000 UTC m=+1.797298623 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/58bc3ea1-f58c-4891-a049-1289a99be055-metrics-tls") pod "dns-default-z5sfp" (UID: "58bc3ea1-f58c-4891-a049-1289a99be055") : secret "dns-default-metrics-tls" not found Oct 31 07:30:27.427002 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.426773 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-host-var-lib-cni-multus\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:30:27.427002 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.426733 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f13bc62a-1894-472c-95de-262afa399b38-crio-socket\") pod \"insights-runtime-extractor-mvmr5\" (UID: \"f13bc62a-1894-472c-95de-262afa399b38\") " pod="openshift-insights/insights-runtime-extractor-mvmr5" Oct 31 07:30:27.427002 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:27.426800 4935 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 31 07:30:27.427002 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.426788 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f13bc62a-1894-472c-95de-262afa399b38-crio-socket\") pod \"insights-runtime-extractor-mvmr5\" (UID: \"f13bc62a-1894-472c-95de-262afa399b38\") " pod="openshift-insights/insights-runtime-extractor-mvmr5" Oct 31 07:30:27.427002 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:27.426813 4935 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69fbc98d44-fn7fh: secret "image-registry-tls" not found Oct 31 07:30:27.427002 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.426873 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f13bc62a-1894-472c-95de-262afa399b38-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mvmr5\" (UID: \"f13bc62a-1894-472c-95de-262afa399b38\") " pod="openshift-insights/insights-runtime-extractor-mvmr5" Oct 31 07:30:27.427002 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:27.426893 4935 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4104ba06-fba8-4c3c-b876-cca5087a7d0a-registry-tls podName:4104ba06-fba8-4c3c-b876-cca5087a7d0a nodeName:}" failed. No retries permitted until 2026-10-31 07:30:27.9268758 +0000 UTC m=+1.797407334 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4104ba06-fba8-4c3c-b876-cca5087a7d0a-registry-tls") pod "image-registry-69fbc98d44-fn7fh" (UID: "4104ba06-fba8-4c3c-b876-cca5087a7d0a") : secret "image-registry-tls" not found Oct 31 07:30:27.427002 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.426927 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-host-kubelet\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:27.427002 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:27.426945 4935 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 31 07:30:27.427002 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.426954 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-systemd-units\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:27.427002 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.426978 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28ec9621-7ec5-421c-a88b-51bb20d3a7bf-metrics-certs\") pod \"network-metrics-daemon-7thxr\" (UID: \"28ec9621-7ec5-421c-a88b-51bb20d3a7bf\") " pod="openshift-multus/network-metrics-daemon-7thxr" Oct 31 07:30:27.427002 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.426994 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-host-kubelet\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427020 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-systemd-units\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:27.427027 4935 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f13bc62a-1894-472c-95de-262afa399b38-insights-runtime-extractor-tls podName:f13bc62a-1894-472c-95de-262afa399b38 nodeName:}" failed. No retries permitted until 2026-10-31 07:30:27.927011006 +0000 UTC m=+1.797542523 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f13bc62a-1894-472c-95de-262afa399b38-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mvmr5" (UID: "f13bc62a-1894-472c-95de-262afa399b38") : secret "insights-runtime-extractor-tls" not found Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427079 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-sys\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427110 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-run\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:27.427148 4935 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427154 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/297b35d7-aa65-49c7-8460-ac3f9860248c-kubelet-config\") pod \"global-pull-secret-syncer-8hpcb\" (UID: \"297b35d7-aa65-49c7-8460-ac3f9860248c\") " pod="kube-system/global-pull-secret-syncer-8hpcb" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427167 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-run\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427170 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-sys\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:27.427194 4935 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28ec9621-7ec5-421c-a88b-51bb20d3a7bf-metrics-certs podName:28ec9621-7ec5-421c-a88b-51bb20d3a7bf nodeName:}" failed. No retries permitted until 2026-10-31 07:30:27.927179985 +0000 UTC m=+1.797711523 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/28ec9621-7ec5-421c-a88b-51bb20d3a7bf-metrics-certs") pod "network-metrics-daemon-7thxr" (UID: "28ec9621-7ec5-421c-a88b-51bb20d3a7bf") : secret "metrics-daemon-secret" not found Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427179 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-host-run-netns\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427232 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/297b35d7-aa65-49c7-8460-ac3f9860248c-kubelet-config\") pod \"global-pull-secret-syncer-8hpcb\" (UID: \"297b35d7-aa65-49c7-8460-ac3f9860248c\") " pod="kube-system/global-pull-secret-syncer-8hpcb" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427239 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-etc-sysctl-conf\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427236 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-host-run-netns\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427267 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-system-cni-dir\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427288 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-etc-openvswitch\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427436 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-lib-modules\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427449 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-etc-sysctl-conf\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427452 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-system-cni-dir\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427474 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-etc-openvswitch\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427477 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-multus-socket-dir-parent\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427517 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-var-lib-kubelet\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427525 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-multus-socket-dir-parent\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427536 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-run-systemd\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427552 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ea9beb38-1491-4cf0-83ca-fd79935fa6a9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r78n8\" (UID: \"ea9beb38-1491-4cf0-83ca-fd79935fa6a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r78n8" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427565 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/297b35d7-aa65-49c7-8460-ac3f9860248c-dbus\") pod \"global-pull-secret-syncer-8hpcb\" (UID: \"297b35d7-aa65-49c7-8460-ac3f9860248c\") " pod="kube-system/global-pull-secret-syncer-8hpcb" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427569 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-var-lib-kubelet\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427581 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-lib-modules\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427591 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/53c36e19-9ef6-47a2-9bc2-387115539850-hosts-file\") pod \"node-resolver-scrms\" (UID: \"53c36e19-9ef6-47a2-9bc2-387115539850\") " pod="openshift-dns/node-resolver-scrms" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427612 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ea9beb38-1491-4cf0-83ca-fd79935fa6a9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r78n8\" (UID: \"ea9beb38-1491-4cf0-83ca-fd79935fa6a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r78n8" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427620 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-etc-kubernetes\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427632 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/53c36e19-9ef6-47a2-9bc2-387115539850-hosts-file\") pod \"node-resolver-scrms\" (UID: \"53c36e19-9ef6-47a2-9bc2-387115539850\") " pod="openshift-dns/node-resolver-scrms" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427640 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-host-cni-netd\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427614 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-run-systemd\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427648 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/297b35d7-aa65-49c7-8460-ac3f9860248c-dbus\") pod \"global-pull-secret-syncer-8hpcb\" (UID: \"297b35d7-aa65-49c7-8460-ac3f9860248c\") " pod="kube-system/global-pull-secret-syncer-8hpcb" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427656 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-etc-kubernetes\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427659 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f13bc62a-1894-472c-95de-262afa399b38-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mvmr5\" (UID: \"f13bc62a-1894-472c-95de-262afa399b38\") " pod="openshift-insights/insights-runtime-extractor-mvmr5" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427680 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-host-cni-netd\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427683 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-hostroot\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427701 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-multus-conf-dir\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427718 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-host-var-lib-kubelet\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427735 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ed522b38-611a-452a-b201-3ca416428d42-cnibin\") pod \"multus-additional-cni-plugins-ktzg4\" (UID: \"ed522b38-611a-452a-b201-3ca416428d42\") " pod="openshift-multus/multus-additional-cni-plugins-ktzg4" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427756 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-host-slash\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427762 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-multus-conf-dir\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427781 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ac4bf75a-377c-49b2-818a-7d2436c21e9c-host-slash\") pod \"iptables-alerter-lmsk9\" (UID: \"ac4bf75a-377c-49b2-818a-7d2436c21e9c\") " pod="openshift-network-operator/iptables-alerter-lmsk9" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427789 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-hostroot\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427808 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-host-cni-bin\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427816 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ed522b38-611a-452a-b201-3ca416428d42-cnibin\") pod \"multus-additional-cni-plugins-ktzg4\" (UID: \"ed522b38-611a-452a-b201-3ca416428d42\") " pod="openshift-multus/multus-additional-cni-plugins-ktzg4" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427811 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-host-slash\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427832 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ea9beb38-1491-4cf0-83ca-fd79935fa6a9-device-dir\") pod \"aws-ebs-csi-driver-node-r78n8\" (UID: \"ea9beb38-1491-4cf0-83ca-fd79935fa6a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r78n8" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427848 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ac4bf75a-377c-49b2-818a-7d2436c21e9c-host-slash\") pod \"iptables-alerter-lmsk9\" (UID: \"ac4bf75a-377c-49b2-818a-7d2436c21e9c\") " pod="openshift-network-operator/iptables-alerter-lmsk9" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427782 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-host-var-lib-kubelet\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427856 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-etc-sysconfig\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427853 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-host-cni-bin\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427895 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-etc-sysconfig\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427896 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ea9beb38-1491-4cf0-83ca-fd79935fa6a9-device-dir\") pod \"aws-ebs-csi-driver-node-r78n8\" (UID: \"ea9beb38-1491-4cf0-83ca-fd79935fa6a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r78n8" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427919 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-host-var-lib-cni-bin\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427943 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/563d3b8b-0040-439e-b0b2-e858eed5abf6-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-s75pm\" (UID: \"563d3b8b-0040-439e-b0b2-e858eed5abf6\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-s75pm" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427953 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-host-var-lib-cni-bin\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427959 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ea9beb38-1491-4cf0-83ca-fd79935fa6a9-sys-fs\") pod \"aws-ebs-csi-driver-node-r78n8\" (UID: \"ea9beb38-1491-4cf0-83ca-fd79935fa6a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r78n8" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427974 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ed522b38-611a-452a-b201-3ca416428d42-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ktzg4\" (UID: \"ed522b38-611a-452a-b201-3ca416428d42\") " pod="openshift-multus/multus-additional-cni-plugins-ktzg4" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.427993 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-etc-sysctl-d\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.428026 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ea9beb38-1491-4cf0-83ca-fd79935fa6a9-sys-fs\") pod \"aws-ebs-csi-driver-node-r78n8\" (UID: \"ea9beb38-1491-4cf0-83ca-fd79935fa6a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r78n8" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.428029 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/236238a7-255d-40f7-a99b-d65903b3bcb7-host\") pod \"node-ca-rhg7w\" (UID: \"236238a7-255d-40f7-a99b-d65903b3bcb7\") " pod="openshift-image-registry/node-ca-rhg7w" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.428036 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ed522b38-611a-452a-b201-3ca416428d42-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ktzg4\" (UID: \"ed522b38-611a-452a-b201-3ca416428d42\") " pod="openshift-multus/multus-additional-cni-plugins-ktzg4" Oct 31 07:30:27.427999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.428054 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/236238a7-255d-40f7-a99b-d65903b3bcb7-host\") pod \"node-ca-rhg7w\" (UID: \"236238a7-255d-40f7-a99b-d65903b3bcb7\") " pod="openshift-image-registry/node-ca-rhg7w" Oct 31 07:30:27.429906 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.428072 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-host-run-netns\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:27.429906 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:27.428091 4935 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Oct 31 07:30:27.429906 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.428111 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-etc-sysctl-d\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:30:27.429906 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.428124 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-node-log\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:27.429906 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.428125 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-host-run-netns\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:27.429906 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:27.428164 4935 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/563d3b8b-0040-439e-b0b2-e858eed5abf6-networking-console-plugin-cert podName:563d3b8b-0040-439e-b0b2-e858eed5abf6 nodeName:}" failed. No retries permitted until 2026-10-31 07:30:27.928148436 +0000 UTC m=+1.798679966 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/563d3b8b-0040-439e-b0b2-e858eed5abf6-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-s75pm" (UID: "563d3b8b-0040-439e-b0b2-e858eed5abf6") : secret "networking-console-plugin-cert" not found Oct 31 07:30:27.429906 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.428168 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f13bc62a-1894-472c-95de-262afa399b38-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mvmr5\" (UID: \"f13bc62a-1894-472c-95de-262afa399b38\") " pod="openshift-insights/insights-runtime-extractor-mvmr5" Oct 31 07:30:27.429906 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.428094 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-node-log\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:27.429906 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.428208 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-etc-modprobe-d\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:30:27.429906 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.428226 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-host-run-ovn-kubernetes\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:27.429906 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.428243 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:27.429906 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.428259 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ea9beb38-1491-4cf0-83ca-fd79935fa6a9-registration-dir\") pod \"aws-ebs-csi-driver-node-r78n8\" (UID: \"ea9beb38-1491-4cf0-83ca-fd79935fa6a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r78n8" Oct 31 07:30:27.429906 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.428273 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-registry-tls\") pod \"image-registry-54854c7897-pc7km\" (UID: \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\") " pod="openshift-image-registry/image-registry-54854c7897-pc7km" Oct 31 07:30:27.429906 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.428289 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-host-run-k8s-cni-cncf-io\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:30:27.429906 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.428294 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-host-run-ovn-kubernetes\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:27.429906 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.428307 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:27.429906 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.428320 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-cnibin\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:30:27.429906 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.428344 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ea9beb38-1491-4cf0-83ca-fd79935fa6a9-registration-dir\") pod \"aws-ebs-csi-driver-node-r78n8\" (UID: \"ea9beb38-1491-4cf0-83ca-fd79935fa6a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r78n8" Oct 31 07:30:27.429906 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.428380 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-host-run-k8s-cni-cncf-io\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:30:27.429906 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.428391 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-cnibin\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:30:27.429906 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.428392 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/046b1de8-d709-4729-9aec-391a2b3294d5-etc-modprobe-d\") pod \"tuned-sqxmw\" (UID: \"046b1de8-d709-4729-9aec-391a2b3294d5\") " pod="openshift-cluster-node-tuning-operator/tuned-sqxmw" Oct 31 07:30:27.429906 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:27.428404 4935 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 31 07:30:27.429906 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:27.428414 4935 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-54854c7897-pc7km: secret "image-registry-tls" not found Oct 31 07:30:27.429906 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.428412 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-etc-kubernetes\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:30:27.429906 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.428436 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-run-ovn\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:27.429906 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:27.428452 4935 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-registry-tls podName:c734ebf9-eba8-4e14-b20d-2d78dd65b10d nodeName:}" failed. No retries permitted until 2026-10-31 07:30:27.928441302 +0000 UTC m=+1.798972837 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-registry-tls") pod "image-registry-54854c7897-pc7km" (UID: "c734ebf9-eba8-4e14-b20d-2d78dd65b10d") : secret "image-registry-tls" not found Oct 31 07:30:27.429906 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.428459 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-run-ovn\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:27.429906 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.428479 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ed522b38-611a-452a-b201-3ca416428d42-system-cni-dir\") pod \"multus-additional-cni-plugins-ktzg4\" (UID: \"ed522b38-611a-452a-b201-3ca416428d42\") " pod="openshift-multus/multus-additional-cni-plugins-ktzg4" Oct 31 07:30:27.429906 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.428485 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-etc-kubernetes\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:30:27.429906 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.428498 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-multus-cni-dir\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:30:27.429906 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.428515 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-run-openvswitch\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:27.429906 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.428532 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ea9beb38-1491-4cf0-83ca-fd79935fa6a9-etc-selinux\") pod \"aws-ebs-csi-driver-node-r78n8\" (UID: \"ea9beb38-1491-4cf0-83ca-fd79935fa6a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r78n8" Oct 31 07:30:27.429906 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.428530 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ed522b38-611a-452a-b201-3ca416428d42-system-cni-dir\") pod \"multus-additional-cni-plugins-ktzg4\" (UID: \"ed522b38-611a-452a-b201-3ca416428d42\") " pod="openshift-multus/multus-additional-cni-plugins-ktzg4" Oct 31 07:30:27.429906 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.428550 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ea9beb38-1491-4cf0-83ca-fd79935fa6a9-socket-dir\") pod \"aws-ebs-csi-driver-node-r78n8\" (UID: \"ea9beb38-1491-4cf0-83ca-fd79935fa6a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r78n8" Oct 31 07:30:27.429906 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.428570 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e00eb9e3-0d3c-4541-b238-4130fe1de2f1-run-openvswitch\") pod \"ovnkube-node-98whn\" (UID: \"e00eb9e3-0d3c-4541-b238-4130fe1de2f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:27.429906 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.428578 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6ae10413-8f49-4dfe-8aad-00dcb9d390c8-multus-cni-dir\") pod \"multus-w6vnh\" (UID: \"6ae10413-8f49-4dfe-8aad-00dcb9d390c8\") " pod="openshift-multus/multus-w6vnh" Oct 31 07:30:27.429906 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.428602 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ea9beb38-1491-4cf0-83ca-fd79935fa6a9-etc-selinux\") pod \"aws-ebs-csi-driver-node-r78n8\" (UID: \"ea9beb38-1491-4cf0-83ca-fd79935fa6a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r78n8" Oct 31 07:30:27.429906 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.428629 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ea9beb38-1491-4cf0-83ca-fd79935fa6a9-socket-dir\") pod \"aws-ebs-csi-driver-node-r78n8\" (UID: \"ea9beb38-1491-4cf0-83ca-fd79935fa6a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r78n8" Oct 31 07:30:27.502622 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.502605 4935 scope.go:122] "RemoveContainer" containerID="25aec100393d03aba086bd02f13db3038b24b6ae07422807f4c23edbb520f3f5" Oct 31 07:30:27.593976 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.593946 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:27.801752 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.801669 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:27.832606 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.832578 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28be803a-a932-4009-ad61-b356366821ef-metrics-certs\") pod \"router-default-5549647ff-cxcdt\" (UID: \"28be803a-a932-4009-ad61-b356366821ef\") " pod="openshift-ingress/router-default-5549647ff-cxcdt" Oct 31 07:30:27.832745 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.832640 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-86n8d\" (UID: \"d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-86n8d" Oct 31 07:30:27.832745 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.832688 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5e743234-2b5f-4c91-9b06-189f546cbce4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-282kr\" (UID: \"5e743234-2b5f-4c91-9b06-189f546cbce4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-282kr" Oct 31 07:30:27.832745 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:27.832725 4935 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Oct 31 07:30:27.832887 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:27.832781 4935 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28be803a-a932-4009-ad61-b356366821ef-metrics-certs podName:28be803a-a932-4009-ad61-b356366821ef nodeName:}" failed. No retries permitted until 2026-10-31 07:30:28.832766687 +0000 UTC m=+2.703298202 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/28be803a-a932-4009-ad61-b356366821ef-metrics-certs") pod "router-default-5549647ff-cxcdt" (UID: "28be803a-a932-4009-ad61-b356366821ef") : secret "router-metrics-certs-default" not found Oct 31 07:30:27.832887 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:27.832790 4935 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Oct 31 07:30:27.832887 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.832807 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28be803a-a932-4009-ad61-b356366821ef-service-ca-bundle\") pod \"router-default-5549647ff-cxcdt\" (UID: \"28be803a-a932-4009-ad61-b356366821ef\") " pod="openshift-ingress/router-default-5549647ff-cxcdt" Oct 31 07:30:27.832887 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:27.832830 4935 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 31 07:30:27.832887 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:27.832841 4935 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e-samples-operator-tls podName:d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e nodeName:}" failed. No retries permitted until 2026-10-31 07:30:28.832826869 +0000 UTC m=+2.703358401 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-86n8d" (UID: "d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e") : secret "samples-operator-tls" not found Oct 31 07:30:27.832887 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:27.832865 4935 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/28be803a-a932-4009-ad61-b356366821ef-service-ca-bundle podName:28be803a-a932-4009-ad61-b356366821ef nodeName:}" failed. No retries permitted until 2026-10-31 07:30:28.832857245 +0000 UTC m=+2.703388760 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/28be803a-a932-4009-ad61-b356366821ef-service-ca-bundle") pod "router-default-5549647ff-cxcdt" (UID: "28be803a-a932-4009-ad61-b356366821ef") : configmap references non-existent config key: service-ca.crt Oct 31 07:30:27.832887 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:27.832880 4935 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e743234-2b5f-4c91-9b06-189f546cbce4-cluster-monitoring-operator-tls podName:5e743234-2b5f-4c91-9b06-189f546cbce4 nodeName:}" failed. No retries permitted until 2026-10-31 07:30:28.832872317 +0000 UTC m=+2.703403835 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5e743234-2b5f-4c91-9b06-189f546cbce4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-282kr" (UID: "5e743234-2b5f-4c91-9b06-189f546cbce4") : secret "cluster-monitoring-operator-tls" not found Oct 31 07:30:27.933250 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.933225 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/563d3b8b-0040-439e-b0b2-e858eed5abf6-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-s75pm\" (UID: \"563d3b8b-0040-439e-b0b2-e858eed5abf6\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-s75pm" Oct 31 07:30:27.933378 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.933266 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-registry-tls\") pod \"image-registry-54854c7897-pc7km\" (UID: \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\") " pod="openshift-image-registry/image-registry-54854c7897-pc7km" Oct 31 07:30:27.933378 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.933315 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/58bc3ea1-f58c-4891-a049-1289a99be055-metrics-tls\") pod \"dns-default-z5sfp\" (UID: \"58bc3ea1-f58c-4891-a049-1289a99be055\") " pod="openshift-dns/dns-default-z5sfp" Oct 31 07:30:27.933378 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.933335 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4104ba06-fba8-4c3c-b876-cca5087a7d0a-registry-tls\") pod \"image-registry-69fbc98d44-fn7fh\" (UID: \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\") " pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" Oct 31 07:30:27.933378 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:27.933371 4935 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Oct 31 07:30:27.933559 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.933379 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f13bc62a-1894-472c-95de-262afa399b38-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mvmr5\" (UID: \"f13bc62a-1894-472c-95de-262afa399b38\") " pod="openshift-insights/insights-runtime-extractor-mvmr5" Oct 31 07:30:27.933559 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:27.933409 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28ec9621-7ec5-421c-a88b-51bb20d3a7bf-metrics-certs\") pod \"network-metrics-daemon-7thxr\" (UID: \"28ec9621-7ec5-421c-a88b-51bb20d3a7bf\") " pod="openshift-multus/network-metrics-daemon-7thxr" Oct 31 07:30:27.933559 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:27.933427 4935 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/563d3b8b-0040-439e-b0b2-e858eed5abf6-networking-console-plugin-cert podName:563d3b8b-0040-439e-b0b2-e858eed5abf6 nodeName:}" failed. No retries permitted until 2026-10-31 07:30:28.933410214 +0000 UTC m=+2.803941742 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/563d3b8b-0040-439e-b0b2-e858eed5abf6-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-s75pm" (UID: "563d3b8b-0040-439e-b0b2-e858eed5abf6") : secret "networking-console-plugin-cert" not found Oct 31 07:30:27.933559 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:27.933447 4935 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 31 07:30:27.933559 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:27.933462 4935 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-54854c7897-pc7km: secret "image-registry-tls" not found Oct 31 07:30:27.933559 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:27.933470 4935 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Oct 31 07:30:27.933559 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:27.933473 4935 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 31 07:30:27.933559 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:27.933490 4935 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69fbc98d44-fn7fh: secret "image-registry-tls" not found Oct 31 07:30:27.933559 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:27.933489 4935 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 31 07:30:27.933559 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:27.933503 4935 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-registry-tls podName:c734ebf9-eba8-4e14-b20d-2d78dd65b10d nodeName:}" failed. No retries permitted until 2026-10-31 07:30:28.93349029 +0000 UTC m=+2.804021822 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-registry-tls") pod "image-registry-54854c7897-pc7km" (UID: "c734ebf9-eba8-4e14-b20d-2d78dd65b10d") : secret "image-registry-tls" not found Oct 31 07:30:27.933559 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:27.933472 4935 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 31 07:30:27.933559 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:27.933519 4935 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28ec9621-7ec5-421c-a88b-51bb20d3a7bf-metrics-certs podName:28ec9621-7ec5-421c-a88b-51bb20d3a7bf nodeName:}" failed. No retries permitted until 2026-10-31 07:30:28.93350921 +0000 UTC m=+2.804040748 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/28ec9621-7ec5-421c-a88b-51bb20d3a7bf-metrics-certs") pod "network-metrics-daemon-7thxr" (UID: "28ec9621-7ec5-421c-a88b-51bb20d3a7bf") : secret "metrics-daemon-secret" not found Oct 31 07:30:27.933559 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:27.933535 4935 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f13bc62a-1894-472c-95de-262afa399b38-insights-runtime-extractor-tls podName:f13bc62a-1894-472c-95de-262afa399b38 nodeName:}" failed. No retries permitted until 2026-10-31 07:30:28.933526672 +0000 UTC m=+2.804058206 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f13bc62a-1894-472c-95de-262afa399b38-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mvmr5" (UID: "f13bc62a-1894-472c-95de-262afa399b38") : secret "insights-runtime-extractor-tls" not found Oct 31 07:30:27.933559 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:27.933550 4935 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4104ba06-fba8-4c3c-b876-cca5087a7d0a-registry-tls podName:4104ba06-fba8-4c3c-b876-cca5087a7d0a nodeName:}" failed. No retries permitted until 2026-10-31 07:30:28.933543241 +0000 UTC m=+2.804074755 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4104ba06-fba8-4c3c-b876-cca5087a7d0a-registry-tls") pod "image-registry-69fbc98d44-fn7fh" (UID: "4104ba06-fba8-4c3c-b876-cca5087a7d0a") : secret "image-registry-tls" not found Oct 31 07:30:27.933559 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:27.933565 4935 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58bc3ea1-f58c-4891-a049-1289a99be055-metrics-tls podName:58bc3ea1-f58c-4891-a049-1289a99be055 nodeName:}" failed. No retries permitted until 2026-10-31 07:30:28.933556822 +0000 UTC m=+2.804088341 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/58bc3ea1-f58c-4891-a049-1289a99be055-metrics-tls") pod "dns-default-z5sfp" (UID: "58bc3ea1-f58c-4891-a049-1289a99be055") : secret "dns-default-metrics-tls" not found Oct 31 07:30:28.001499 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:28.001465 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:28.293435 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:28.293411 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jztcz_396e1c5e-3700-4fe6-b367-54ad2a244225/console-operator/2.log" Oct 31 07:30:28.293835 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:28.293822 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jztcz_396e1c5e-3700-4fe6-b367-54ad2a244225/console-operator/1.log" Oct 31 07:30:28.293875 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:28.293852 4935 generic.go:356] "Generic (PLEG): container finished" podID="396e1c5e-3700-4fe6-b367-54ad2a244225" containerID="392988b2f5bd601185b02a171468b8153e144d07a9e60e83d903b2ed879be684" exitCode=255 Oct 31 07:30:28.293941 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:28.293922 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-jztcz" event={"ID":"396e1c5e-3700-4fe6-b367-54ad2a244225","Type":"ContainerDied","Data":"392988b2f5bd601185b02a171468b8153e144d07a9e60e83d903b2ed879be684"} Oct 31 07:30:28.294022 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:28.293981 4935 scope.go:122] "RemoveContainer" containerID="25aec100393d03aba086bd02f13db3038b24b6ae07422807f4c23edbb520f3f5" Oct 31 07:30:28.294515 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:28.294500 4935 scope.go:122] "RemoveContainer" containerID="392988b2f5bd601185b02a171468b8153e144d07a9e60e83d903b2ed879be684" Oct 31 07:30:28.294692 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:28.294672 4935 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-jztcz_openshift-console-operator(396e1c5e-3700-4fe6-b367-54ad2a244225)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-jztcz" podUID="396e1c5e-3700-4fe6-b367-54ad2a244225" Oct 31 07:30:28.556411 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:28.556326 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-scrms_53c36e19-9ef6-47a2-9bc2-387115539850/dns-node-resolver/0.log" Oct 31 07:30:28.841358 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:28.841279 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28be803a-a932-4009-ad61-b356366821ef-service-ca-bundle\") pod \"router-default-5549647ff-cxcdt\" (UID: \"28be803a-a932-4009-ad61-b356366821ef\") " pod="openshift-ingress/router-default-5549647ff-cxcdt" Oct 31 07:30:28.841358 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:28.841336 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28be803a-a932-4009-ad61-b356366821ef-metrics-certs\") pod \"router-default-5549647ff-cxcdt\" (UID: \"28be803a-a932-4009-ad61-b356366821ef\") " pod="openshift-ingress/router-default-5549647ff-cxcdt" Oct 31 07:30:28.841535 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:28.841383 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-86n8d\" (UID: \"d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-86n8d" Oct 31 07:30:28.841535 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:28.841413 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5e743234-2b5f-4c91-9b06-189f546cbce4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-282kr\" (UID: \"5e743234-2b5f-4c91-9b06-189f546cbce4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-282kr" Oct 31 07:30:28.841535 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:28.841442 4935 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/28be803a-a932-4009-ad61-b356366821ef-service-ca-bundle podName:28be803a-a932-4009-ad61-b356366821ef nodeName:}" failed. No retries permitted until 2026-10-31 07:30:30.841420453 +0000 UTC m=+4.711951985 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/28be803a-a932-4009-ad61-b356366821ef-service-ca-bundle") pod "router-default-5549647ff-cxcdt" (UID: "28be803a-a932-4009-ad61-b356366821ef") : configmap references non-existent config key: service-ca.crt Oct 31 07:30:28.841535 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:28.841475 4935 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 31 07:30:28.841535 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:28.841476 4935 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Oct 31 07:30:28.841535 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:28.841487 4935 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Oct 31 07:30:28.841535 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:28.841513 4935 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e743234-2b5f-4c91-9b06-189f546cbce4-cluster-monitoring-operator-tls podName:5e743234-2b5f-4c91-9b06-189f546cbce4 nodeName:}" failed. No retries permitted until 2026-10-31 07:30:30.841502818 +0000 UTC m=+4.712034352 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5e743234-2b5f-4c91-9b06-189f546cbce4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-282kr" (UID: "5e743234-2b5f-4c91-9b06-189f546cbce4") : secret "cluster-monitoring-operator-tls" not found Oct 31 07:30:28.841871 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:28.841545 4935 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28be803a-a932-4009-ad61-b356366821ef-metrics-certs podName:28be803a-a932-4009-ad61-b356366821ef nodeName:}" failed. No retries permitted until 2026-10-31 07:30:30.84153868 +0000 UTC m=+4.712070195 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/28be803a-a932-4009-ad61-b356366821ef-metrics-certs") pod "router-default-5549647ff-cxcdt" (UID: "28be803a-a932-4009-ad61-b356366821ef") : secret "router-metrics-certs-default" not found Oct 31 07:30:28.841871 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:28.841558 4935 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e-samples-operator-tls podName:d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e nodeName:}" failed. No retries permitted until 2026-10-31 07:30:30.841552912 +0000 UTC m=+4.712084427 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-86n8d" (UID: "d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e") : secret "samples-operator-tls" not found Oct 31 07:30:28.941799 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:28.941771 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/563d3b8b-0040-439e-b0b2-e858eed5abf6-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-s75pm\" (UID: \"563d3b8b-0040-439e-b0b2-e858eed5abf6\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-s75pm" Oct 31 07:30:28.941926 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:28.941812 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-registry-tls\") pod \"image-registry-54854c7897-pc7km\" (UID: \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\") " pod="openshift-image-registry/image-registry-54854c7897-pc7km" Oct 31 07:30:28.941926 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:28.941888 4935 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Oct 31 07:30:28.942313 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:28.942223 4935 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 31 07:30:28.942313 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:28.942231 4935 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/563d3b8b-0040-439e-b0b2-e858eed5abf6-networking-console-plugin-cert podName:563d3b8b-0040-439e-b0b2-e858eed5abf6 nodeName:}" failed. No retries permitted until 2026-10-31 07:30:30.941916744 +0000 UTC m=+4.812448291 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/563d3b8b-0040-439e-b0b2-e858eed5abf6-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-s75pm" (UID: "563d3b8b-0040-439e-b0b2-e858eed5abf6") : secret "networking-console-plugin-cert" not found Oct 31 07:30:28.942313 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:28.942246 4935 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-54854c7897-pc7km: secret "image-registry-tls" not found Oct 31 07:30:28.942313 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:28.942304 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/58bc3ea1-f58c-4891-a049-1289a99be055-metrics-tls\") pod \"dns-default-z5sfp\" (UID: \"58bc3ea1-f58c-4891-a049-1289a99be055\") " pod="openshift-dns/dns-default-z5sfp" Oct 31 07:30:28.942313 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:28.942319 4935 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-registry-tls podName:c734ebf9-eba8-4e14-b20d-2d78dd65b10d nodeName:}" failed. No retries permitted until 2026-10-31 07:30:30.942304144 +0000 UTC m=+4.812835676 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-registry-tls") pod "image-registry-54854c7897-pc7km" (UID: "c734ebf9-eba8-4e14-b20d-2d78dd65b10d") : secret "image-registry-tls" not found Oct 31 07:30:28.943683 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:28.942396 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4104ba06-fba8-4c3c-b876-cca5087a7d0a-registry-tls\") pod \"image-registry-69fbc98d44-fn7fh\" (UID: \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\") " pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" Oct 31 07:30:28.943683 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:28.942416 4935 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 31 07:30:28.943683 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:28.942466 4935 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58bc3ea1-f58c-4891-a049-1289a99be055-metrics-tls podName:58bc3ea1-f58c-4891-a049-1289a99be055 nodeName:}" failed. No retries permitted until 2026-10-31 07:30:30.942453684 +0000 UTC m=+4.812985202 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/58bc3ea1-f58c-4891-a049-1289a99be055-metrics-tls") pod "dns-default-z5sfp" (UID: "58bc3ea1-f58c-4891-a049-1289a99be055") : secret "dns-default-metrics-tls" not found Oct 31 07:30:28.943683 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:28.942496 4935 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 31 07:30:28.943683 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:28.942509 4935 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69fbc98d44-fn7fh: secret "image-registry-tls" not found Oct 31 07:30:28.943683 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:28.942557 4935 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4104ba06-fba8-4c3c-b876-cca5087a7d0a-registry-tls podName:4104ba06-fba8-4c3c-b876-cca5087a7d0a nodeName:}" failed. No retries permitted until 2026-10-31 07:30:30.942543674 +0000 UTC m=+4.813075204 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4104ba06-fba8-4c3c-b876-cca5087a7d0a-registry-tls") pod "image-registry-69fbc98d44-fn7fh" (UID: "4104ba06-fba8-4c3c-b876-cca5087a7d0a") : secret "image-registry-tls" not found Oct 31 07:30:28.943683 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:28.942495 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f13bc62a-1894-472c-95de-262afa399b38-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mvmr5\" (UID: \"f13bc62a-1894-472c-95de-262afa399b38\") " pod="openshift-insights/insights-runtime-extractor-mvmr5" Oct 31 07:30:28.943683 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:28.942611 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28ec9621-7ec5-421c-a88b-51bb20d3a7bf-metrics-certs\") pod \"network-metrics-daemon-7thxr\" (UID: \"28ec9621-7ec5-421c-a88b-51bb20d3a7bf\") " pod="openshift-multus/network-metrics-daemon-7thxr" Oct 31 07:30:28.943683 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:28.942748 4935 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Oct 31 07:30:28.943683 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:28.942794 4935 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28ec9621-7ec5-421c-a88b-51bb20d3a7bf-metrics-certs podName:28ec9621-7ec5-421c-a88b-51bb20d3a7bf nodeName:}" failed. No retries permitted until 2026-10-31 07:30:30.942780815 +0000 UTC m=+4.813312358 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/28ec9621-7ec5-421c-a88b-51bb20d3a7bf-metrics-certs") pod "network-metrics-daemon-7thxr" (UID: "28ec9621-7ec5-421c-a88b-51bb20d3a7bf") : secret "metrics-daemon-secret" not found Oct 31 07:30:28.943683 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:28.942865 4935 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Oct 31 07:30:28.946786 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:28.946744 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f13bc62a-1894-472c-95de-262afa399b38-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mvmr5\" (UID: \"f13bc62a-1894-472c-95de-262afa399b38\") " pod="openshift-insights/insights-runtime-extractor-mvmr5" Oct 31 07:30:29.018836 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:29.018815 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mvmr5" Oct 31 07:30:29.032275 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:29.032249 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:29.236470 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:29.236441 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:29.289304 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:29.289279 4935 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mvmr5"] Oct 31 07:30:29.292649 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:29.292625 4935 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf13bc62a_1894_472c_95de_262afa399b38.slice/crio-e94c14eaf0a82a4adc316439657f45cece8dc4ab2c6b097b797f2ec558e0ebc9 WatchSource:0}: Error finding container e94c14eaf0a82a4adc316439657f45cece8dc4ab2c6b097b797f2ec558e0ebc9: Status 404 returned error can't find the container with id e94c14eaf0a82a4adc316439657f45cece8dc4ab2c6b097b797f2ec558e0ebc9 Oct 31 07:30:29.298568 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:29.298551 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jztcz_396e1c5e-3700-4fe6-b367-54ad2a244225/console-operator/2.log" Oct 31 07:30:29.299753 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:29.299730 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mvmr5" event={"ID":"f13bc62a-1894-472c-95de-262afa399b38","Type":"ContainerStarted","Data":"e94c14eaf0a82a4adc316439657f45cece8dc4ab2c6b097b797f2ec558e0ebc9"} Oct 31 07:30:29.346567 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:29.346548 4935 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 31 07:30:29.356388 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:29.356371 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rhg7w_236238a7-255d-40f7-a99b-d65903b3bcb7/node-ca/0.log" Oct 31 07:30:29.438520 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:29.438490 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:29.639837 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:29.639765 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:29.841907 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:29.841882 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:29.956458 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:29.956434 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-22cmd_00a0d978-8020-46df-932d-b0dd22f946f9/serve-healthcheck-canary/0.log" Oct 31 07:30:30.238707 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:30.238634 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:30.305090 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:30.305060 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mvmr5" event={"ID":"f13bc62a-1894-472c-95de-262afa399b38","Type":"ContainerStarted","Data":"9fd94cb4e19c3f27adde6137115030b50def4f226e750570c9930120b5edefbf"} Oct 31 07:30:30.305437 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:30.305119 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mvmr5" event={"ID":"f13bc62a-1894-472c-95de-262afa399b38","Type":"ContainerStarted","Data":"e66fbd20573404aa97454bbfbc2729cdf32f1cd4edb52ce252d5207ec253bad5"} Oct 31 07:30:30.356201 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:30.356179 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-66df5d496b-dwlls_acda6b08-2faa-4836-888e-d4ecf324cb1d/migrator/0.log" Oct 31 07:30:30.718660 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:30.718626 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:30.758490 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:30.758467 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7d4bc45dd6-46qvv_f3bf19fd-bc58-4dff-a1a8-4f61381e1fde/kube-storage-version-migrator-operator/0.log" Oct 31 07:30:30.860779 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:30.860745 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28be803a-a932-4009-ad61-b356366821ef-metrics-certs\") pod \"router-default-5549647ff-cxcdt\" (UID: \"28be803a-a932-4009-ad61-b356366821ef\") " pod="openshift-ingress/router-default-5549647ff-cxcdt" Oct 31 07:30:30.860949 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:30.860805 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-86n8d\" (UID: \"d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-86n8d" Oct 31 07:30:30.860949 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:30.860849 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5e743234-2b5f-4c91-9b06-189f546cbce4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-282kr\" (UID: \"5e743234-2b5f-4c91-9b06-189f546cbce4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-282kr" Oct 31 07:30:30.860949 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:30.860929 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28be803a-a932-4009-ad61-b356366821ef-service-ca-bundle\") pod \"router-default-5549647ff-cxcdt\" (UID: \"28be803a-a932-4009-ad61-b356366821ef\") " pod="openshift-ingress/router-default-5549647ff-cxcdt" Oct 31 07:30:30.861309 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:30.861258 4935 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 31 07:30:30.861458 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:30.861328 4935 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e743234-2b5f-4c91-9b06-189f546cbce4-cluster-monitoring-operator-tls podName:5e743234-2b5f-4c91-9b06-189f546cbce4 nodeName:}" failed. No retries permitted until 2026-10-31 07:30:34.861309996 +0000 UTC m=+8.731841531 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5e743234-2b5f-4c91-9b06-189f546cbce4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-282kr" (UID: "5e743234-2b5f-4c91-9b06-189f546cbce4") : secret "cluster-monitoring-operator-tls" not found Oct 31 07:30:30.861625 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:30.861606 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28be803a-a932-4009-ad61-b356366821ef-service-ca-bundle\") pod \"router-default-5549647ff-cxcdt\" (UID: \"28be803a-a932-4009-ad61-b356366821ef\") " pod="openshift-ingress/router-default-5549647ff-cxcdt" Oct 31 07:30:30.863594 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:30.863569 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-86n8d\" (UID: \"d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-86n8d" Oct 31 07:30:30.863688 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:30.863675 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28be803a-a932-4009-ad61-b356366821ef-metrics-certs\") pod \"router-default-5549647ff-cxcdt\" (UID: \"28be803a-a932-4009-ad61-b356366821ef\") " pod="openshift-ingress/router-default-5549647ff-cxcdt" Oct 31 07:30:30.920803 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:30.920744 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:30.962099 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:30.962072 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/563d3b8b-0040-439e-b0b2-e858eed5abf6-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-s75pm\" (UID: \"563d3b8b-0040-439e-b0b2-e858eed5abf6\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-s75pm" Oct 31 07:30:30.962222 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:30.962116 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-registry-tls\") pod \"image-registry-54854c7897-pc7km\" (UID: \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\") " pod="openshift-image-registry/image-registry-54854c7897-pc7km" Oct 31 07:30:30.962222 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:30.962170 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/58bc3ea1-f58c-4891-a049-1289a99be055-metrics-tls\") pod \"dns-default-z5sfp\" (UID: \"58bc3ea1-f58c-4891-a049-1289a99be055\") " pod="openshift-dns/dns-default-z5sfp" Oct 31 07:30:30.962222 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:30.962199 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4104ba06-fba8-4c3c-b876-cca5087a7d0a-registry-tls\") pod \"image-registry-69fbc98d44-fn7fh\" (UID: \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\") " pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" Oct 31 07:30:30.962394 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:30.962232 4935 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Oct 31 07:30:30.962394 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:30.962231 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28ec9621-7ec5-421c-a88b-51bb20d3a7bf-metrics-certs\") pod \"network-metrics-daemon-7thxr\" (UID: \"28ec9621-7ec5-421c-a88b-51bb20d3a7bf\") " pod="openshift-multus/network-metrics-daemon-7thxr" Oct 31 07:30:30.962394 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:30.962293 4935 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/563d3b8b-0040-439e-b0b2-e858eed5abf6-networking-console-plugin-cert podName:563d3b8b-0040-439e-b0b2-e858eed5abf6 nodeName:}" failed. No retries permitted until 2026-10-31 07:30:34.962276496 +0000 UTC m=+8.832808030 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/563d3b8b-0040-439e-b0b2-e858eed5abf6-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-s75pm" (UID: "563d3b8b-0040-439e-b0b2-e858eed5abf6") : secret "networking-console-plugin-cert" not found Oct 31 07:30:30.962394 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:30.962290 4935 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 31 07:30:30.962394 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:30.962364 4935 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Oct 31 07:30:30.962394 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:30.962378 4935 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-54854c7897-pc7km: secret "image-registry-tls" not found Oct 31 07:30:30.962589 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:30.962294 4935 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 31 07:30:30.962589 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:30.962427 4935 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69fbc98d44-fn7fh: secret "image-registry-tls" not found Oct 31 07:30:30.962589 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:30.962300 4935 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 31 07:30:30.962589 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:30.962428 4935 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28ec9621-7ec5-421c-a88b-51bb20d3a7bf-metrics-certs podName:28ec9621-7ec5-421c-a88b-51bb20d3a7bf nodeName:}" failed. No retries permitted until 2026-10-31 07:30:34.962409642 +0000 UTC m=+8.832941161 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/28ec9621-7ec5-421c-a88b-51bb20d3a7bf-metrics-certs") pod "network-metrics-daemon-7thxr" (UID: "28ec9621-7ec5-421c-a88b-51bb20d3a7bf") : secret "metrics-daemon-secret" not found Oct 31 07:30:30.962589 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:30.962471 4935 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-registry-tls podName:c734ebf9-eba8-4e14-b20d-2d78dd65b10d nodeName:}" failed. No retries permitted until 2026-10-31 07:30:34.962456939 +0000 UTC m=+8.832988454 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-registry-tls") pod "image-registry-54854c7897-pc7km" (UID: "c734ebf9-eba8-4e14-b20d-2d78dd65b10d") : secret "image-registry-tls" not found Oct 31 07:30:30.962589 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:30.962486 4935 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4104ba06-fba8-4c3c-b876-cca5087a7d0a-registry-tls podName:4104ba06-fba8-4c3c-b876-cca5087a7d0a nodeName:}" failed. No retries permitted until 2026-10-31 07:30:34.962477419 +0000 UTC m=+8.833008933 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4104ba06-fba8-4c3c-b876-cca5087a7d0a-registry-tls") pod "image-registry-69fbc98d44-fn7fh" (UID: "4104ba06-fba8-4c3c-b876-cca5087a7d0a") : secret "image-registry-tls" not found Oct 31 07:30:30.962589 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:30.962500 4935 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58bc3ea1-f58c-4891-a049-1289a99be055-metrics-tls podName:58bc3ea1-f58c-4891-a049-1289a99be055 nodeName:}" failed. No retries permitted until 2026-10-31 07:30:34.962492345 +0000 UTC m=+8.833023861 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/58bc3ea1-f58c-4891-a049-1289a99be055-metrics-tls") pod "dns-default-z5sfp" (UID: "58bc3ea1-f58c-4891-a049-1289a99be055") : secret "dns-default-metrics-tls" not found Oct 31 07:30:31.102734 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:31.102655 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5549647ff-cxcdt" Oct 31 07:30:31.103677 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:31.103655 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-86n8d" Oct 31 07:30:31.118818 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:31.118792 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:31.324777 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:31.324749 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:31.386454 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:31.386430 4935 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5549647ff-cxcdt"] Oct 31 07:30:31.386912 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:31.386890 4935 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-86n8d"] Oct 31 07:30:31.389846 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:31.389815 4935 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28be803a_a932_4009_ad61_b356366821ef.slice/crio-dc72baf50eb70ade42b4c6886a93f8a9f2ec8728c1dc418e23778aca35a969bd WatchSource:0}: Error finding container dc72baf50eb70ade42b4c6886a93f8a9f2ec8728c1dc418e23778aca35a969bd: Status 404 returned error can't find the container with id dc72baf50eb70ade42b4c6886a93f8a9f2ec8728c1dc418e23778aca35a969bd Oct 31 07:30:31.522156 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:31.522113 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:31.717127 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:31.717091 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:31.913998 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:31.913958 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:32.179126 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:32.179048 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:32.314323 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:32.314287 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5549647ff-cxcdt" event={"ID":"28be803a-a932-4009-ad61-b356366821ef","Type":"ContainerStarted","Data":"ac15faff315cc8b369d2249866b183d0b8126e172bc9a1e3e451eb8650e03c5e"} Oct 31 07:30:32.314535 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:32.314341 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5549647ff-cxcdt" event={"ID":"28be803a-a932-4009-ad61-b356366821ef","Type":"ContainerStarted","Data":"dc72baf50eb70ade42b4c6886a93f8a9f2ec8728c1dc418e23778aca35a969bd"} Oct 31 07:30:32.316996 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:32.316937 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mvmr5" event={"ID":"f13bc62a-1894-472c-95de-262afa399b38","Type":"ContainerStarted","Data":"e0a20ab3c9016389387f396f435edcee552cb97947755d9cd5cd3c53f7bac4fd"} Oct 31 07:30:32.318312 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:32.318289 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-86n8d" event={"ID":"d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e","Type":"ContainerStarted","Data":"96deff506be334042ab1d09e0c13cdafd19bf239c85b57827ced3e54ff5dfb1a"} Oct 31 07:30:32.521448 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:32.521413 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:32.962965 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:32.962756 4935 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-jztcz" Oct 31 07:30:32.963178 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:32.963162 4935 scope.go:122] "RemoveContainer" containerID="392988b2f5bd601185b02a171468b8153e144d07a9e60e83d903b2ed879be684" Oct 31 07:30:32.963413 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:32.963389 4935 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-jztcz_openshift-console-operator(396e1c5e-3700-4fe6-b367-54ad2a244225)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-jztcz" podUID="396e1c5e-3700-4fe6-b367-54ad2a244225" Oct 31 07:30:33.033589 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:33.033562 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:33.103059 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:33.103023 4935 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-5549647ff-cxcdt" Oct 31 07:30:33.105489 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:33.105467 4935 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-5549647ff-cxcdt" Oct 31 07:30:33.232540 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:33.232463 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:33.323509 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:33.323476 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-86n8d" event={"ID":"d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e","Type":"ContainerStarted","Data":"4f667ffa9d7a262cc0407d508ff7502fd2e8ca3aee9080d7132c77b301d082b6"} Oct 31 07:30:33.323642 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:33.323528 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-86n8d" event={"ID":"d7ee81e9-a0f3-4c8c-9152-2b4e0feebb4e","Type":"ContainerStarted","Data":"9d31d24c8a5417d71afc05af115b97ea21a92eae0787986d0442bfed352efe72"} Oct 31 07:30:33.436818 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:33.436788 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:33.636689 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:33.636593 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:33.839835 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:33.839785 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:33.864090 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:33.864072 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-jztcz" Oct 31 07:30:33.864460 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:33.864447 4935 scope.go:122] "RemoveContainer" containerID="392988b2f5bd601185b02a171468b8153e144d07a9e60e83d903b2ed879be684" Oct 31 07:30:33.864620 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:33.864602 4935 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-jztcz_openshift-console-operator(396e1c5e-3700-4fe6-b367-54ad2a244225)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-jztcz" podUID="396e1c5e-3700-4fe6-b367-54ad2a244225" Oct 31 07:30:34.326336 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:34.326306 4935 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 31 07:30:34.877079 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:34.877045 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-5549647ff-cxcdt" Oct 31 07:30:34.878362 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:34.878327 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-5549647ff-cxcdt" Oct 31 07:30:34.901590 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:34.901564 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5e743234-2b5f-4c91-9b06-189f546cbce4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-282kr\" (UID: \"5e743234-2b5f-4c91-9b06-189f546cbce4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-282kr" Oct 31 07:30:34.901717 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:34.901702 4935 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 31 07:30:34.901769 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:34.901763 4935 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e743234-2b5f-4c91-9b06-189f546cbce4-cluster-monitoring-operator-tls podName:5e743234-2b5f-4c91-9b06-189f546cbce4 nodeName:}" failed. No retries permitted until 2026-10-31 07:30:42.901745107 +0000 UTC m=+16.772276623 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5e743234-2b5f-4c91-9b06-189f546cbce4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-282kr" (UID: "5e743234-2b5f-4c91-9b06-189f546cbce4") : secret "cluster-monitoring-operator-tls" not found Oct 31 07:30:35.002220 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:35.002180 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/563d3b8b-0040-439e-b0b2-e858eed5abf6-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-s75pm\" (UID: \"563d3b8b-0040-439e-b0b2-e858eed5abf6\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-s75pm" Oct 31 07:30:35.002411 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:35.002231 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-registry-tls\") pod \"image-registry-54854c7897-pc7km\" (UID: \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\") " pod="openshift-image-registry/image-registry-54854c7897-pc7km" Oct 31 07:30:35.002411 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:35.002267 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/58bc3ea1-f58c-4891-a049-1289a99be055-metrics-tls\") pod \"dns-default-z5sfp\" (UID: \"58bc3ea1-f58c-4891-a049-1289a99be055\") " pod="openshift-dns/dns-default-z5sfp" Oct 31 07:30:35.002411 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:35.002288 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4104ba06-fba8-4c3c-b876-cca5087a7d0a-registry-tls\") pod \"image-registry-69fbc98d44-fn7fh\" (UID: \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\") " pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" Oct 31 07:30:35.002411 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:35.002311 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28ec9621-7ec5-421c-a88b-51bb20d3a7bf-metrics-certs\") pod \"network-metrics-daemon-7thxr\" (UID: \"28ec9621-7ec5-421c-a88b-51bb20d3a7bf\") " pod="openshift-multus/network-metrics-daemon-7thxr" Oct 31 07:30:35.002548 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:35.002453 4935 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 31 07:30:35.002548 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:30:35.002541 4935 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58bc3ea1-f58c-4891-a049-1289a99be055-metrics-tls podName:58bc3ea1-f58c-4891-a049-1289a99be055 nodeName:}" failed. No retries permitted until 2026-10-31 07:30:43.002521711 +0000 UTC m=+16.873053228 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/58bc3ea1-f58c-4891-a049-1289a99be055-metrics-tls") pod "dns-default-z5sfp" (UID: "58bc3ea1-f58c-4891-a049-1289a99be055") : secret "dns-default-metrics-tls" not found Oct 31 07:30:35.004902 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:35.004878 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28ec9621-7ec5-421c-a88b-51bb20d3a7bf-metrics-certs\") pod \"network-metrics-daemon-7thxr\" (UID: \"28ec9621-7ec5-421c-a88b-51bb20d3a7bf\") " pod="openshift-multus/network-metrics-daemon-7thxr" Oct 31 07:30:35.005125 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:35.005104 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4104ba06-fba8-4c3c-b876-cca5087a7d0a-registry-tls\") pod \"image-registry-69fbc98d44-fn7fh\" (UID: \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\") " pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" Oct 31 07:30:35.005190 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:35.005128 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/563d3b8b-0040-439e-b0b2-e858eed5abf6-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-s75pm\" (UID: \"563d3b8b-0040-439e-b0b2-e858eed5abf6\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-s75pm" Oct 31 07:30:35.005233 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:35.005218 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-registry-tls\") pod \"image-registry-54854c7897-pc7km\" (UID: \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\") " pod="openshift-image-registry/image-registry-54854c7897-pc7km" Oct 31 07:30:35.005963 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:35.005944 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-54854c7897-pc7km" Oct 31 07:30:35.008982 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:35.008967 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7thxr" Oct 31 07:30:35.009124 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:35.009104 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-s75pm" Oct 31 07:30:35.016948 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:35.016927 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" Oct 31 07:30:35.032524 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:35.032451 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:35.232293 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:35.232263 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:35.302086 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:35.302062 4935 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7thxr"] Oct 31 07:30:35.305087 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:35.305059 4935 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-s75pm"] Oct 31 07:30:35.305613 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:35.305592 4935 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28ec9621_7ec5_421c_a88b_51bb20d3a7bf.slice/crio-032fe17acf93c63a9396fc83cda939c6f18ae708dffcc8cce2fc54dd4a1a7c66 WatchSource:0}: Error finding container 032fe17acf93c63a9396fc83cda939c6f18ae708dffcc8cce2fc54dd4a1a7c66: Status 404 returned error can't find the container with id 032fe17acf93c63a9396fc83cda939c6f18ae708dffcc8cce2fc54dd4a1a7c66 Oct 31 07:30:35.307911 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:35.307888 4935 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod563d3b8b_0040_439e_b0b2_e858eed5abf6.slice/crio-2fffd0cbc8150422d15526e6be5bea868bcb14c567bea90c5db5e27543716c55 WatchSource:0}: Error finding container 2fffd0cbc8150422d15526e6be5bea868bcb14c567bea90c5db5e27543716c55: Status 404 returned error can't find the container with id 2fffd0cbc8150422d15526e6be5bea868bcb14c567bea90c5db5e27543716c55 Oct 31 07:30:35.308508 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:35.308487 4935 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-54854c7897-pc7km"] Oct 31 07:30:35.311714 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:35.311698 4935 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc734ebf9_eba8_4e14_b20d_2d78dd65b10d.slice/crio-ebe5dc1b1f412bffbc145e0f2a20280e7baec7e23d0958ee54f98e5422e8fa7f WatchSource:0}: Error finding container ebe5dc1b1f412bffbc145e0f2a20280e7baec7e23d0958ee54f98e5422e8fa7f: Status 404 returned error can't find the container with id ebe5dc1b1f412bffbc145e0f2a20280e7baec7e23d0958ee54f98e5422e8fa7f Oct 31 07:30:35.321321 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:35.321299 4935 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-69fbc98d44-fn7fh"] Oct 31 07:30:35.325340 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:35.325319 4935 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4104ba06_fba8_4c3c_b876_cca5087a7d0a.slice/crio-0fec60a49fc43b941c7392f68e5a4cba1ae8a7faa4e753722866bd4fdfda2e70 WatchSource:0}: Error finding container 0fec60a49fc43b941c7392f68e5a4cba1ae8a7faa4e753722866bd4fdfda2e70: Status 404 returned error can't find the container with id 0fec60a49fc43b941c7392f68e5a4cba1ae8a7faa4e753722866bd4fdfda2e70 Oct 31 07:30:35.335495 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:35.335475 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-s75pm" event={"ID":"563d3b8b-0040-439e-b0b2-e858eed5abf6","Type":"ContainerStarted","Data":"2fffd0cbc8150422d15526e6be5bea868bcb14c567bea90c5db5e27543716c55"} Oct 31 07:30:35.336777 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:35.336759 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-54854c7897-pc7km" event={"ID":"c734ebf9-eba8-4e14-b20d-2d78dd65b10d","Type":"ContainerStarted","Data":"ebe5dc1b1f412bffbc145e0f2a20280e7baec7e23d0958ee54f98e5422e8fa7f"} Oct 31 07:30:35.338736 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:35.338710 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" event={"ID":"4104ba06-fba8-4c3c-b876-cca5087a7d0a","Type":"ContainerStarted","Data":"0fec60a49fc43b941c7392f68e5a4cba1ae8a7faa4e753722866bd4fdfda2e70"} Oct 31 07:30:35.340013 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:35.339988 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7thxr" event={"ID":"28ec9621-7ec5-421c-a88b-51bb20d3a7bf","Type":"ContainerStarted","Data":"032fe17acf93c63a9396fc83cda939c6f18ae708dffcc8cce2fc54dd4a1a7c66"} Oct 31 07:30:35.432802 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:35.432724 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:35.453657 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:35.453633 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-ncnd7" Oct 31 07:30:35.634222 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:35.634187 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:35.836486 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:35.836406 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:36.345031 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:36.344994 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-54854c7897-pc7km" event={"ID":"c734ebf9-eba8-4e14-b20d-2d78dd65b10d","Type":"ContainerStarted","Data":"f0c7da94e3ffa95f7634081a30349dc030d9f53c880c3fa9618e6afd38c3888e"} Oct 31 07:30:36.346871 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:36.346846 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" event={"ID":"4104ba06-fba8-4c3c-b876-cca5087a7d0a","Type":"ContainerStarted","Data":"16464bcba6f6fcfa1442ea3a557977410b19cb22bbacf7940475030834b0f402"} Oct 31 07:30:37.031979 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:37.031943 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:37.151104 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:37.151075 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:37.151413 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:37.151398 4935 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 31 07:30:37.231482 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:37.231453 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:37.352408 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:37.352303 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-s75pm" event={"ID":"563d3b8b-0040-439e-b0b2-e858eed5abf6","Type":"ContainerStarted","Data":"0cc6639609c7f41f5a7b6c395197f52b0538499d8ffe5967d665226e42608acb"} Oct 31 07:30:37.357937 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:37.357906 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7thxr" event={"ID":"28ec9621-7ec5-421c-a88b-51bb20d3a7bf","Type":"ContainerStarted","Data":"708316c9e986a72914da7f24fff0f94f2a3a9b71693d04b138391771b4513a0d"} Oct 31 07:30:37.358041 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:37.357952 4935 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 31 07:30:37.358041 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:37.357971 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7thxr" event={"ID":"28ec9621-7ec5-421c-a88b-51bb20d3a7bf","Type":"ContainerStarted","Data":"f388a55323fad1fd890818c8f020ba534a7d0a169fba5a82ed7123d8bec1708f"} Oct 31 07:30:37.358041 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:37.357952 4935 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 31 07:30:37.430176 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:37.430150 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:37.628369 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:37.628280 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:37.830613 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:37.830582 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:39.237581 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:39.237550 4935 scope.go:122] "RemoveContainer" containerID="392988b2f5bd601185b02a171468b8153e144d07a9e60e83d903b2ed879be684" Oct 31 07:30:39.366974 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:39.366948 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jztcz_396e1c5e-3700-4fe6-b367-54ad2a244225/console-operator/2.log" Oct 31 07:30:39.367126 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:39.367010 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-jztcz" event={"ID":"396e1c5e-3700-4fe6-b367-54ad2a244225","Type":"ContainerStarted","Data":"041d6cf721cc95d262855d2098068fc912a5742aa3028bd737921f78b914a6de"} Oct 31 07:30:39.367307 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:39.367287 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-jztcz" Oct 31 07:30:39.521510 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:39.521439 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-jztcz" Oct 31 07:30:39.835372 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:39.835283 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:40.036512 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:40.036481 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:40.340880 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:40.340853 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:40.541585 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:40.541553 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:40.655988 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:40.655924 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" Oct 31 07:30:40.767538 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:40.767505 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:40.858157 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:40.858127 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-pl5cl" Oct 31 07:30:40.975769 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:40.975744 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:41.174805 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:41.174775 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:42.820009 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:42.819982 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-54854c7897-pc7km" Oct 31 07:30:42.973677 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:42.973638 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5e743234-2b5f-4c91-9b06-189f546cbce4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-282kr\" (UID: \"5e743234-2b5f-4c91-9b06-189f546cbce4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-282kr" Oct 31 07:30:42.976245 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:42.976223 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5e743234-2b5f-4c91-9b06-189f546cbce4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-282kr\" (UID: \"5e743234-2b5f-4c91-9b06-189f546cbce4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-282kr" Oct 31 07:30:43.074930 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:43.074868 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/58bc3ea1-f58c-4891-a049-1289a99be055-metrics-tls\") pod \"dns-default-z5sfp\" (UID: \"58bc3ea1-f58c-4891-a049-1289a99be055\") " pod="openshift-dns/dns-default-z5sfp" Oct 31 07:30:43.077192 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:43.077160 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/58bc3ea1-f58c-4891-a049-1289a99be055-metrics-tls\") pod \"dns-default-z5sfp\" (UID: \"58bc3ea1-f58c-4891-a049-1289a99be055\") " pod="openshift-dns/dns-default-z5sfp" Oct 31 07:30:43.103121 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:43.103097 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-282kr" Oct 31 07:30:43.117114 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:43.117095 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-z5sfp" Oct 31 07:30:43.380979 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:43.380901 4935 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-282kr"] Oct 31 07:30:43.384168 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:43.384140 4935 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e743234_2b5f_4c91_9b06_189f546cbce4.slice/crio-855d5514d0e3df022936797759ae6f03101b9f05396bd6b76be9a015754fd155 WatchSource:0}: Error finding container 855d5514d0e3df022936797759ae6f03101b9f05396bd6b76be9a015754fd155: Status 404 returned error can't find the container with id 855d5514d0e3df022936797759ae6f03101b9f05396bd6b76be9a015754fd155 Oct 31 07:30:43.401209 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:43.401182 4935 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-z5sfp"] Oct 31 07:30:43.403907 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:43.403885 4935 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58bc3ea1_f58c_4891_a049_1289a99be055.slice/crio-05eb07366ff8606beb8215e9719ffd7cf78ae3d669de6fa8985aba9a803a76ad WatchSource:0}: Error finding container 05eb07366ff8606beb8215e9719ffd7cf78ae3d669de6fa8985aba9a803a76ad: Status 404 returned error can't find the container with id 05eb07366ff8606beb8215e9719ffd7cf78ae3d669de6fa8985aba9a803a76ad Oct 31 07:30:44.074079 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:44.074047 4935 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-54854c7897-pc7km"] Oct 31 07:30:44.189301 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:44.189217 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:44.387463 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:44.387372 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-z5sfp" event={"ID":"58bc3ea1-f58c-4891-a049-1289a99be055","Type":"ContainerStarted","Data":"05eb07366ff8606beb8215e9719ffd7cf78ae3d669de6fa8985aba9a803a76ad"} Oct 31 07:30:44.388755 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:44.388727 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-282kr" event={"ID":"5e743234-2b5f-4c91-9b06-189f546cbce4","Type":"ContainerStarted","Data":"855d5514d0e3df022936797759ae6f03101b9f05396bd6b76be9a015754fd155"} Oct 31 07:30:44.393334 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:44.393309 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:44.675851 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:44.675761 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:44.881630 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:44.881593 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:45.401112 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:45.401083 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-z5sfp" event={"ID":"58bc3ea1-f58c-4891-a049-1289a99be055","Type":"ContainerStarted","Data":"7e4a968c8a94723958f76e4a507a91a559a99729dafe3343a9e459a66954e68e"} Oct 31 07:30:45.403081 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:45.403011 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-282kr" event={"ID":"5e743234-2b5f-4c91-9b06-189f546cbce4","Type":"ContainerStarted","Data":"8634f5a010a85a9a0c3529d5380542f59a83688be6699bb6de9292b4ee6214a6"} Oct 31 07:30:45.875175 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:45.875135 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:46.071833 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:46.071806 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:46.279333 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:46.279292 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:46.409656 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:46.409623 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-z5sfp" event={"ID":"58bc3ea1-f58c-4891-a049-1289a99be055","Type":"ContainerStarted","Data":"6da9464cfd789fcc5b13898cd035bc6512cc50944c8ac2d1118bf003d5cd8a69"} Oct 31 07:30:47.413881 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:47.413850 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-z5sfp" Oct 31 07:30:48.053306 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:48.053266 4935 patch_prober.go:29] interesting pod/image-registry-69fbc98d44-fn7fh container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Oct 31 07:30:48.053466 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:48.053335 4935 prober.go:122] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" podUID="4104ba06-fba8-4c3c-b876-cca5087a7d0a" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 31 07:30:50.138146 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:50.138112 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:50.335911 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:50.335880 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:50.660211 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:50.660182 4935 patch_prober.go:29] interesting pod/image-registry-69fbc98d44-fn7fh container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Oct 31 07:30:50.660386 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:50.660233 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" podUID="4104ba06-fba8-4c3c-b876-cca5087a7d0a" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 31 07:30:52.260552 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:52.260522 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:53.772016 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:53.771987 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:53.970864 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:53.970838 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:54.079749 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:54.079678 4935 patch_prober.go:29] interesting pod/image-registry-54854c7897-pc7km container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Oct 31 07:30:54.079749 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:54.079723 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-54854c7897-pc7km" podUID="c734ebf9-eba8-4e14-b20d-2d78dd65b10d" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 31 07:30:54.310695 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:54.310650 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:54.434527 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:54.434447 4935 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-8c6xb"] Oct 31 07:30:54.469626 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:54.469599 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8c6xb" Oct 31 07:30:54.517326 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:54.517291 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Oct 31 07:30:54.517326 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:54.517314 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Oct 31 07:30:54.517525 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:54.517416 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-prf6j\"" Oct 31 07:30:54.519756 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:54.519730 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:54.537398 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:54.537379 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Oct 31 07:30:54.537487 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:54.537416 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Oct 31 07:30:54.575737 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:54.575712 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0c09aa6f-d0d8-4276-b1f2-f61446aadd99-node-exporter-tls\") pod \"node-exporter-8c6xb\" (UID: \"0c09aa6f-d0d8-4276-b1f2-f61446aadd99\") " pod="openshift-monitoring/node-exporter-8c6xb" Oct 31 07:30:54.575838 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:54.575750 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0c09aa6f-d0d8-4276-b1f2-f61446aadd99-sys\") pod \"node-exporter-8c6xb\" (UID: \"0c09aa6f-d0d8-4276-b1f2-f61446aadd99\") " pod="openshift-monitoring/node-exporter-8c6xb" Oct 31 07:30:54.575898 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:54.575837 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0c09aa6f-d0d8-4276-b1f2-f61446aadd99-root\") pod \"node-exporter-8c6xb\" (UID: \"0c09aa6f-d0d8-4276-b1f2-f61446aadd99\") " pod="openshift-monitoring/node-exporter-8c6xb" Oct 31 07:30:54.575898 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:54.575885 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0c09aa6f-d0d8-4276-b1f2-f61446aadd99-node-exporter-wtmp\") pod \"node-exporter-8c6xb\" (UID: \"0c09aa6f-d0d8-4276-b1f2-f61446aadd99\") " pod="openshift-monitoring/node-exporter-8c6xb" Oct 31 07:30:54.576000 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:54.575924 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0c09aa6f-d0d8-4276-b1f2-f61446aadd99-metrics-client-ca\") pod \"node-exporter-8c6xb\" (UID: \"0c09aa6f-d0d8-4276-b1f2-f61446aadd99\") " pod="openshift-monitoring/node-exporter-8c6xb" Oct 31 07:30:54.576000 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:54.575973 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0c09aa6f-d0d8-4276-b1f2-f61446aadd99-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8c6xb\" (UID: \"0c09aa6f-d0d8-4276-b1f2-f61446aadd99\") " pod="openshift-monitoring/node-exporter-8c6xb" Oct 31 07:30:54.576000 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:54.575996 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0c09aa6f-d0d8-4276-b1f2-f61446aadd99-node-exporter-accelerators-collector-config\") pod \"node-exporter-8c6xb\" (UID: \"0c09aa6f-d0d8-4276-b1f2-f61446aadd99\") " pod="openshift-monitoring/node-exporter-8c6xb" Oct 31 07:30:54.576095 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:54.576040 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76r5c\" (UniqueName: \"kubernetes.io/projected/0c09aa6f-d0d8-4276-b1f2-f61446aadd99-kube-api-access-76r5c\") pod \"node-exporter-8c6xb\" (UID: \"0c09aa6f-d0d8-4276-b1f2-f61446aadd99\") " pod="openshift-monitoring/node-exporter-8c6xb" Oct 31 07:30:54.576095 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:54.576068 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0c09aa6f-d0d8-4276-b1f2-f61446aadd99-node-exporter-textfile\") pod \"node-exporter-8c6xb\" (UID: \"0c09aa6f-d0d8-4276-b1f2-f61446aadd99\") " pod="openshift-monitoring/node-exporter-8c6xb" Oct 31 07:30:54.676558 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:54.676534 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0c09aa6f-d0d8-4276-b1f2-f61446aadd99-metrics-client-ca\") pod \"node-exporter-8c6xb\" (UID: \"0c09aa6f-d0d8-4276-b1f2-f61446aadd99\") " pod="openshift-monitoring/node-exporter-8c6xb" Oct 31 07:30:54.676672 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:54.676565 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0c09aa6f-d0d8-4276-b1f2-f61446aadd99-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8c6xb\" (UID: \"0c09aa6f-d0d8-4276-b1f2-f61446aadd99\") " pod="openshift-monitoring/node-exporter-8c6xb" Oct 31 07:30:54.676672 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:54.676586 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0c09aa6f-d0d8-4276-b1f2-f61446aadd99-node-exporter-accelerators-collector-config\") pod \"node-exporter-8c6xb\" (UID: \"0c09aa6f-d0d8-4276-b1f2-f61446aadd99\") " pod="openshift-monitoring/node-exporter-8c6xb" Oct 31 07:30:54.676672 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:54.676603 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76r5c\" (UniqueName: \"kubernetes.io/projected/0c09aa6f-d0d8-4276-b1f2-f61446aadd99-kube-api-access-76r5c\") pod \"node-exporter-8c6xb\" (UID: \"0c09aa6f-d0d8-4276-b1f2-f61446aadd99\") " pod="openshift-monitoring/node-exporter-8c6xb" Oct 31 07:30:54.676790 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:54.676736 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0c09aa6f-d0d8-4276-b1f2-f61446aadd99-node-exporter-textfile\") pod \"node-exporter-8c6xb\" (UID: \"0c09aa6f-d0d8-4276-b1f2-f61446aadd99\") " pod="openshift-monitoring/node-exporter-8c6xb" Oct 31 07:30:54.676874 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:54.676815 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0c09aa6f-d0d8-4276-b1f2-f61446aadd99-node-exporter-tls\") pod \"node-exporter-8c6xb\" (UID: \"0c09aa6f-d0d8-4276-b1f2-f61446aadd99\") " pod="openshift-monitoring/node-exporter-8c6xb" Oct 31 07:30:54.676874 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:54.676849 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0c09aa6f-d0d8-4276-b1f2-f61446aadd99-sys\") pod \"node-exporter-8c6xb\" (UID: \"0c09aa6f-d0d8-4276-b1f2-f61446aadd99\") " pod="openshift-monitoring/node-exporter-8c6xb" Oct 31 07:30:54.676974 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:54.676880 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0c09aa6f-d0d8-4276-b1f2-f61446aadd99-root\") pod \"node-exporter-8c6xb\" (UID: \"0c09aa6f-d0d8-4276-b1f2-f61446aadd99\") " pod="openshift-monitoring/node-exporter-8c6xb" Oct 31 07:30:54.676974 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:54.676902 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0c09aa6f-d0d8-4276-b1f2-f61446aadd99-node-exporter-wtmp\") pod \"node-exporter-8c6xb\" (UID: \"0c09aa6f-d0d8-4276-b1f2-f61446aadd99\") " pod="openshift-monitoring/node-exporter-8c6xb" Oct 31 07:30:54.677075 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:54.676996 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0c09aa6f-d0d8-4276-b1f2-f61446aadd99-sys\") pod \"node-exporter-8c6xb\" (UID: \"0c09aa6f-d0d8-4276-b1f2-f61446aadd99\") " pod="openshift-monitoring/node-exporter-8c6xb" Oct 31 07:30:54.677075 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:54.677054 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0c09aa6f-d0d8-4276-b1f2-f61446aadd99-root\") pod \"node-exporter-8c6xb\" (UID: \"0c09aa6f-d0d8-4276-b1f2-f61446aadd99\") " pod="openshift-monitoring/node-exporter-8c6xb" Oct 31 07:30:54.677167 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:54.677077 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0c09aa6f-d0d8-4276-b1f2-f61446aadd99-node-exporter-wtmp\") pod \"node-exporter-8c6xb\" (UID: \"0c09aa6f-d0d8-4276-b1f2-f61446aadd99\") " pod="openshift-monitoring/node-exporter-8c6xb" Oct 31 07:30:54.677231 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:54.677204 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0c09aa6f-d0d8-4276-b1f2-f61446aadd99-node-exporter-textfile\") pod \"node-exporter-8c6xb\" (UID: \"0c09aa6f-d0d8-4276-b1f2-f61446aadd99\") " pod="openshift-monitoring/node-exporter-8c6xb" Oct 31 07:30:54.677286 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:54.677214 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0c09aa6f-d0d8-4276-b1f2-f61446aadd99-metrics-client-ca\") pod \"node-exporter-8c6xb\" (UID: \"0c09aa6f-d0d8-4276-b1f2-f61446aadd99\") " pod="openshift-monitoring/node-exporter-8c6xb" Oct 31 07:30:54.677337 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:54.677320 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0c09aa6f-d0d8-4276-b1f2-f61446aadd99-node-exporter-accelerators-collector-config\") pod \"node-exporter-8c6xb\" (UID: \"0c09aa6f-d0d8-4276-b1f2-f61446aadd99\") " pod="openshift-monitoring/node-exporter-8c6xb" Oct 31 07:30:54.679186 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:54.679167 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0c09aa6f-d0d8-4276-b1f2-f61446aadd99-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8c6xb\" (UID: \"0c09aa6f-d0d8-4276-b1f2-f61446aadd99\") " pod="openshift-monitoring/node-exporter-8c6xb" Oct 31 07:30:54.679362 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:54.679335 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0c09aa6f-d0d8-4276-b1f2-f61446aadd99-node-exporter-tls\") pod \"node-exporter-8c6xb\" (UID: \"0c09aa6f-d0d8-4276-b1f2-f61446aadd99\") " pod="openshift-monitoring/node-exporter-8c6xb" Oct 31 07:30:54.691134 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:54.691094 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-76r5c\" (UniqueName: \"kubernetes.io/projected/0c09aa6f-d0d8-4276-b1f2-f61446aadd99-kube-api-access-76r5c\") pod \"node-exporter-8c6xb\" (UID: \"0c09aa6f-d0d8-4276-b1f2-f61446aadd99\") " pod="openshift-monitoring/node-exporter-8c6xb" Oct 31 07:30:54.717900 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:54.717878 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:54.816258 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:54.816225 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8c6xb" Oct 31 07:30:54.840932 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:30:54.840900 4935 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c09aa6f_d0d8_4276_b1f2_f61446aadd99.slice/crio-c52993ed0cb19ca7c770a6bee4b260bc72b798df5cd5997bfb6881ecd193cf6d WatchSource:0}: Error finding container c52993ed0cb19ca7c770a6bee4b260bc72b798df5cd5997bfb6881ecd193cf6d: Status 404 returned error can't find the container with id c52993ed0cb19ca7c770a6bee4b260bc72b798df5cd5997bfb6881ecd193cf6d Oct 31 07:30:54.924226 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:54.924202 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:55.137958 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:55.137930 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:55.349211 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:55.349165 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:55.442767 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:55.442741 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8c6xb" event={"ID":"0c09aa6f-d0d8-4276-b1f2-f61446aadd99","Type":"ContainerStarted","Data":"c52993ed0cb19ca7c770a6bee4b260bc72b798df5cd5997bfb6881ecd193cf6d"} Oct 31 07:30:55.457131 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:55.457106 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-ncnd7" Oct 31 07:30:55.561366 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:55.561314 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:55.818046 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:55.817962 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:56.483251 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:56.483221 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:56.693445 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:56.693422 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:57.001361 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:57.001286 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-98whn" Oct 31 07:30:57.419083 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:57.419012 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-z5sfp" Oct 31 07:30:57.453905 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:57.453870 4935 generic.go:356] "Generic (PLEG): container finished" podID="0c09aa6f-d0d8-4276-b1f2-f61446aadd99" containerID="d49f1ea59e5129ae3199833186a81d07d6446e03175924e47b6e99c20042e731" exitCode=0 Oct 31 07:30:57.454063 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:57.453953 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8c6xb" event={"ID":"0c09aa6f-d0d8-4276-b1f2-f61446aadd99","Type":"ContainerDied","Data":"d49f1ea59e5129ae3199833186a81d07d6446e03175924e47b6e99c20042e731"} Oct 31 07:30:57.479643 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:57.479591 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:57.678880 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:57.678810 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:57.879488 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:57.879453 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:58.076092 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:58.076068 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:58.320468 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:58.320441 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:58.459563 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:58.459534 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8c6xb" event={"ID":"0c09aa6f-d0d8-4276-b1f2-f61446aadd99","Type":"ContainerStarted","Data":"5d0cf1c20b113976e5af55b0d64c910d5ffbd66f10042e1fa71cff4fac05720e"} Oct 31 07:30:58.459748 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:58.459588 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8c6xb" event={"ID":"0c09aa6f-d0d8-4276-b1f2-f61446aadd99","Type":"ContainerStarted","Data":"0461b6e8f5e14014f931e1e3ac4e9437f89f524205035b6ea39ec8a12fd72fe7"} Oct 31 07:30:58.483848 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:58.483794 4935 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-8c6xb" podStartSLOduration=2.851986199 podStartE2EDuration="4.483775794s" totalImagesPullingTime="1.538728094s" totalInitContainerRuntime="93.061501ms" isStatefulPod=true podCreationTimestamp="2026-10-31 07:30:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:30:58.481395087 +0000 UTC m=+32.351926623" watchObservedRunningTime="2026-10-31 07:30:58.483775794 +0000 UTC m=+32.354307334" Oct 31 07:30:58.520299 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:58.520271 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:59.336372 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:59.336324 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:59.551665 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:59.551636 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:30:59.751304 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:30:59.751267 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:31:00.660960 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:00.660931 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" Oct 31 07:31:00.868361 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:00.868313 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:31:01.083734 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:01.083699 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:31:01.369441 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:01.369367 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:31:01.456267 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:31:01.456234 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 07:31:02.977300 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:02.977268 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:31:03.175778 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:03.175739 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:31:04.078236 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:04.078212 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-54854c7897-pc7km" Oct 31 07:31:04.266042 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:04.266011 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:31:04.458982 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:04.458955 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:31:04.978079 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:04.978050 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:31:05.177794 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:05.177765 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:31:05.704959 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:05.704927 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:31:05.900298 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:05.900262 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:31:06.595665 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:06.595634 4935 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-69fbc98d44-fn7fh"] Oct 31 07:31:06.799741 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:06.799708 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:31:07.033988 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:07.033955 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:31:07.241260 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:07.241217 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:31:08.978045 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:08.978014 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:31:09.100184 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:09.100147 4935 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-54854c7897-pc7km" podUID="c734ebf9-eba8-4e14-b20d-2d78dd65b10d" containerName="registry" containerID="cri-o://f0c7da94e3ffa95f7634081a30349dc030d9f53c880c3fa9618e6afd38c3888e" gracePeriod=30 Oct 31 07:31:09.174945 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:09.174917 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:31:09.345245 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:09.345223 4935 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-54854c7897-pc7km" Oct 31 07:31:09.395750 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:09.395723 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-bound-sa-token\") pod \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\" (UID: \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\") " Oct 31 07:31:09.395914 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:09.395755 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-ca-trust-extracted\") pod \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\" (UID: \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\") " Oct 31 07:31:09.395914 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:09.395806 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-registry-certificates\") pod \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\" (UID: \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\") " Oct 31 07:31:09.395914 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:09.395850 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-image-registry-private-configuration\") pod \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\" (UID: \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\") " Oct 31 07:31:09.395914 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:09.395873 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-trusted-ca\") pod \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\" (UID: \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\") " Oct 31 07:31:09.395914 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:09.395899 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-kube-api-access-z5s89\" (UniqueName: \"kubernetes.io/projected/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-kube-api-access-z5s89\") pod \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\" (UID: \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\") " Oct 31 07:31:09.396186 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:09.395955 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-registry-tls\" (UniqueName: \"kubernetes.io/projected/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-registry-tls\") pod \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\" (UID: \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\") " Oct 31 07:31:09.396186 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:09.396009 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-installation-pull-secrets\") pod \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\" (UID: \"c734ebf9-eba8-4e14-b20d-2d78dd65b10d\") " Oct 31 07:31:09.396697 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:09.396607 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-registry-certificates" pod "c734ebf9-eba8-4e14-b20d-2d78dd65b10d" (UID: "c734ebf9-eba8-4e14-b20d-2d78dd65b10d"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 31 07:31:09.397145 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:09.397116 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-trusted-ca" pod "c734ebf9-eba8-4e14-b20d-2d78dd65b10d" (UID: "c734ebf9-eba8-4e14-b20d-2d78dd65b10d"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 31 07:31:09.401004 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:09.400964 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-kube-api-access-z5s89" pod "c734ebf9-eba8-4e14-b20d-2d78dd65b10d" (UID: "c734ebf9-eba8-4e14-b20d-2d78dd65b10d"). InnerVolumeSpecName "kube-api-access-z5s89". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 31 07:31:09.401116 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:09.401025 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-registry-tls" pod "c734ebf9-eba8-4e14-b20d-2d78dd65b10d" (UID: "c734ebf9-eba8-4e14-b20d-2d78dd65b10d"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 31 07:31:09.402594 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:09.402558 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-installation-pull-secrets" pod "c734ebf9-eba8-4e14-b20d-2d78dd65b10d" (UID: "c734ebf9-eba8-4e14-b20d-2d78dd65b10d"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 31 07:31:09.402906 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:09.402717 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-image-registry-private-configuration" pod "c734ebf9-eba8-4e14-b20d-2d78dd65b10d" (UID: "c734ebf9-eba8-4e14-b20d-2d78dd65b10d"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 31 07:31:09.404655 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:09.404627 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-bound-sa-token" pod "c734ebf9-eba8-4e14-b20d-2d78dd65b10d" (UID: "c734ebf9-eba8-4e14-b20d-2d78dd65b10d"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 31 07:31:09.406251 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:09.406225 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-ca-trust-extracted" pod "c734ebf9-eba8-4e14-b20d-2d78dd65b10d" (UID: "c734ebf9-eba8-4e14-b20d-2d78dd65b10d"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 31 07:31:09.497818 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:09.497785 4935 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-registry-certificates\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 07:31:09.497962 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:09.497813 4935 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-image-registry-private-configuration\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 07:31:09.497962 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:09.497847 4935 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-trusted-ca\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 07:31:09.497962 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:09.497856 4935 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5s89\" (UniqueName: \"kubernetes.io/projected/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-kube-api-access-z5s89\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 07:31:09.497962 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:09.497866 4935 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-registry-tls\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 07:31:09.497962 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:09.497875 4935 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-installation-pull-secrets\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 07:31:09.497962 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:09.497883 4935 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-bound-sa-token\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 07:31:09.497962 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:09.497892 4935 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c734ebf9-eba8-4e14-b20d-2d78dd65b10d-ca-trust-extracted\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 07:31:09.508215 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:09.508179 4935 generic.go:356] "Generic (PLEG): container finished" podID="c734ebf9-eba8-4e14-b20d-2d78dd65b10d" containerID="f0c7da94e3ffa95f7634081a30349dc030d9f53c880c3fa9618e6afd38c3888e" exitCode=0 Oct 31 07:31:09.508334 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:09.508245 4935 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-54854c7897-pc7km" Oct 31 07:31:09.508334 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:09.508264 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-54854c7897-pc7km" event={"ID":"c734ebf9-eba8-4e14-b20d-2d78dd65b10d","Type":"ContainerDied","Data":"f0c7da94e3ffa95f7634081a30349dc030d9f53c880c3fa9618e6afd38c3888e"} Oct 31 07:31:09.508418 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:09.508358 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-54854c7897-pc7km" event={"ID":"c734ebf9-eba8-4e14-b20d-2d78dd65b10d","Type":"ContainerDied","Data":"ebe5dc1b1f412bffbc145e0f2a20280e7baec7e23d0958ee54f98e5422e8fa7f"} Oct 31 07:31:09.508418 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:09.508381 4935 scope.go:122] "RemoveContainer" containerID="f0c7da94e3ffa95f7634081a30349dc030d9f53c880c3fa9618e6afd38c3888e" Oct 31 07:31:09.516848 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:09.516833 4935 scope.go:122] "RemoveContainer" containerID="f0c7da94e3ffa95f7634081a30349dc030d9f53c880c3fa9618e6afd38c3888e" Oct 31 07:31:09.517125 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:31:09.517102 4935 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0c7da94e3ffa95f7634081a30349dc030d9f53c880c3fa9618e6afd38c3888e\": container with ID starting with f0c7da94e3ffa95f7634081a30349dc030d9f53c880c3fa9618e6afd38c3888e not found: ID does not exist" containerID="f0c7da94e3ffa95f7634081a30349dc030d9f53c880c3fa9618e6afd38c3888e" Oct 31 07:31:09.517171 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:09.517137 4935 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0c7da94e3ffa95f7634081a30349dc030d9f53c880c3fa9618e6afd38c3888e"} err="failed to get container status \"f0c7da94e3ffa95f7634081a30349dc030d9f53c880c3fa9618e6afd38c3888e\": rpc error: code = NotFound desc = could not find container \"f0c7da94e3ffa95f7634081a30349dc030d9f53c880c3fa9618e6afd38c3888e\": container with ID starting with f0c7da94e3ffa95f7634081a30349dc030d9f53c880c3fa9618e6afd38c3888e not found: ID does not exist" Oct 31 07:31:09.530162 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:09.530140 4935 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-54854c7897-pc7km"] Oct 31 07:31:09.533787 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:09.533742 4935 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-54854c7897-pc7km"] Oct 31 07:31:10.242104 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:10.242075 4935 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c734ebf9-eba8-4e14-b20d-2d78dd65b10d" path="/var/lib/kubelet/pods/c734ebf9-eba8-4e14-b20d-2d78dd65b10d/volumes" Oct 31 07:31:16.321672 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:16.321639 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:31:16.547119 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:16.547086 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:31:19.783903 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:19.783866 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:31:19.980212 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:19.980178 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:31:24.204187 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:24.204152 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:31:24.402541 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:24.402510 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:31:29.680808 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:29.680770 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:31:29.880378 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:29.880332 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:31:31.454666 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:31:31.454634 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 07:31:31.615410 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:31.615375 4935 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" podUID="4104ba06-fba8-4c3c-b876-cca5087a7d0a" containerName="registry" containerID="cri-o://16464bcba6f6fcfa1442ea3a557977410b19cb22bbacf7940475030834b0f402" gracePeriod=30 Oct 31 07:31:31.858409 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:31.858379 4935 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" Oct 31 07:31:31.964704 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:31.964674 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4104ba06-fba8-4c3c-b876-cca5087a7d0a-registry-tls\" (UniqueName: \"kubernetes.io/projected/4104ba06-fba8-4c3c-b876-cca5087a7d0a-registry-tls\") pod \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\" (UID: \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\") " Oct 31 07:31:31.964857 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:31.964712 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4104ba06-fba8-4c3c-b876-cca5087a7d0a-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4104ba06-fba8-4c3c-b876-cca5087a7d0a-image-registry-private-configuration\") pod \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\" (UID: \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\") " Oct 31 07:31:31.964857 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:31.964747 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4104ba06-fba8-4c3c-b876-cca5087a7d0a-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4104ba06-fba8-4c3c-b876-cca5087a7d0a-registry-certificates\") pod \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\" (UID: \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\") " Oct 31 07:31:31.964857 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:31.964768 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4104ba06-fba8-4c3c-b876-cca5087a7d0a-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4104ba06-fba8-4c3c-b876-cca5087a7d0a-ca-trust-extracted\") pod \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\" (UID: \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\") " Oct 31 07:31:31.964857 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:31.964796 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4104ba06-fba8-4c3c-b876-cca5087a7d0a-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4104ba06-fba8-4c3c-b876-cca5087a7d0a-installation-pull-secrets\") pod \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\" (UID: \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\") " Oct 31 07:31:31.964857 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:31.964836 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4104ba06-fba8-4c3c-b876-cca5087a7d0a-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4104ba06-fba8-4c3c-b876-cca5087a7d0a-bound-sa-token\") pod \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\" (UID: \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\") " Oct 31 07:31:31.964857 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:31.964860 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4104ba06-fba8-4c3c-b876-cca5087a7d0a-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4104ba06-fba8-4c3c-b876-cca5087a7d0a-trusted-ca\") pod \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\" (UID: \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\") " Oct 31 07:31:31.965162 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:31.964886 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4104ba06-fba8-4c3c-b876-cca5087a7d0a-kube-api-access-hldt6\" (UniqueName: \"kubernetes.io/projected/4104ba06-fba8-4c3c-b876-cca5087a7d0a-kube-api-access-hldt6\") pod \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\" (UID: \"4104ba06-fba8-4c3c-b876-cca5087a7d0a\") " Oct 31 07:31:31.965415 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:31.965337 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4104ba06-fba8-4c3c-b876-cca5087a7d0a-registry-certificates" pod "4104ba06-fba8-4c3c-b876-cca5087a7d0a" (UID: "4104ba06-fba8-4c3c-b876-cca5087a7d0a"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 31 07:31:31.965629 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:31.965594 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4104ba06-fba8-4c3c-b876-cca5087a7d0a-trusted-ca" pod "4104ba06-fba8-4c3c-b876-cca5087a7d0a" (UID: "4104ba06-fba8-4c3c-b876-cca5087a7d0a"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 31 07:31:31.967561 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:31.967534 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4104ba06-fba8-4c3c-b876-cca5087a7d0a-registry-tls" pod "4104ba06-fba8-4c3c-b876-cca5087a7d0a" (UID: "4104ba06-fba8-4c3c-b876-cca5087a7d0a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 31 07:31:31.967686 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:31.967660 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4104ba06-fba8-4c3c-b876-cca5087a7d0a-image-registry-private-configuration" pod "4104ba06-fba8-4c3c-b876-cca5087a7d0a" (UID: "4104ba06-fba8-4c3c-b876-cca5087a7d0a"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 31 07:31:31.967898 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:31.967872 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4104ba06-fba8-4c3c-b876-cca5087a7d0a-bound-sa-token" pod "4104ba06-fba8-4c3c-b876-cca5087a7d0a" (UID: "4104ba06-fba8-4c3c-b876-cca5087a7d0a"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 31 07:31:31.968018 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:31.968001 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4104ba06-fba8-4c3c-b876-cca5087a7d0a-kube-api-access-hldt6" pod "4104ba06-fba8-4c3c-b876-cca5087a7d0a" (UID: "4104ba06-fba8-4c3c-b876-cca5087a7d0a"). InnerVolumeSpecName "kube-api-access-hldt6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 31 07:31:31.968078 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:31.968014 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4104ba06-fba8-4c3c-b876-cca5087a7d0a-installation-pull-secrets" pod "4104ba06-fba8-4c3c-b876-cca5087a7d0a" (UID: "4104ba06-fba8-4c3c-b876-cca5087a7d0a"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 31 07:31:31.972066 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:31.971947 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4104ba06-fba8-4c3c-b876-cca5087a7d0a-ca-trust-extracted" pod "4104ba06-fba8-4c3c-b876-cca5087a7d0a" (UID: "4104ba06-fba8-4c3c-b876-cca5087a7d0a"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 31 07:31:32.065452 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:32.065379 4935 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4104ba06-fba8-4c3c-b876-cca5087a7d0a-registry-tls\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 07:31:32.065452 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:32.065410 4935 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4104ba06-fba8-4c3c-b876-cca5087a7d0a-image-registry-private-configuration\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 07:31:32.065452 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:32.065426 4935 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4104ba06-fba8-4c3c-b876-cca5087a7d0a-registry-certificates\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 07:31:32.065452 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:32.065441 4935 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4104ba06-fba8-4c3c-b876-cca5087a7d0a-ca-trust-extracted\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 07:31:32.065667 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:32.065454 4935 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4104ba06-fba8-4c3c-b876-cca5087a7d0a-installation-pull-secrets\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 07:31:32.065667 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:32.065468 4935 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4104ba06-fba8-4c3c-b876-cca5087a7d0a-bound-sa-token\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 07:31:32.065667 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:32.065480 4935 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4104ba06-fba8-4c3c-b876-cca5087a7d0a-trusted-ca\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 07:31:32.065667 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:32.065490 4935 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hldt6\" (UniqueName: \"kubernetes.io/projected/4104ba06-fba8-4c3c-b876-cca5087a7d0a-kube-api-access-hldt6\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 07:31:32.595926 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:32.595896 4935 generic.go:356] "Generic (PLEG): container finished" podID="4104ba06-fba8-4c3c-b876-cca5087a7d0a" containerID="16464bcba6f6fcfa1442ea3a557977410b19cb22bbacf7940475030834b0f402" exitCode=0 Oct 31 07:31:32.596429 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:32.595944 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" event={"ID":"4104ba06-fba8-4c3c-b876-cca5087a7d0a","Type":"ContainerDied","Data":"16464bcba6f6fcfa1442ea3a557977410b19cb22bbacf7940475030834b0f402"} Oct 31 07:31:32.596429 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:32.595961 4935 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" Oct 31 07:31:32.596429 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:32.595979 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69fbc98d44-fn7fh" event={"ID":"4104ba06-fba8-4c3c-b876-cca5087a7d0a","Type":"ContainerDied","Data":"0fec60a49fc43b941c7392f68e5a4cba1ae8a7faa4e753722866bd4fdfda2e70"} Oct 31 07:31:32.596429 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:32.596001 4935 scope.go:122] "RemoveContainer" containerID="16464bcba6f6fcfa1442ea3a557977410b19cb22bbacf7940475030834b0f402" Oct 31 07:31:32.604329 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:32.604311 4935 scope.go:122] "RemoveContainer" containerID="16464bcba6f6fcfa1442ea3a557977410b19cb22bbacf7940475030834b0f402" Oct 31 07:31:32.604614 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:31:32.604595 4935 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16464bcba6f6fcfa1442ea3a557977410b19cb22bbacf7940475030834b0f402\": container with ID starting with 16464bcba6f6fcfa1442ea3a557977410b19cb22bbacf7940475030834b0f402 not found: ID does not exist" containerID="16464bcba6f6fcfa1442ea3a557977410b19cb22bbacf7940475030834b0f402" Oct 31 07:31:32.604760 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:32.604620 4935 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16464bcba6f6fcfa1442ea3a557977410b19cb22bbacf7940475030834b0f402"} err="failed to get container status \"16464bcba6f6fcfa1442ea3a557977410b19cb22bbacf7940475030834b0f402\": rpc error: code = NotFound desc = could not find container \"16464bcba6f6fcfa1442ea3a557977410b19cb22bbacf7940475030834b0f402\": container with ID starting with 16464bcba6f6fcfa1442ea3a557977410b19cb22bbacf7940475030834b0f402 not found: ID does not exist" Oct 31 07:31:32.613928 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:32.613900 4935 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-69fbc98d44-fn7fh"] Oct 31 07:31:32.619717 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:32.619696 4935 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-69fbc98d44-fn7fh"] Oct 31 07:31:33.695805 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:33.695773 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:31:33.896705 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:33.896675 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:31:34.242474 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:34.242445 4935 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4104ba06-fba8-4c3c-b876-cca5087a7d0a" path="/var/lib/kubelet/pods/4104ba06-fba8-4c3c-b876-cca5087a7d0a/volumes" Oct 31 07:31:38.067599 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:38.067567 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:31:38.265451 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:38.265413 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:31:38.468499 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:38.468467 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:31:40.657499 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:40.657468 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:31:40.858442 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:40.858412 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:31:42.635735 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:42.635658 4935 generic.go:356] "Generic (PLEG): container finished" podID="b6997c76-e103-4c3f-b39d-a6a17fb98270" containerID="802d75b7d7289ffc5378d73cc340e8c3619c827734d0500160e9b60b2167c7ab" exitCode=0 Oct 31 07:31:42.636118 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:42.635734 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-62tn2" event={"ID":"b6997c76-e103-4c3f-b39d-a6a17fb98270","Type":"ContainerDied","Data":"802d75b7d7289ffc5378d73cc340e8c3619c827734d0500160e9b60b2167c7ab"} Oct 31 07:31:42.636118 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:42.636093 4935 scope.go:122] "RemoveContainer" containerID="802d75b7d7289ffc5378d73cc340e8c3619c827734d0500160e9b60b2167c7ab" Oct 31 07:31:42.637252 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:42.637231 4935 generic.go:356] "Generic (PLEG): container finished" podID="f3bf19fd-bc58-4dff-a1a8-4f61381e1fde" containerID="a4d14e51cf34ef6255e491bd4714d520f3505fa33dc1a5af8dda8e7ebfd24534" exitCode=0 Oct 31 07:31:42.637372 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:42.637286 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-46qvv" event={"ID":"f3bf19fd-bc58-4dff-a1a8-4f61381e1fde","Type":"ContainerDied","Data":"a4d14e51cf34ef6255e491bd4714d520f3505fa33dc1a5af8dda8e7ebfd24534"} Oct 31 07:31:42.637635 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:42.637616 4935 scope.go:122] "RemoveContainer" containerID="a4d14e51cf34ef6255e491bd4714d520f3505fa33dc1a5af8dda8e7ebfd24534" Oct 31 07:31:43.446924 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:43.446893 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:31:43.643004 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:43.642970 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-62tn2" event={"ID":"b6997c76-e103-4c3f-b39d-a6a17fb98270","Type":"ContainerStarted","Data":"3dba7d191d4b4a93ea18c15961592d073a34c9d70e257aa9c6391b82f821ffbd"} Oct 31 07:31:43.644640 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:43.644616 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-46qvv" event={"ID":"f3bf19fd-bc58-4dff-a1a8-4f61381e1fde","Type":"ContainerStarted","Data":"ec06c3ab6a30236887ed39792ca3ce1273f89683fe95091cab7dc2edb17db964"} Oct 31 07:31:43.687223 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:43.687193 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:31:43.891780 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:43.891704 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:31:44.679404 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:44.679343 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:31:44.875913 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:44.875883 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:31:47.661322 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:47.661287 4935 generic.go:356] "Generic (PLEG): container finished" podID="b4dd8884-1d6e-4abe-ad6c-3c947623bb60" containerID="c163670e087dcc1d3bf484637029cd5dc9bdeacff71daa0efb650167bf38ffd5" exitCode=0 Oct 31 07:31:47.661719 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:47.661372 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-hz58r" event={"ID":"b4dd8884-1d6e-4abe-ad6c-3c947623bb60","Type":"ContainerDied","Data":"c163670e087dcc1d3bf484637029cd5dc9bdeacff71daa0efb650167bf38ffd5"} Oct 31 07:31:47.661719 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:47.661694 4935 scope.go:122] "RemoveContainer" containerID="c163670e087dcc1d3bf484637029cd5dc9bdeacff71daa0efb650167bf38ffd5" Oct 31 07:31:48.084738 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:48.084705 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:31:48.172108 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:48.172076 4935 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-rmx96-5ppn9"] Oct 31 07:31:48.172379 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:48.172367 4935 cpu_manager.go:414] "containerMap: removing container" podUID="c734ebf9-eba8-4e14-b20d-2d78dd65b10d" containerName="registry" Oct 31 07:31:48.172435 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:48.172381 4935 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c734ebf9-eba8-4e14-b20d-2d78dd65b10d" containerName="registry" Oct 31 07:31:48.172435 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:48.172401 4935 cpu_manager.go:414] "containerMap: removing container" podUID="4104ba06-fba8-4c3c-b876-cca5087a7d0a" containerName="registry" Oct 31 07:31:48.172435 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:48.172406 4935 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4104ba06-fba8-4c3c-b876-cca5087a7d0a" containerName="registry" Oct 31 07:31:48.172537 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:48.172464 4935 memory_manager.go:365] "RemoveStaleState removing state" podUID="c734ebf9-eba8-4e14-b20d-2d78dd65b10d" containerName="registry" Oct 31 07:31:48.172537 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:48.172474 4935 memory_manager.go:365] "RemoveStaleState removing state" podUID="4104ba06-fba8-4c3c-b876-cca5087a7d0a" containerName="registry" Oct 31 07:31:48.175280 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:48.175266 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-rmx96-5ppn9" Oct 31 07:31:48.178694 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:48.178674 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-gvr8l\"" Oct 31 07:31:48.187280 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:48.187262 4935 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-rmx96-5ppn9"] Oct 31 07:31:48.283953 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:48.283927 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:31:48.286072 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:48.286051 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff188433-6826-4666-9e78-f61f337069d4-service-ca-bundle\") pod \"periodic-gathering-rmx96-5ppn9\" (UID: \"ff188433-6826-4666-9e78-f61f337069d4\") " pod="openshift-insights/periodic-gathering-rmx96-5ppn9" Oct 31 07:31:48.286191 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:48.286079 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxz9g\" (UniqueName: \"kubernetes.io/projected/ff188433-6826-4666-9e78-f61f337069d4-kube-api-access-jxz9g\") pod \"periodic-gathering-rmx96-5ppn9\" (UID: \"ff188433-6826-4666-9e78-f61f337069d4\") " pod="openshift-insights/periodic-gathering-rmx96-5ppn9" Oct 31 07:31:48.286191 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:48.286101 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff188433-6826-4666-9e78-f61f337069d4-trusted-ca-bundle\") pod \"periodic-gathering-rmx96-5ppn9\" (UID: \"ff188433-6826-4666-9e78-f61f337069d4\") " pod="openshift-insights/periodic-gathering-rmx96-5ppn9" Oct 31 07:31:48.286191 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:48.286124 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/ff188433-6826-4666-9e78-f61f337069d4-archives-path\") pod \"periodic-gathering-rmx96-5ppn9\" (UID: \"ff188433-6826-4666-9e78-f61f337069d4\") " pod="openshift-insights/periodic-gathering-rmx96-5ppn9" Oct 31 07:31:48.387088 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:48.387026 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/ff188433-6826-4666-9e78-f61f337069d4-archives-path\") pod \"periodic-gathering-rmx96-5ppn9\" (UID: \"ff188433-6826-4666-9e78-f61f337069d4\") " pod="openshift-insights/periodic-gathering-rmx96-5ppn9" Oct 31 07:31:48.387088 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:48.387081 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff188433-6826-4666-9e78-f61f337069d4-service-ca-bundle\") pod \"periodic-gathering-rmx96-5ppn9\" (UID: \"ff188433-6826-4666-9e78-f61f337069d4\") " pod="openshift-insights/periodic-gathering-rmx96-5ppn9" Oct 31 07:31:48.387257 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:48.387101 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxz9g\" (UniqueName: \"kubernetes.io/projected/ff188433-6826-4666-9e78-f61f337069d4-kube-api-access-jxz9g\") pod \"periodic-gathering-rmx96-5ppn9\" (UID: \"ff188433-6826-4666-9e78-f61f337069d4\") " pod="openshift-insights/periodic-gathering-rmx96-5ppn9" Oct 31 07:31:48.387257 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:48.387123 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff188433-6826-4666-9e78-f61f337069d4-trusted-ca-bundle\") pod \"periodic-gathering-rmx96-5ppn9\" (UID: \"ff188433-6826-4666-9e78-f61f337069d4\") " pod="openshift-insights/periodic-gathering-rmx96-5ppn9" Oct 31 07:31:48.387534 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:48.387512 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/ff188433-6826-4666-9e78-f61f337069d4-archives-path\") pod \"periodic-gathering-rmx96-5ppn9\" (UID: \"ff188433-6826-4666-9e78-f61f337069d4\") " pod="openshift-insights/periodic-gathering-rmx96-5ppn9" Oct 31 07:31:48.387745 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:48.387729 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff188433-6826-4666-9e78-f61f337069d4-service-ca-bundle\") pod \"periodic-gathering-rmx96-5ppn9\" (UID: \"ff188433-6826-4666-9e78-f61f337069d4\") " pod="openshift-insights/periodic-gathering-rmx96-5ppn9" Oct 31 07:31:48.387869 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:48.387851 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff188433-6826-4666-9e78-f61f337069d4-trusted-ca-bundle\") pod \"periodic-gathering-rmx96-5ppn9\" (UID: \"ff188433-6826-4666-9e78-f61f337069d4\") " pod="openshift-insights/periodic-gathering-rmx96-5ppn9" Oct 31 07:31:48.403242 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:48.403216 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxz9g\" (UniqueName: \"kubernetes.io/projected/ff188433-6826-4666-9e78-f61f337069d4-kube-api-access-jxz9g\") pod \"periodic-gathering-rmx96-5ppn9\" (UID: \"ff188433-6826-4666-9e78-f61f337069d4\") " pod="openshift-insights/periodic-gathering-rmx96-5ppn9" Oct 31 07:31:48.486665 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:48.486621 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-rmx96-5ppn9" Oct 31 07:31:48.666639 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:48.666570 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-hz58r" event={"ID":"b4dd8884-1d6e-4abe-ad6c-3c947623bb60","Type":"ContainerStarted","Data":"3f0a45eb78c3f7f07de9f2088fa57b2af85d99b0c353646e56ee6ed222b4de19"} Oct 31 07:31:48.760901 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:48.760872 4935 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-rmx96-5ppn9"] Oct 31 07:31:48.764197 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:31:48.764158 4935 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff188433_6826_4666_9e78_f61f337069d4.slice/crio-6be238f3f39a1656b30a2c599b825bc55d5a7711c6ce2d9057a37e0668fa700c WatchSource:0}: Error finding container 6be238f3f39a1656b30a2c599b825bc55d5a7711c6ce2d9057a37e0668fa700c: Status 404 returned error can't find the container with id 6be238f3f39a1656b30a2c599b825bc55d5a7711c6ce2d9057a37e0668fa700c Oct 31 07:31:49.672173 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:49.672138 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-rmx96-5ppn9" event={"ID":"ff188433-6826-4666-9e78-f61f337069d4","Type":"ContainerStarted","Data":"d3603c89bde6ec75474c33ba8ccbbdc9b775581855adfc88bd3e478ae2d63ee1"} Oct 31 07:31:49.672581 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:49.672186 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-rmx96-5ppn9" event={"ID":"ff188433-6826-4666-9e78-f61f337069d4","Type":"ContainerStarted","Data":"6be238f3f39a1656b30a2c599b825bc55d5a7711c6ce2d9057a37e0668fa700c"} Oct 31 07:31:49.690004 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:49.689944 4935 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-rmx96-5ppn9" podStartSLOduration=1.689932665 podStartE2EDuration="1.689932665s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:31:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:31:49.689165089 +0000 UTC m=+83.559696624" watchObservedRunningTime="2026-10-31 07:31:49.689932665 +0000 UTC m=+83.560464199" Oct 31 07:31:59.687421 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:59.687388 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:31:59.891838 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:31:59.891804 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:32:01.455742 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:32:01.455706 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 07:32:16.378249 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:32:16.378213 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:32:31.455022 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:32:31.454985 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 07:33:01.453936 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:33:01.453899 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 07:33:12.993433 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:33:12.993399 4935 generic.go:356] "Generic (PLEG): container finished" podID="ff188433-6826-4666-9e78-f61f337069d4" containerID="d3603c89bde6ec75474c33ba8ccbbdc9b775581855adfc88bd3e478ae2d63ee1" exitCode=0 Oct 31 07:33:12.993433 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:33:12.993434 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-rmx96-5ppn9" event={"ID":"ff188433-6826-4666-9e78-f61f337069d4","Type":"ContainerDied","Data":"d3603c89bde6ec75474c33ba8ccbbdc9b775581855adfc88bd3e478ae2d63ee1"} Oct 31 07:33:14.118624 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:33:14.118602 4935 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-rmx96-5ppn9" Oct 31 07:33:14.231437 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:33:14.231411 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ff188433-6826-4666-9e78-f61f337069d4-kube-api-access-jxz9g\" (UniqueName: \"kubernetes.io/projected/ff188433-6826-4666-9e78-f61f337069d4-kube-api-access-jxz9g\") pod \"ff188433-6826-4666-9e78-f61f337069d4\" (UID: \"ff188433-6826-4666-9e78-f61f337069d4\") " Oct 31 07:33:14.231602 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:33:14.231452 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ff188433-6826-4666-9e78-f61f337069d4-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff188433-6826-4666-9e78-f61f337069d4-service-ca-bundle\") pod \"ff188433-6826-4666-9e78-f61f337069d4\" (UID: \"ff188433-6826-4666-9e78-f61f337069d4\") " Oct 31 07:33:14.231602 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:33:14.231498 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ff188433-6826-4666-9e78-f61f337069d4-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/ff188433-6826-4666-9e78-f61f337069d4-archives-path\") pod \"ff188433-6826-4666-9e78-f61f337069d4\" (UID: \"ff188433-6826-4666-9e78-f61f337069d4\") " Oct 31 07:33:14.231602 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:33:14.231539 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ff188433-6826-4666-9e78-f61f337069d4-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff188433-6826-4666-9e78-f61f337069d4-trusted-ca-bundle\") pod \"ff188433-6826-4666-9e78-f61f337069d4\" (UID: \"ff188433-6826-4666-9e78-f61f337069d4\") " Oct 31 07:33:14.231870 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:33:14.231849 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff188433-6826-4666-9e78-f61f337069d4-service-ca-bundle" pod "ff188433-6826-4666-9e78-f61f337069d4" (UID: "ff188433-6826-4666-9e78-f61f337069d4"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 31 07:33:14.231870 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:33:14.231852 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff188433-6826-4666-9e78-f61f337069d4-archives-path" pod "ff188433-6826-4666-9e78-f61f337069d4" (UID: "ff188433-6826-4666-9e78-f61f337069d4"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 31 07:33:14.231963 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:33:14.231937 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff188433-6826-4666-9e78-f61f337069d4-trusted-ca-bundle" pod "ff188433-6826-4666-9e78-f61f337069d4" (UID: "ff188433-6826-4666-9e78-f61f337069d4"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 31 07:33:14.233861 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:33:14.233845 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff188433-6826-4666-9e78-f61f337069d4-kube-api-access-jxz9g" pod "ff188433-6826-4666-9e78-f61f337069d4" (UID: "ff188433-6826-4666-9e78-f61f337069d4"). InnerVolumeSpecName "kube-api-access-jxz9g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 31 07:33:14.332369 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:33:14.332304 4935 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff188433-6826-4666-9e78-f61f337069d4-trusted-ca-bundle\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 07:33:14.332369 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:33:14.332325 4935 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jxz9g\" (UniqueName: \"kubernetes.io/projected/ff188433-6826-4666-9e78-f61f337069d4-kube-api-access-jxz9g\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 07:33:14.332369 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:33:14.332337 4935 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff188433-6826-4666-9e78-f61f337069d4-service-ca-bundle\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 07:33:14.332369 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:33:14.332365 4935 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/ff188433-6826-4666-9e78-f61f337069d4-archives-path\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 07:33:15.002362 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:33:15.002314 4935 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-rmx96-5ppn9" Oct 31 07:33:15.002362 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:33:15.002323 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-rmx96-5ppn9" event={"ID":"ff188433-6826-4666-9e78-f61f337069d4","Type":"ContainerDied","Data":"6be238f3f39a1656b30a2c599b825bc55d5a7711c6ce2d9057a37e0668fa700c"} Oct 31 07:33:15.002584 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:33:15.002397 4935 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6be238f3f39a1656b30a2c599b825bc55d5a7711c6ce2d9057a37e0668fa700c" Oct 31 07:33:15.423813 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:33:15.423730 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:33:15.627232 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:33:15.627202 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:33:16.429759 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:33:16.429715 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:33:31.454447 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:33:31.454412 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 07:33:48.316730 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:33:48.316690 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:33:48.517224 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:33:48.517187 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:34:01.454069 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:34:01.454027 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 07:34:16.486640 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:16.486565 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:34:18.718449 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:18.718418 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:34:18.921867 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:18.921835 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:34:19.546527 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:19.546492 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:34:19.746416 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:19.746382 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:34:21.701661 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:21.701601 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:34:21.896950 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:21.896919 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:34:31.454208 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:34:31.454172 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 07:34:34.108612 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:34.108577 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:34:42.431408 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:42.431359 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:34:42.646966 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:42.646915 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:34:48.160765 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:48.160735 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:34:48.442566 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:48.442479 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:34:48.515583 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:48.515549 4935 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-86bd856cfb-f2m7n"] Oct 31 07:34:48.515859 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:48.515847 4935 cpu_manager.go:414] "containerMap: removing container" podUID="ff188433-6826-4666-9e78-f61f337069d4" containerName="insights-gathering" Oct 31 07:34:48.515906 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:48.515860 4935 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ff188433-6826-4666-9e78-f61f337069d4" containerName="insights-gathering" Oct 31 07:34:48.515939 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:48.515924 4935 memory_manager.go:365] "RemoveStaleState removing state" podUID="ff188433-6826-4666-9e78-f61f337069d4" containerName="insights-gathering" Oct 31 07:34:48.518629 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:48.518614 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-f2m7n" Oct 31 07:34:48.523573 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:48.523551 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\"" Oct 31 07:34:48.523704 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:48.523618 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\"" Oct 31 07:34:48.523759 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:48.523711 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-sm468\"" Oct 31 07:34:48.523759 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:48.523722 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\"" Oct 31 07:34:48.523759 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:48.523716 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\"" Oct 31 07:34:48.523865 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:48.523762 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\"" Oct 31 07:34:48.530850 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:48.530829 4935 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-86bd856cfb-f2m7n"] Oct 31 07:34:48.665672 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:48.665643 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:34:48.694018 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:48.693998 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/600354d6-c24d-4aca-9701-24a3d072d7e8-certificates\") pod \"keda-metrics-apiserver-86bd856cfb-f2m7n\" (UID: \"600354d6-c24d-4aca-9701-24a3d072d7e8\") " pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-f2m7n" Oct 31 07:34:48.694121 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:48.694050 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzch5\" (UniqueName: \"kubernetes.io/projected/600354d6-c24d-4aca-9701-24a3d072d7e8-kube-api-access-qzch5\") pod \"keda-metrics-apiserver-86bd856cfb-f2m7n\" (UID: \"600354d6-c24d-4aca-9701-24a3d072d7e8\") " pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-f2m7n" Oct 31 07:34:48.694121 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:48.694115 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/600354d6-c24d-4aca-9701-24a3d072d7e8-temp-vol\") pod \"keda-metrics-apiserver-86bd856cfb-f2m7n\" (UID: \"600354d6-c24d-4aca-9701-24a3d072d7e8\") " pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-f2m7n" Oct 31 07:34:48.795363 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:48.795330 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/600354d6-c24d-4aca-9701-24a3d072d7e8-temp-vol\") pod \"keda-metrics-apiserver-86bd856cfb-f2m7n\" (UID: \"600354d6-c24d-4aca-9701-24a3d072d7e8\") " pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-f2m7n" Oct 31 07:34:48.795485 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:48.795421 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/600354d6-c24d-4aca-9701-24a3d072d7e8-certificates\") pod \"keda-metrics-apiserver-86bd856cfb-f2m7n\" (UID: \"600354d6-c24d-4aca-9701-24a3d072d7e8\") " pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-f2m7n" Oct 31 07:34:48.795547 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:48.795495 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzch5\" (UniqueName: \"kubernetes.io/projected/600354d6-c24d-4aca-9701-24a3d072d7e8-kube-api-access-qzch5\") pod \"keda-metrics-apiserver-86bd856cfb-f2m7n\" (UID: \"600354d6-c24d-4aca-9701-24a3d072d7e8\") " pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-f2m7n" Oct 31 07:34:48.795600 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:34:48.795579 4935 secret.go:281] references non-existent secret key: tls.crt Oct 31 07:34:48.795652 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:34:48.795612 4935 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: tls.crt Oct 31 07:34:48.795652 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:34:48.795638 4935 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-86bd856cfb-f2m7n: references non-existent secret key: tls.crt Oct 31 07:34:48.795744 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:48.795666 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/600354d6-c24d-4aca-9701-24a3d072d7e8-temp-vol\") pod \"keda-metrics-apiserver-86bd856cfb-f2m7n\" (UID: \"600354d6-c24d-4aca-9701-24a3d072d7e8\") " pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-f2m7n" Oct 31 07:34:48.795744 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:34:48.795706 4935 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/600354d6-c24d-4aca-9701-24a3d072d7e8-certificates podName:600354d6-c24d-4aca-9701-24a3d072d7e8 nodeName:}" failed. No retries permitted until 2026-10-31 07:34:49.29568647 +0000 UTC m=+263.166218000 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/600354d6-c24d-4aca-9701-24a3d072d7e8-certificates") pod "keda-metrics-apiserver-86bd856cfb-f2m7n" (UID: "600354d6-c24d-4aca-9701-24a3d072d7e8") : references non-existent secret key: tls.crt Oct 31 07:34:48.807975 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:48.807951 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzch5\" (UniqueName: \"kubernetes.io/projected/600354d6-c24d-4aca-9701-24a3d072d7e8-kube-api-access-qzch5\") pod \"keda-metrics-apiserver-86bd856cfb-f2m7n\" (UID: \"600354d6-c24d-4aca-9701-24a3d072d7e8\") " pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-f2m7n" Oct 31 07:34:48.867324 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:48.867300 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:34:49.038292 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:49.038224 4935 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-7b7c985c58-cclv5"] Oct 31 07:34:49.041400 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:49.041384 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-7b7c985c58-cclv5" Oct 31 07:34:49.044899 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:49.044866 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\"" Oct 31 07:34:49.052559 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:49.052539 4935 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-7b7c985c58-cclv5"] Oct 31 07:34:49.126874 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:49.126848 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:34:49.199337 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:49.199292 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/2bdf5c10-fa7e-4c76-a5b9-450a87345366-certificates\") pod \"keda-admission-7b7c985c58-cclv5\" (UID: \"2bdf5c10-fa7e-4c76-a5b9-450a87345366\") " pod="openshift-keda/keda-admission-7b7c985c58-cclv5" Oct 31 07:34:49.199711 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:49.199359 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6bng\" (UniqueName: \"kubernetes.io/projected/2bdf5c10-fa7e-4c76-a5b9-450a87345366-kube-api-access-s6bng\") pod \"keda-admission-7b7c985c58-cclv5\" (UID: \"2bdf5c10-fa7e-4c76-a5b9-450a87345366\") " pod="openshift-keda/keda-admission-7b7c985c58-cclv5" Oct 31 07:34:49.299852 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:49.299768 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/600354d6-c24d-4aca-9701-24a3d072d7e8-certificates\") pod \"keda-metrics-apiserver-86bd856cfb-f2m7n\" (UID: \"600354d6-c24d-4aca-9701-24a3d072d7e8\") " pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-f2m7n" Oct 31 07:34:49.299852 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:49.299807 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6bng\" (UniqueName: \"kubernetes.io/projected/2bdf5c10-fa7e-4c76-a5b9-450a87345366-kube-api-access-s6bng\") pod \"keda-admission-7b7c985c58-cclv5\" (UID: \"2bdf5c10-fa7e-4c76-a5b9-450a87345366\") " pod="openshift-keda/keda-admission-7b7c985c58-cclv5" Oct 31 07:34:49.300076 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:49.299864 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/2bdf5c10-fa7e-4c76-a5b9-450a87345366-certificates\") pod \"keda-admission-7b7c985c58-cclv5\" (UID: \"2bdf5c10-fa7e-4c76-a5b9-450a87345366\") " pod="openshift-keda/keda-admission-7b7c985c58-cclv5" Oct 31 07:34:49.302423 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:49.302402 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/2bdf5c10-fa7e-4c76-a5b9-450a87345366-certificates\") pod \"keda-admission-7b7c985c58-cclv5\" (UID: \"2bdf5c10-fa7e-4c76-a5b9-450a87345366\") " pod="openshift-keda/keda-admission-7b7c985c58-cclv5" Oct 31 07:34:49.302532 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:49.302433 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/600354d6-c24d-4aca-9701-24a3d072d7e8-certificates\") pod \"keda-metrics-apiserver-86bd856cfb-f2m7n\" (UID: \"600354d6-c24d-4aca-9701-24a3d072d7e8\") " pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-f2m7n" Oct 31 07:34:49.311719 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:49.311699 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6bng\" (UniqueName: \"kubernetes.io/projected/2bdf5c10-fa7e-4c76-a5b9-450a87345366-kube-api-access-s6bng\") pod \"keda-admission-7b7c985c58-cclv5\" (UID: \"2bdf5c10-fa7e-4c76-a5b9-450a87345366\") " pod="openshift-keda/keda-admission-7b7c985c58-cclv5" Oct 31 07:34:49.346945 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:49.346920 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:34:49.353603 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:49.353586 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-7b7c985c58-cclv5" Oct 31 07:34:49.431852 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:49.431828 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-f2m7n" Oct 31 07:34:49.572555 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:49.572484 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:34:49.626311 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:49.626276 4935 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-7b7c985c58-cclv5"] Oct 31 07:34:49.629049 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:34:49.629029 4935 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bdf5c10_fa7e_4c76_a5b9_450a87345366.slice/crio-35bc458f3593413a371038b567921fc06a9474f47bb85194da334f4cb3184e2d WatchSource:0}: Error finding container 35bc458f3593413a371038b567921fc06a9474f47bb85194da334f4cb3184e2d: Status 404 returned error can't find the container with id 35bc458f3593413a371038b567921fc06a9474f47bb85194da334f4cb3184e2d Oct 31 07:34:49.702734 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:49.702712 4935 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-86bd856cfb-f2m7n"] Oct 31 07:34:49.705986 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:34:49.705963 4935 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod600354d6_c24d_4aca_9701_24a3d072d7e8.slice/crio-4e43e3599b884a792f618e33963ac3cdb172d10d70aff6776bf9830217275b4c WatchSource:0}: Error finding container 4e43e3599b884a792f618e33963ac3cdb172d10d70aff6776bf9830217275b4c: Status 404 returned error can't find the container with id 4e43e3599b884a792f618e33963ac3cdb172d10d70aff6776bf9830217275b4c Oct 31 07:34:49.773586 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:49.773565 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:34:50.354165 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:50.354123 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-f2m7n" event={"ID":"600354d6-c24d-4aca-9701-24a3d072d7e8","Type":"ContainerStarted","Data":"4e43e3599b884a792f618e33963ac3cdb172d10d70aff6776bf9830217275b4c"} Oct 31 07:34:50.355274 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:50.355251 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-7b7c985c58-cclv5" event={"ID":"2bdf5c10-fa7e-4c76-a5b9-450a87345366","Type":"ContainerStarted","Data":"35bc458f3593413a371038b567921fc06a9474f47bb85194da334f4cb3184e2d"} Oct 31 07:34:53.376589 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:53.376554 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-f2m7n" event={"ID":"600354d6-c24d-4aca-9701-24a3d072d7e8","Type":"ContainerStarted","Data":"a8b8cba38c83ce6abee72faeac1f6761316c09f94f8f07763ef117f75e6d0a5b"} Oct 31 07:34:53.377030 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:53.376640 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-f2m7n" Oct 31 07:34:53.378187 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:53.378162 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-7b7c985c58-cclv5" event={"ID":"2bdf5c10-fa7e-4c76-a5b9-450a87345366","Type":"ContainerStarted","Data":"fc4cc0a257d65a546f37c7b0498a55e1af31bf07f221b9b47303f33338e90bd9"} Oct 31 07:34:53.378285 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:53.378221 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-7b7c985c58-cclv5" Oct 31 07:34:53.396456 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:53.396415 4935 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-f2m7n" podStartSLOduration=1.8500690579999999 podStartE2EDuration="5.396378462s" totalImagesPullingTime="3.546309404s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:34:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:34:53.394396645 +0000 UTC m=+267.264928178" watchObservedRunningTime="2026-10-31 07:34:53.396378462 +0000 UTC m=+267.266909996" Oct 31 07:34:53.412303 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:34:53.412257 4935 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-keda/keda-admission-7b7c985c58-cclv5" podStartSLOduration=1.233429529 podStartE2EDuration="4.412244724s" totalImagesPullingTime="3.178815195s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:34:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:34:53.411848363 +0000 UTC m=+267.282379902" watchObservedRunningTime="2026-10-31 07:34:53.412244724 +0000 UTC m=+267.282776261" Oct 31 07:35:01.458464 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:35:01.458421 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 07:35:04.387166 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:35:04.387139 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-f2m7n" Oct 31 07:35:14.384651 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:35:14.384624 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-7b7c985c58-cclv5" Oct 31 07:35:15.953557 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:35:15.953518 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:35:16.152976 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:35:16.152946 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:35:16.542384 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:35:16.542334 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:35:26.253009 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:35:26.252972 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jztcz_396e1c5e-3700-4fe6-b367-54ad2a244225/console-operator/2.log" Oct 31 07:35:26.253577 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:35:26.253158 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jztcz_396e1c5e-3700-4fe6-b367-54ad2a244225/console-operator/2.log" Oct 31 07:35:31.454248 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:35:31.454215 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 07:35:42.772206 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:35:42.772167 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:35:42.974622 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:35:42.974589 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:36:01.454247 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:36:01.454210 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 07:36:15.944537 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:36:15.944508 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:36:15.982514 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:36:15.982487 4935 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kserve/llmisvc-controller-manager-86c86ffb6b-gfp9q"] Oct 31 07:36:15.985540 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:36:15.985527 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-86c86ffb6b-gfp9q" Oct 31 07:36:15.995342 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:36:15.995322 4935 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-86c86ffb6b-gfp9q"] Oct 31 07:36:16.063132 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:36:16.063112 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\"" Oct 31 07:36:16.063258 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:36:16.063173 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\"" Oct 31 07:36:16.063258 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:36:16.063193 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-controller-manager-dockercfg-f2rph\"" Oct 31 07:36:16.063258 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:36:16.063222 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-webhook-server-cert\"" Oct 31 07:36:16.112880 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:36:16.112853 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8l98p\" (UniqueName: \"kubernetes.io/projected/53ccf16f-450f-46cf-8c1e-5bc99e15789a-kube-api-access-8l98p\") pod \"llmisvc-controller-manager-86c86ffb6b-gfp9q\" (UID: \"53ccf16f-450f-46cf-8c1e-5bc99e15789a\") " pod="kserve/llmisvc-controller-manager-86c86ffb6b-gfp9q" Oct 31 07:36:16.112981 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:36:16.112907 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/53ccf16f-450f-46cf-8c1e-5bc99e15789a-cert\") pod \"llmisvc-controller-manager-86c86ffb6b-gfp9q\" (UID: \"53ccf16f-450f-46cf-8c1e-5bc99e15789a\") " pod="kserve/llmisvc-controller-manager-86c86ffb6b-gfp9q" Oct 31 07:36:16.145316 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:36:16.145294 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:36:16.213824 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:36:16.213767 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8l98p\" (UniqueName: \"kubernetes.io/projected/53ccf16f-450f-46cf-8c1e-5bc99e15789a-kube-api-access-8l98p\") pod \"llmisvc-controller-manager-86c86ffb6b-gfp9q\" (UID: \"53ccf16f-450f-46cf-8c1e-5bc99e15789a\") " pod="kserve/llmisvc-controller-manager-86c86ffb6b-gfp9q" Oct 31 07:36:16.213931 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:36:16.213821 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/53ccf16f-450f-46cf-8c1e-5bc99e15789a-cert\") pod \"llmisvc-controller-manager-86c86ffb6b-gfp9q\" (UID: \"53ccf16f-450f-46cf-8c1e-5bc99e15789a\") " pod="kserve/llmisvc-controller-manager-86c86ffb6b-gfp9q" Oct 31 07:36:16.216370 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:36:16.216337 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/53ccf16f-450f-46cf-8c1e-5bc99e15789a-cert\") pod \"llmisvc-controller-manager-86c86ffb6b-gfp9q\" (UID: \"53ccf16f-450f-46cf-8c1e-5bc99e15789a\") " pod="kserve/llmisvc-controller-manager-86c86ffb6b-gfp9q" Oct 31 07:36:16.226192 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:36:16.226170 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8l98p\" (UniqueName: \"kubernetes.io/projected/53ccf16f-450f-46cf-8c1e-5bc99e15789a-kube-api-access-8l98p\") pod \"llmisvc-controller-manager-86c86ffb6b-gfp9q\" (UID: \"53ccf16f-450f-46cf-8c1e-5bc99e15789a\") " pod="kserve/llmisvc-controller-manager-86c86ffb6b-gfp9q" Oct 31 07:36:16.298050 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:36:16.298026 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-86c86ffb6b-gfp9q" Oct 31 07:36:16.350677 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:36:16.350586 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:36:16.568328 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:36:16.568186 4935 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-86c86ffb6b-gfp9q"] Oct 31 07:36:16.570878 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:36:16.570853 4935 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53ccf16f_450f_46cf_8c1e_5bc99e15789a.slice/crio-11e9f634b954ebda0daa8e34bcaa4461868e8a7f6f12d626ecbe2f6245a8de0f WatchSource:0}: Error finding container 11e9f634b954ebda0daa8e34bcaa4461868e8a7f6f12d626ecbe2f6245a8de0f: Status 404 returned error can't find the container with id 11e9f634b954ebda0daa8e34bcaa4461868e8a7f6f12d626ecbe2f6245a8de0f Oct 31 07:36:16.572336 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:36:16.572319 4935 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 31 07:36:16.590875 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:36:16.590851 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:36:16.702568 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:36:16.702542 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-86c86ffb6b-gfp9q" event={"ID":"53ccf16f-450f-46cf-8c1e-5bc99e15789a","Type":"ContainerStarted","Data":"11e9f634b954ebda0daa8e34bcaa4461868e8a7f6f12d626ecbe2f6245a8de0f"} Oct 31 07:36:17.487778 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:36:17.487674 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:36:17.687143 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:36:17.687070 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:36:19.717741 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:36:19.717706 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-86c86ffb6b-gfp9q" event={"ID":"53ccf16f-450f-46cf-8c1e-5bc99e15789a","Type":"ContainerStarted","Data":"f665fc282e90f2e563a8d1374634f767767759c625af969f4b818714fe916e77"} Oct 31 07:36:19.718114 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:36:19.717854 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-86c86ffb6b-gfp9q" Oct 31 07:36:19.736335 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:36:19.736294 4935 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kserve/llmisvc-controller-manager-86c86ffb6b-gfp9q" podStartSLOduration=2.140644323 podStartE2EDuration="4.736283561s" totalImagesPullingTime="2.595639238s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:36:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:36:19.73557072 +0000 UTC m=+353.606102262" watchObservedRunningTime="2026-10-31 07:36:19.736283561 +0000 UTC m=+353.606815092" Oct 31 07:36:31.454341 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:36:31.454309 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 07:36:40.634939 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:36:40.634898 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:36:40.834118 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:36:40.834083 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:36:50.722771 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:36:50.722740 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-86c86ffb6b-gfp9q" Oct 31 07:37:01.454741 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:37:01.454707 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 07:37:16.651499 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:37:16.651423 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:37:31.453693 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:37:31.453657 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 07:37:37.112117 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:37:37.112084 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:37:37.328654 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:37:37.328618 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:37:37.533879 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:37:37.533849 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:37:40.667322 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:37:40.667284 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:37:40.866505 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:37:40.866478 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:37:59.981745 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:37:59.981716 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:38:01.454408 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:38:01.454376 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 07:38:16.703848 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:38:16.703814 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:38:17.007800 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:38:17.007723 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:38:17.206636 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:38:17.206604 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:38:31.454120 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:38:31.454084 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 07:39:01.454913 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:39:01.454876 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 07:39:06.391851 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:39:06.391819 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:39:06.595511 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:39:06.595479 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:39:11.906289 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:39:11.906244 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:39:12.106103 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:39:12.106070 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:39:16.812454 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:39:16.812422 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:39:27.331927 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:39:27.331898 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:39:28.969567 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:39:28.969538 4935 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-03ede73268/error-404-isvc-43360-predictor-5785b8dcc7-zcjgr"] Oct 31 07:39:28.973057 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:39:28.973041 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-03ede73268/error-404-isvc-43360-predictor-5785b8dcc7-zcjgr" Oct 31 07:39:28.981014 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:39:28.980996 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-03ede73268\"/\"default-dockercfg-dwzsp\"" Oct 31 07:39:28.981014 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:39:28.980995 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-03ede73268\"/\"error-404-isvc-43360-predictor-serving-cert\"" Oct 31 07:39:28.981985 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:39:28.981970 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-03ede73268\"/\"openshift-service-ca.crt\"" Oct 31 07:39:28.982055 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:39:28.982023 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-03ede73268\"/\"error-404-isvc-43360-kube-rbac-proxy-sar-config\"" Oct 31 07:39:28.982112 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:39:28.982103 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-03ede73268\"/\"kube-root-ca.crt\"" Oct 31 07:39:28.985626 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:39:28.985600 4935 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-03ede73268/error-404-isvc-43360-predictor-5785b8dcc7-zcjgr"] Oct 31 07:39:29.009996 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:39:29.009968 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:39:29.110786 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:39:29.110756 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5hs7\" (UniqueName: \"kubernetes.io/projected/9643855c-6524-4580-aafb-ccc4ffc2cab4-kube-api-access-m5hs7\") pod \"error-404-isvc-43360-predictor-5785b8dcc7-zcjgr\" (UID: \"9643855c-6524-4580-aafb-ccc4ffc2cab4\") " pod="e2e-gw1-03ede73268/error-404-isvc-43360-predictor-5785b8dcc7-zcjgr" Oct 31 07:39:29.110956 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:39:29.110794 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9643855c-6524-4580-aafb-ccc4ffc2cab4-proxy-tls\") pod \"error-404-isvc-43360-predictor-5785b8dcc7-zcjgr\" (UID: \"9643855c-6524-4580-aafb-ccc4ffc2cab4\") " pod="e2e-gw1-03ede73268/error-404-isvc-43360-predictor-5785b8dcc7-zcjgr" Oct 31 07:39:29.110956 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:39:29.110900 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9643855c-6524-4580-aafb-ccc4ffc2cab4-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-43360-predictor-5785b8dcc7-zcjgr\" (UID: \"9643855c-6524-4580-aafb-ccc4ffc2cab4\") " pod="e2e-gw1-03ede73268/error-404-isvc-43360-predictor-5785b8dcc7-zcjgr" Oct 31 07:39:29.210521 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:39:29.210488 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:39:29.211494 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:39:29.211474 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9643855c-6524-4580-aafb-ccc4ffc2cab4-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-43360-predictor-5785b8dcc7-zcjgr\" (UID: \"9643855c-6524-4580-aafb-ccc4ffc2cab4\") " pod="e2e-gw1-03ede73268/error-404-isvc-43360-predictor-5785b8dcc7-zcjgr" Oct 31 07:39:29.211552 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:39:29.211513 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5hs7\" (UniqueName: \"kubernetes.io/projected/9643855c-6524-4580-aafb-ccc4ffc2cab4-kube-api-access-m5hs7\") pod \"error-404-isvc-43360-predictor-5785b8dcc7-zcjgr\" (UID: \"9643855c-6524-4580-aafb-ccc4ffc2cab4\") " pod="e2e-gw1-03ede73268/error-404-isvc-43360-predictor-5785b8dcc7-zcjgr" Oct 31 07:39:29.211552 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:39:29.211542 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9643855c-6524-4580-aafb-ccc4ffc2cab4-proxy-tls\") pod \"error-404-isvc-43360-predictor-5785b8dcc7-zcjgr\" (UID: \"9643855c-6524-4580-aafb-ccc4ffc2cab4\") " pod="e2e-gw1-03ede73268/error-404-isvc-43360-predictor-5785b8dcc7-zcjgr" Oct 31 07:39:29.212137 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:39:29.212116 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9643855c-6524-4580-aafb-ccc4ffc2cab4-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-43360-predictor-5785b8dcc7-zcjgr\" (UID: \"9643855c-6524-4580-aafb-ccc4ffc2cab4\") " pod="e2e-gw1-03ede73268/error-404-isvc-43360-predictor-5785b8dcc7-zcjgr" Oct 31 07:39:29.214069 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:39:29.214051 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9643855c-6524-4580-aafb-ccc4ffc2cab4-proxy-tls\") pod \"error-404-isvc-43360-predictor-5785b8dcc7-zcjgr\" (UID: \"9643855c-6524-4580-aafb-ccc4ffc2cab4\") " pod="e2e-gw1-03ede73268/error-404-isvc-43360-predictor-5785b8dcc7-zcjgr" Oct 31 07:39:29.223206 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:39:29.223154 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5hs7\" (UniqueName: \"kubernetes.io/projected/9643855c-6524-4580-aafb-ccc4ffc2cab4-kube-api-access-m5hs7\") pod \"error-404-isvc-43360-predictor-5785b8dcc7-zcjgr\" (UID: \"9643855c-6524-4580-aafb-ccc4ffc2cab4\") " pod="e2e-gw1-03ede73268/error-404-isvc-43360-predictor-5785b8dcc7-zcjgr" Oct 31 07:39:29.287618 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:39:29.287584 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-03ede73268/error-404-isvc-43360-predictor-5785b8dcc7-zcjgr" Oct 31 07:39:29.412548 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:39:29.412518 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:39:29.565110 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:39:29.565037 4935 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-03ede73268/error-404-isvc-43360-predictor-5785b8dcc7-zcjgr"] Oct 31 07:39:29.568411 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:39:29.568386 4935 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9643855c_6524_4580_aafb_ccc4ffc2cab4.slice/crio-2730dd66f70d8099550036d5abc6f34c866fabf1a27ccbf70e0104d0a2eceddc WatchSource:0}: Error finding container 2730dd66f70d8099550036d5abc6f34c866fabf1a27ccbf70e0104d0a2eceddc: Status 404 returned error can't find the container with id 2730dd66f70d8099550036d5abc6f34c866fabf1a27ccbf70e0104d0a2eceddc Oct 31 07:39:30.460542 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:39:30.460504 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-03ede73268/error-404-isvc-43360-predictor-5785b8dcc7-zcjgr" event={"ID":"9643855c-6524-4580-aafb-ccc4ffc2cab4","Type":"ContainerStarted","Data":"2730dd66f70d8099550036d5abc6f34c866fabf1a27ccbf70e0104d0a2eceddc"} Oct 31 07:39:31.406606 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:39:31.406496 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:39:31.466719 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:39:31.466666 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 07:39:31.600775 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:39:31.600733 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:39:42.532849 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:39:42.532809 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-03ede73268/error-404-isvc-43360-predictor-5785b8dcc7-zcjgr" event={"ID":"9643855c-6524-4580-aafb-ccc4ffc2cab4","Type":"ContainerStarted","Data":"dca71e32bfafdb91354c75841dded01c2218188cc230cdf1a0f88feda02d5553"} Oct 31 07:39:44.543631 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:39:44.543586 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-03ede73268/error-404-isvc-43360-predictor-5785b8dcc7-zcjgr" event={"ID":"9643855c-6524-4580-aafb-ccc4ffc2cab4","Type":"ContainerStarted","Data":"5df7617f8c17c6d3274ce8e8bfeb18c9b7e1c8e707ba479f77b2e111be472b5c"} Oct 31 07:39:44.544080 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:39:44.543660 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-03ede73268/error-404-isvc-43360-predictor-5785b8dcc7-zcjgr" Oct 31 07:39:44.544080 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:39:44.543710 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-03ede73268/error-404-isvc-43360-predictor-5785b8dcc7-zcjgr" Oct 31 07:39:44.545340 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:39:44.545317 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/error-404-isvc-43360-predictor-5785b8dcc7-zcjgr" podUID="9643855c-6524-4580-aafb-ccc4ffc2cab4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.29:8080: connect: connection refused" Oct 31 07:39:44.563928 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:39:44.563890 4935 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-03ede73268/error-404-isvc-43360-predictor-5785b8dcc7-zcjgr" podStartSLOduration=2.295559181 podStartE2EDuration="16.563858614s" totalImagesPullingTime="14.268299433s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:39:28 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:39:44.562732891 +0000 UTC m=+558.433264428" watchObservedRunningTime="2026-10-31 07:39:44.563858614 +0000 UTC m=+558.434390151" Oct 31 07:39:45.547910 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:39:45.547868 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/error-404-isvc-43360-predictor-5785b8dcc7-zcjgr" podUID="9643855c-6524-4580-aafb-ccc4ffc2cab4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.29:8080: connect: connection refused" Oct 31 07:39:50.552142 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:39:50.552115 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-03ede73268/error-404-isvc-43360-predictor-5785b8dcc7-zcjgr" Oct 31 07:39:50.554577 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:39:50.552720 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/error-404-isvc-43360-predictor-5785b8dcc7-zcjgr" podUID="9643855c-6524-4580-aafb-ccc4ffc2cab4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.29:8080: connect: connection refused" Oct 31 07:40:00.553135 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:40:00.553093 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/error-404-isvc-43360-predictor-5785b8dcc7-zcjgr" podUID="9643855c-6524-4580-aafb-ccc4ffc2cab4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.29:8080: connect: connection refused" Oct 31 07:40:01.454201 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:40:01.454155 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 07:40:10.552841 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:40:10.552756 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/error-404-isvc-43360-predictor-5785b8dcc7-zcjgr" podUID="9643855c-6524-4580-aafb-ccc4ffc2cab4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.29:8080: connect: connection refused" Oct 31 07:40:16.810215 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:40:16.810177 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:40:20.553174 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:40:20.553133 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/error-404-isvc-43360-predictor-5785b8dcc7-zcjgr" podUID="9643855c-6524-4580-aafb-ccc4ffc2cab4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.29:8080: connect: connection refused" Oct 31 07:40:26.276368 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:40:26.276316 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jztcz_396e1c5e-3700-4fe6-b367-54ad2a244225/console-operator/2.log" Oct 31 07:40:26.278086 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:40:26.278061 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jztcz_396e1c5e-3700-4fe6-b367-54ad2a244225/console-operator/2.log" Oct 31 07:40:30.553558 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:40:30.553522 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-03ede73268/error-404-isvc-43360-predictor-5785b8dcc7-zcjgr" Oct 31 07:40:31.455012 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:40:31.454976 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 07:40:58.613972 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:40:58.613937 4935 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-03ede73268/switch-graph-43360-f8cd54dcb-4959p"] Oct 31 07:40:58.617224 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:40:58.617207 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-03ede73268/switch-graph-43360-f8cd54dcb-4959p" Oct 31 07:40:58.628550 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:40:58.628528 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-03ede73268\"/\"switch-graph-43360-serving-cert\"" Oct 31 07:40:58.628750 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:40:58.628709 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-03ede73268\"/\"switch-graph-43360-kube-rbac-proxy-sar-config\"" Oct 31 07:40:58.629944 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:40:58.629922 4935 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-03ede73268/switch-graph-43360-f8cd54dcb-4959p"] Oct 31 07:40:58.701806 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:40:58.701770 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6ada0c88-70ac-48e5-9e91-f6fd589b40cb-proxy-tls\") pod \"switch-graph-43360-f8cd54dcb-4959p\" (UID: \"6ada0c88-70ac-48e5-9e91-f6fd589b40cb\") " pod="e2e-gw1-03ede73268/switch-graph-43360-f8cd54dcb-4959p" Oct 31 07:40:58.701985 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:40:58.701829 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ada0c88-70ac-48e5-9e91-f6fd589b40cb-openshift-service-ca-bundle\") pod \"switch-graph-43360-f8cd54dcb-4959p\" (UID: \"6ada0c88-70ac-48e5-9e91-f6fd589b40cb\") " pod="e2e-gw1-03ede73268/switch-graph-43360-f8cd54dcb-4959p" Oct 31 07:40:58.742615 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:40:58.742573 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:40:58.803094 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:40:58.803065 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6ada0c88-70ac-48e5-9e91-f6fd589b40cb-proxy-tls\") pod \"switch-graph-43360-f8cd54dcb-4959p\" (UID: \"6ada0c88-70ac-48e5-9e91-f6fd589b40cb\") " pod="e2e-gw1-03ede73268/switch-graph-43360-f8cd54dcb-4959p" Oct 31 07:40:58.803223 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:40:58.803113 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ada0c88-70ac-48e5-9e91-f6fd589b40cb-openshift-service-ca-bundle\") pod \"switch-graph-43360-f8cd54dcb-4959p\" (UID: \"6ada0c88-70ac-48e5-9e91-f6fd589b40cb\") " pod="e2e-gw1-03ede73268/switch-graph-43360-f8cd54dcb-4959p" Oct 31 07:40:58.803704 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:40:58.803683 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ada0c88-70ac-48e5-9e91-f6fd589b40cb-openshift-service-ca-bundle\") pod \"switch-graph-43360-f8cd54dcb-4959p\" (UID: \"6ada0c88-70ac-48e5-9e91-f6fd589b40cb\") " pod="e2e-gw1-03ede73268/switch-graph-43360-f8cd54dcb-4959p" Oct 31 07:40:58.805595 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:40:58.805576 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6ada0c88-70ac-48e5-9e91-f6fd589b40cb-proxy-tls\") pod \"switch-graph-43360-f8cd54dcb-4959p\" (UID: \"6ada0c88-70ac-48e5-9e91-f6fd589b40cb\") " pod="e2e-gw1-03ede73268/switch-graph-43360-f8cd54dcb-4959p" Oct 31 07:40:58.929189 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:40:58.929101 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-03ede73268/switch-graph-43360-f8cd54dcb-4959p" Oct 31 07:40:58.949991 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:40:58.949964 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:40:59.199190 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:40:59.199171 4935 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-03ede73268/switch-graph-43360-f8cd54dcb-4959p"] Oct 31 07:40:59.201518 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:40:59.201491 4935 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ada0c88_70ac_48e5_9e91_f6fd589b40cb.slice/crio-91de279a18c859bca6dad8129dd2cf5522bcec4e55606312f8d3d9e7365f2117 WatchSource:0}: Error finding container 91de279a18c859bca6dad8129dd2cf5522bcec4e55606312f8d3d9e7365f2117: Status 404 returned error can't find the container with id 91de279a18c859bca6dad8129dd2cf5522bcec4e55606312f8d3d9e7365f2117 Oct 31 07:40:59.835953 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:40:59.835925 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-03ede73268/switch-graph-43360-f8cd54dcb-4959p" event={"ID":"6ada0c88-70ac-48e5-9e91-f6fd589b40cb","Type":"ContainerStarted","Data":"91de279a18c859bca6dad8129dd2cf5522bcec4e55606312f8d3d9e7365f2117"} Oct 31 07:41:01.455480 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:41:01.455446 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 07:41:03.853260 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:03.853230 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-03ede73268/switch-graph-43360-f8cd54dcb-4959p" event={"ID":"6ada0c88-70ac-48e5-9e91-f6fd589b40cb","Type":"ContainerStarted","Data":"83ab399634f48fa661c5a3d7c2a75ec489fe3201223e5beb7bdb3a881ec08e68"} Oct 31 07:41:03.853633 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:03.853374 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-03ede73268/switch-graph-43360-f8cd54dcb-4959p" Oct 31 07:41:03.871940 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:03.871899 4935 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-03ede73268/switch-graph-43360-f8cd54dcb-4959p" podStartSLOduration=1.375688611 podStartE2EDuration="5.871886703s" totalImagesPullingTime="4.496198092s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:40:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:41:03.871031124 +0000 UTC m=+637.741562662" watchObservedRunningTime="2026-10-31 07:41:03.871886703 +0000 UTC m=+637.742418237" Oct 31 07:41:09.862837 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:09.862806 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-03ede73268/switch-graph-43360-f8cd54dcb-4959p" Oct 31 07:41:14.048154 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:14.048114 4935 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-03ede73268/error-404-isvc-43360-predictor-5785b8dcc7-zcjgr"] Oct 31 07:41:14.048683 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:14.048432 4935 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-03ede73268/error-404-isvc-43360-predictor-5785b8dcc7-zcjgr" podUID="9643855c-6524-4580-aafb-ccc4ffc2cab4" containerName="kserve-container" containerID="cri-o://dca71e32bfafdb91354c75841dded01c2218188cc230cdf1a0f88feda02d5553" gracePeriod=30 Oct 31 07:41:14.048683 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:14.048522 4935 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-03ede73268/error-404-isvc-43360-predictor-5785b8dcc7-zcjgr" podUID="9643855c-6524-4580-aafb-ccc4ffc2cab4" containerName="kube-rbac-proxy" containerID="cri-o://5df7617f8c17c6d3274ce8e8bfeb18c9b7e1c8e707ba479f77b2e111be472b5c" gracePeriod=30 Oct 31 07:41:14.193506 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:14.193469 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:41:14.392298 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:14.392226 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:41:14.897423 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:14.897390 4935 generic.go:356] "Generic (PLEG): container finished" podID="9643855c-6524-4580-aafb-ccc4ffc2cab4" containerID="5df7617f8c17c6d3274ce8e8bfeb18c9b7e1c8e707ba479f77b2e111be472b5c" exitCode=0 Oct 31 07:41:14.897585 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:14.897459 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-03ede73268/error-404-isvc-43360-predictor-5785b8dcc7-zcjgr" event={"ID":"9643855c-6524-4580-aafb-ccc4ffc2cab4","Type":"ContainerDied","Data":"5df7617f8c17c6d3274ce8e8bfeb18c9b7e1c8e707ba479f77b2e111be472b5c"} Oct 31 07:41:15.548585 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:15.548546 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/error-404-isvc-43360-predictor-5785b8dcc7-zcjgr" podUID="9643855c-6524-4580-aafb-ccc4ffc2cab4" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.29:8643/healthz\": dial tcp 10.132.0.29:8643: connect: connection refused" Oct 31 07:41:16.853114 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:16.853068 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:41:17.285109 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:17.285088 4935 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-03ede73268/error-404-isvc-43360-predictor-5785b8dcc7-zcjgr" Oct 31 07:41:17.335952 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:17.335930 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9643855c-6524-4580-aafb-ccc4ffc2cab4-kube-api-access-m5hs7\" (UniqueName: \"kubernetes.io/projected/9643855c-6524-4580-aafb-ccc4ffc2cab4-kube-api-access-m5hs7\") pod \"9643855c-6524-4580-aafb-ccc4ffc2cab4\" (UID: \"9643855c-6524-4580-aafb-ccc4ffc2cab4\") " Oct 31 07:41:17.336086 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:17.335983 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9643855c-6524-4580-aafb-ccc4ffc2cab4-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9643855c-6524-4580-aafb-ccc4ffc2cab4-kube-rbac-proxy-sar-config\") pod \"9643855c-6524-4580-aafb-ccc4ffc2cab4\" (UID: \"9643855c-6524-4580-aafb-ccc4ffc2cab4\") " Oct 31 07:41:17.336086 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:17.336014 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9643855c-6524-4580-aafb-ccc4ffc2cab4-proxy-tls\" (UniqueName: \"kubernetes.io/secret/9643855c-6524-4580-aafb-ccc4ffc2cab4-proxy-tls\") pod \"9643855c-6524-4580-aafb-ccc4ffc2cab4\" (UID: \"9643855c-6524-4580-aafb-ccc4ffc2cab4\") " Oct 31 07:41:17.336337 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:17.336315 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9643855c-6524-4580-aafb-ccc4ffc2cab4-kube-rbac-proxy-sar-config" pod "9643855c-6524-4580-aafb-ccc4ffc2cab4" (UID: "9643855c-6524-4580-aafb-ccc4ffc2cab4"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 31 07:41:17.338264 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:17.338239 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9643855c-6524-4580-aafb-ccc4ffc2cab4-kube-api-access-m5hs7" pod "9643855c-6524-4580-aafb-ccc4ffc2cab4" (UID: "9643855c-6524-4580-aafb-ccc4ffc2cab4"). InnerVolumeSpecName "kube-api-access-m5hs7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 31 07:41:17.338264 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:17.338244 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9643855c-6524-4580-aafb-ccc4ffc2cab4-proxy-tls" pod "9643855c-6524-4580-aafb-ccc4ffc2cab4" (UID: "9643855c-6524-4580-aafb-ccc4ffc2cab4"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 31 07:41:17.436687 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:17.436624 4935 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9643855c-6524-4580-aafb-ccc4ffc2cab4-kube-rbac-proxy-sar-config\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 07:41:17.436687 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:17.436650 4935 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9643855c-6524-4580-aafb-ccc4ffc2cab4-proxy-tls\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 07:41:17.436687 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:17.436660 4935 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m5hs7\" (UniqueName: \"kubernetes.io/projected/9643855c-6524-4580-aafb-ccc4ffc2cab4-kube-api-access-m5hs7\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 07:41:17.911821 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:17.911781 4935 generic.go:356] "Generic (PLEG): container finished" podID="9643855c-6524-4580-aafb-ccc4ffc2cab4" containerID="dca71e32bfafdb91354c75841dded01c2218188cc230cdf1a0f88feda02d5553" exitCode=0 Oct 31 07:41:17.911821 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:17.911820 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-03ede73268/error-404-isvc-43360-predictor-5785b8dcc7-zcjgr" event={"ID":"9643855c-6524-4580-aafb-ccc4ffc2cab4","Type":"ContainerDied","Data":"dca71e32bfafdb91354c75841dded01c2218188cc230cdf1a0f88feda02d5553"} Oct 31 07:41:17.912297 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:17.911836 4935 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-03ede73268/error-404-isvc-43360-predictor-5785b8dcc7-zcjgr" Oct 31 07:41:17.912297 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:17.911867 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-03ede73268/error-404-isvc-43360-predictor-5785b8dcc7-zcjgr" event={"ID":"9643855c-6524-4580-aafb-ccc4ffc2cab4","Type":"ContainerDied","Data":"2730dd66f70d8099550036d5abc6f34c866fabf1a27ccbf70e0104d0a2eceddc"} Oct 31 07:41:17.912297 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:17.911892 4935 scope.go:122] "RemoveContainer" containerID="5df7617f8c17c6d3274ce8e8bfeb18c9b7e1c8e707ba479f77b2e111be472b5c" Oct 31 07:41:17.920728 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:17.920710 4935 scope.go:122] "RemoveContainer" containerID="dca71e32bfafdb91354c75841dded01c2218188cc230cdf1a0f88feda02d5553" Oct 31 07:41:17.928035 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:17.928018 4935 scope.go:122] "RemoveContainer" containerID="5df7617f8c17c6d3274ce8e8bfeb18c9b7e1c8e707ba479f77b2e111be472b5c" Oct 31 07:41:17.928295 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:41:17.928278 4935 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5df7617f8c17c6d3274ce8e8bfeb18c9b7e1c8e707ba479f77b2e111be472b5c\": container with ID starting with 5df7617f8c17c6d3274ce8e8bfeb18c9b7e1c8e707ba479f77b2e111be472b5c not found: ID does not exist" containerID="5df7617f8c17c6d3274ce8e8bfeb18c9b7e1c8e707ba479f77b2e111be472b5c" Oct 31 07:41:17.928378 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:17.928309 4935 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5df7617f8c17c6d3274ce8e8bfeb18c9b7e1c8e707ba479f77b2e111be472b5c"} err="failed to get container status \"5df7617f8c17c6d3274ce8e8bfeb18c9b7e1c8e707ba479f77b2e111be472b5c\": rpc error: code = NotFound desc = could not find container \"5df7617f8c17c6d3274ce8e8bfeb18c9b7e1c8e707ba479f77b2e111be472b5c\": container with ID starting with 5df7617f8c17c6d3274ce8e8bfeb18c9b7e1c8e707ba479f77b2e111be472b5c not found: ID does not exist" Oct 31 07:41:17.928378 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:17.928330 4935 scope.go:122] "RemoveContainer" containerID="dca71e32bfafdb91354c75841dded01c2218188cc230cdf1a0f88feda02d5553" Oct 31 07:41:17.928612 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:41:17.928596 4935 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dca71e32bfafdb91354c75841dded01c2218188cc230cdf1a0f88feda02d5553\": container with ID starting with dca71e32bfafdb91354c75841dded01c2218188cc230cdf1a0f88feda02d5553 not found: ID does not exist" containerID="dca71e32bfafdb91354c75841dded01c2218188cc230cdf1a0f88feda02d5553" Oct 31 07:41:17.928650 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:17.928619 4935 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dca71e32bfafdb91354c75841dded01c2218188cc230cdf1a0f88feda02d5553"} err="failed to get container status \"dca71e32bfafdb91354c75841dded01c2218188cc230cdf1a0f88feda02d5553\": rpc error: code = NotFound desc = could not find container \"dca71e32bfafdb91354c75841dded01c2218188cc230cdf1a0f88feda02d5553\": container with ID starting with dca71e32bfafdb91354c75841dded01c2218188cc230cdf1a0f88feda02d5553 not found: ID does not exist" Oct 31 07:41:17.932442 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:17.932421 4935 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-03ede73268/error-404-isvc-43360-predictor-5785b8dcc7-zcjgr"] Oct 31 07:41:17.936238 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:17.936217 4935 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-03ede73268/error-404-isvc-43360-predictor-5785b8dcc7-zcjgr"] Oct 31 07:41:18.242467 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:18.242438 4935 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9643855c-6524-4580-aafb-ccc4ffc2cab4" path="/var/lib/kubelet/pods/9643855c-6524-4580-aafb-ccc4ffc2cab4/volumes" Oct 31 07:41:19.509105 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:19.509061 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:41:19.611554 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:19.611522 4935 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-03ede73268/error-404-isvc-490af-predictor-778f84ffcc-44dgp"] Oct 31 07:41:19.611900 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:19.611884 4935 cpu_manager.go:414] "containerMap: removing container" podUID="9643855c-6524-4580-aafb-ccc4ffc2cab4" containerName="kserve-container" Oct 31 07:41:19.611988 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:19.611902 4935 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9643855c-6524-4580-aafb-ccc4ffc2cab4" containerName="kserve-container" Oct 31 07:41:19.611988 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:19.611937 4935 cpu_manager.go:414] "containerMap: removing container" podUID="9643855c-6524-4580-aafb-ccc4ffc2cab4" containerName="kube-rbac-proxy" Oct 31 07:41:19.611988 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:19.611946 4935 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9643855c-6524-4580-aafb-ccc4ffc2cab4" containerName="kube-rbac-proxy" Oct 31 07:41:19.612135 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:19.612038 4935 memory_manager.go:365] "RemoveStaleState removing state" podUID="9643855c-6524-4580-aafb-ccc4ffc2cab4" containerName="kserve-container" Oct 31 07:41:19.612135 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:19.612054 4935 memory_manager.go:365] "RemoveStaleState removing state" podUID="9643855c-6524-4580-aafb-ccc4ffc2cab4" containerName="kube-rbac-proxy" Oct 31 07:41:19.616410 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:19.616391 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-03ede73268/error-404-isvc-490af-predictor-778f84ffcc-44dgp" Oct 31 07:41:19.623502 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:19.623473 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-03ede73268\"/\"error-404-isvc-490af-predictor-serving-cert\"" Oct 31 07:41:19.625003 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:19.624566 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-03ede73268\"/\"error-404-isvc-490af-kube-rbac-proxy-sar-config\"" Oct 31 07:41:19.625003 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:19.624598 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-03ede73268\"/\"kube-root-ca.crt\"" Oct 31 07:41:19.625605 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:19.625587 4935 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-03ede73268/error-404-isvc-490af-predictor-778f84ffcc-44dgp"] Oct 31 07:41:19.649482 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:19.649457 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9f56af77-23b5-45e9-aa93-cdfcb8919917-proxy-tls\") pod \"error-404-isvc-490af-predictor-778f84ffcc-44dgp\" (UID: \"9f56af77-23b5-45e9-aa93-cdfcb8919917\") " pod="e2e-gw1-03ede73268/error-404-isvc-490af-predictor-778f84ffcc-44dgp" Oct 31 07:41:19.649575 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:19.649494 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9f56af77-23b5-45e9-aa93-cdfcb8919917-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-490af-predictor-778f84ffcc-44dgp\" (UID: \"9f56af77-23b5-45e9-aa93-cdfcb8919917\") " pod="e2e-gw1-03ede73268/error-404-isvc-490af-predictor-778f84ffcc-44dgp" Oct 31 07:41:19.649575 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:19.649524 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kt6sd\" (UniqueName: \"kubernetes.io/projected/9f56af77-23b5-45e9-aa93-cdfcb8919917-kube-api-access-kt6sd\") pod \"error-404-isvc-490af-predictor-778f84ffcc-44dgp\" (UID: \"9f56af77-23b5-45e9-aa93-cdfcb8919917\") " pod="e2e-gw1-03ede73268/error-404-isvc-490af-predictor-778f84ffcc-44dgp" Oct 31 07:41:19.727853 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:19.727825 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:41:19.749950 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:19.749913 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kt6sd\" (UniqueName: \"kubernetes.io/projected/9f56af77-23b5-45e9-aa93-cdfcb8919917-kube-api-access-kt6sd\") pod \"error-404-isvc-490af-predictor-778f84ffcc-44dgp\" (UID: \"9f56af77-23b5-45e9-aa93-cdfcb8919917\") " pod="e2e-gw1-03ede73268/error-404-isvc-490af-predictor-778f84ffcc-44dgp" Oct 31 07:41:19.750038 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:19.749981 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9f56af77-23b5-45e9-aa93-cdfcb8919917-proxy-tls\") pod \"error-404-isvc-490af-predictor-778f84ffcc-44dgp\" (UID: \"9f56af77-23b5-45e9-aa93-cdfcb8919917\") " pod="e2e-gw1-03ede73268/error-404-isvc-490af-predictor-778f84ffcc-44dgp" Oct 31 07:41:19.750038 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:19.750016 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9f56af77-23b5-45e9-aa93-cdfcb8919917-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-490af-predictor-778f84ffcc-44dgp\" (UID: \"9f56af77-23b5-45e9-aa93-cdfcb8919917\") " pod="e2e-gw1-03ede73268/error-404-isvc-490af-predictor-778f84ffcc-44dgp" Oct 31 07:41:19.750139 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:41:19.750124 4935 secret.go:189] Couldn't get secret e2e-gw1-03ede73268/error-404-isvc-490af-predictor-serving-cert: secret "error-404-isvc-490af-predictor-serving-cert" not found Oct 31 07:41:19.750210 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:41:19.750196 4935 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f56af77-23b5-45e9-aa93-cdfcb8919917-proxy-tls podName:9f56af77-23b5-45e9-aa93-cdfcb8919917 nodeName:}" failed. No retries permitted until 2026-10-31 07:41:20.25017999 +0000 UTC m=+654.120711506 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/9f56af77-23b5-45e9-aa93-cdfcb8919917-proxy-tls") pod "error-404-isvc-490af-predictor-778f84ffcc-44dgp" (UID: "9f56af77-23b5-45e9-aa93-cdfcb8919917") : secret "error-404-isvc-490af-predictor-serving-cert" not found Oct 31 07:41:19.750611 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:19.750596 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9f56af77-23b5-45e9-aa93-cdfcb8919917-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-490af-predictor-778f84ffcc-44dgp\" (UID: \"9f56af77-23b5-45e9-aa93-cdfcb8919917\") " pod="e2e-gw1-03ede73268/error-404-isvc-490af-predictor-778f84ffcc-44dgp" Oct 31 07:41:19.762504 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:19.762448 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kt6sd\" (UniqueName: \"kubernetes.io/projected/9f56af77-23b5-45e9-aa93-cdfcb8919917-kube-api-access-kt6sd\") pod \"error-404-isvc-490af-predictor-778f84ffcc-44dgp\" (UID: \"9f56af77-23b5-45e9-aa93-cdfcb8919917\") " pod="e2e-gw1-03ede73268/error-404-isvc-490af-predictor-778f84ffcc-44dgp" Oct 31 07:41:19.942460 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:19.942433 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:41:20.139690 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:20.139581 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:41:20.253599 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:20.253571 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9f56af77-23b5-45e9-aa93-cdfcb8919917-proxy-tls\") pod \"error-404-isvc-490af-predictor-778f84ffcc-44dgp\" (UID: \"9f56af77-23b5-45e9-aa93-cdfcb8919917\") " pod="e2e-gw1-03ede73268/error-404-isvc-490af-predictor-778f84ffcc-44dgp" Oct 31 07:41:20.256055 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:20.256025 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9f56af77-23b5-45e9-aa93-cdfcb8919917-proxy-tls\") pod \"error-404-isvc-490af-predictor-778f84ffcc-44dgp\" (UID: \"9f56af77-23b5-45e9-aa93-cdfcb8919917\") " pod="e2e-gw1-03ede73268/error-404-isvc-490af-predictor-778f84ffcc-44dgp" Oct 31 07:41:20.531727 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:20.531704 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-03ede73268/error-404-isvc-490af-predictor-778f84ffcc-44dgp" Oct 31 07:41:20.806663 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:20.806598 4935 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-03ede73268/error-404-isvc-490af-predictor-778f84ffcc-44dgp"] Oct 31 07:41:20.809544 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:41:20.809516 4935 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f56af77_23b5_45e9_aa93_cdfcb8919917.slice/crio-b0049f716749a922e78ab0e61071896e2fb65814f4ebdd4c807b5adffc262a46 WatchSource:0}: Error finding container b0049f716749a922e78ab0e61071896e2fb65814f4ebdd4c807b5adffc262a46: Status 404 returned error can't find the container with id b0049f716749a922e78ab0e61071896e2fb65814f4ebdd4c807b5adffc262a46 Oct 31 07:41:20.925316 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:20.925292 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-03ede73268/error-404-isvc-490af-predictor-778f84ffcc-44dgp" event={"ID":"9f56af77-23b5-45e9-aa93-cdfcb8919917","Type":"ContainerStarted","Data":"ade88ef5151a97600abab52cf46422c9b52e2a26d192efa075ec91f6ab8b9d19"} Oct 31 07:41:20.925443 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:20.925396 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-03ede73268/error-404-isvc-490af-predictor-778f84ffcc-44dgp" event={"ID":"9f56af77-23b5-45e9-aa93-cdfcb8919917","Type":"ContainerStarted","Data":"6b7d4081f9022e77020ce5bb9eb274f97ab66ec4632254eed5f0e2baf5665d97"} Oct 31 07:41:20.925443 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:20.925439 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-03ede73268/error-404-isvc-490af-predictor-778f84ffcc-44dgp" Oct 31 07:41:20.925547 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:20.925470 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-03ede73268/error-404-isvc-490af-predictor-778f84ffcc-44dgp" event={"ID":"9f56af77-23b5-45e9-aa93-cdfcb8919917","Type":"ContainerStarted","Data":"b0049f716749a922e78ab0e61071896e2fb65814f4ebdd4c807b5adffc262a46"} Oct 31 07:41:20.946759 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:20.946714 4935 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-03ede73268/error-404-isvc-490af-predictor-778f84ffcc-44dgp" podStartSLOduration=1.946699995 podStartE2EDuration="1.946699995s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:41:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:41:20.945096132 +0000 UTC m=+654.815627668" watchObservedRunningTime="2026-10-31 07:41:20.946699995 +0000 UTC m=+654.817231534" Oct 31 07:41:21.929783 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:21.929755 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-03ede73268/error-404-isvc-490af-predictor-778f84ffcc-44dgp" Oct 31 07:41:21.930811 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:21.930786 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/error-404-isvc-490af-predictor-778f84ffcc-44dgp" podUID="9f56af77-23b5-45e9-aa93-cdfcb8919917" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused" Oct 31 07:41:22.933790 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:22.933752 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/error-404-isvc-490af-predictor-778f84ffcc-44dgp" podUID="9f56af77-23b5-45e9-aa93-cdfcb8919917" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused" Oct 31 07:41:27.938854 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:27.938825 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-03ede73268/error-404-isvc-490af-predictor-778f84ffcc-44dgp" Oct 31 07:41:27.939260 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:27.939235 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/error-404-isvc-490af-predictor-778f84ffcc-44dgp" podUID="9f56af77-23b5-45e9-aa93-cdfcb8919917" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused" Oct 31 07:41:31.455169 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:41:31.455133 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 07:41:37.939489 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:37.939412 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/error-404-isvc-490af-predictor-778f84ffcc-44dgp" podUID="9f56af77-23b5-45e9-aa93-cdfcb8919917" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused" Oct 31 07:41:38.556766 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:38.556734 4935 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-5c1f590758/model-chainer-6b979c876d-j6srw"] Oct 31 07:41:38.562832 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:38.562816 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-5c1f590758/model-chainer-6b979c876d-j6srw" Oct 31 07:41:38.566688 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:38.566643 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-5c1f590758\"/\"openshift-service-ca.crt\"" Oct 31 07:41:38.568035 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:38.567962 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-5c1f590758\"/\"default-dockercfg-kdf9f\"" Oct 31 07:41:38.568035 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:38.568004 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-5c1f590758\"/\"model-chainer-serving-cert\"" Oct 31 07:41:38.568214 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:38.568053 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-5c1f590758\"/\"model-chainer-kube-rbac-proxy-sar-config\"" Oct 31 07:41:38.569852 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:38.569485 4935 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-5c1f590758/model-chainer-6b979c876d-j6srw"] Oct 31 07:41:38.584829 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:38.584807 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bbf5852-c16d-41a1-b2c1-b9a6ecddd898-openshift-service-ca-bundle\") pod \"model-chainer-6b979c876d-j6srw\" (UID: \"5bbf5852-c16d-41a1-b2c1-b9a6ecddd898\") " pod="e2e-gw0-5c1f590758/model-chainer-6b979c876d-j6srw" Oct 31 07:41:38.584918 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:38.584859 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5bbf5852-c16d-41a1-b2c1-b9a6ecddd898-proxy-tls\") pod \"model-chainer-6b979c876d-j6srw\" (UID: \"5bbf5852-c16d-41a1-b2c1-b9a6ecddd898\") " pod="e2e-gw0-5c1f590758/model-chainer-6b979c876d-j6srw" Oct 31 07:41:38.681984 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:38.681952 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:41:38.686037 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:38.686016 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bbf5852-c16d-41a1-b2c1-b9a6ecddd898-openshift-service-ca-bundle\") pod \"model-chainer-6b979c876d-j6srw\" (UID: \"5bbf5852-c16d-41a1-b2c1-b9a6ecddd898\") " pod="e2e-gw0-5c1f590758/model-chainer-6b979c876d-j6srw" Oct 31 07:41:38.686097 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:38.686067 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5bbf5852-c16d-41a1-b2c1-b9a6ecddd898-proxy-tls\") pod \"model-chainer-6b979c876d-j6srw\" (UID: \"5bbf5852-c16d-41a1-b2c1-b9a6ecddd898\") " pod="e2e-gw0-5c1f590758/model-chainer-6b979c876d-j6srw" Oct 31 07:41:38.686182 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:41:38.686171 4935 secret.go:189] Couldn't get secret e2e-gw0-5c1f590758/model-chainer-serving-cert: secret "model-chainer-serving-cert" not found Oct 31 07:41:38.686253 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:41:38.686245 4935 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bbf5852-c16d-41a1-b2c1-b9a6ecddd898-proxy-tls podName:5bbf5852-c16d-41a1-b2c1-b9a6ecddd898 nodeName:}" failed. No retries permitted until 2026-10-31 07:41:39.186230934 +0000 UTC m=+673.056762449 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/5bbf5852-c16d-41a1-b2c1-b9a6ecddd898-proxy-tls") pod "model-chainer-6b979c876d-j6srw" (UID: "5bbf5852-c16d-41a1-b2c1-b9a6ecddd898") : secret "model-chainer-serving-cert" not found Oct 31 07:41:38.686638 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:38.686622 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bbf5852-c16d-41a1-b2c1-b9a6ecddd898-openshift-service-ca-bundle\") pod \"model-chainer-6b979c876d-j6srw\" (UID: \"5bbf5852-c16d-41a1-b2c1-b9a6ecddd898\") " pod="e2e-gw0-5c1f590758/model-chainer-6b979c876d-j6srw" Oct 31 07:41:38.919791 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:38.919733 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:41:39.190113 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:39.190043 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5bbf5852-c16d-41a1-b2c1-b9a6ecddd898-proxy-tls\") pod \"model-chainer-6b979c876d-j6srw\" (UID: \"5bbf5852-c16d-41a1-b2c1-b9a6ecddd898\") " pod="e2e-gw0-5c1f590758/model-chainer-6b979c876d-j6srw" Oct 31 07:41:39.192560 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:39.192539 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5bbf5852-c16d-41a1-b2c1-b9a6ecddd898-proxy-tls\") pod \"model-chainer-6b979c876d-j6srw\" (UID: \"5bbf5852-c16d-41a1-b2c1-b9a6ecddd898\") " pod="e2e-gw0-5c1f590758/model-chainer-6b979c876d-j6srw" Oct 31 07:41:39.477523 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:39.477496 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-5c1f590758/model-chainer-6b979c876d-j6srw" Oct 31 07:41:39.749999 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:39.749974 4935 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-5c1f590758/model-chainer-6b979c876d-j6srw"] Oct 31 07:41:39.752444 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:41:39.752407 4935 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bbf5852_c16d_41a1_b2c1_b9a6ecddd898.slice/crio-35c97a945244c779e203422ed96e7af76fbd26966515c1d9c3f878d2e1e9b828 WatchSource:0}: Error finding container 35c97a945244c779e203422ed96e7af76fbd26966515c1d9c3f878d2e1e9b828: Status 404 returned error can't find the container with id 35c97a945244c779e203422ed96e7af76fbd26966515c1d9c3f878d2e1e9b828 Oct 31 07:41:40.005736 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:40.005653 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-5c1f590758/model-chainer-6b979c876d-j6srw" event={"ID":"5bbf5852-c16d-41a1-b2c1-b9a6ecddd898","Type":"ContainerStarted","Data":"9c3a1f01995027b8648284d8dea326546e8e862c2baa8a4e4e5031a19c30be96"} Oct 31 07:41:40.005736 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:40.005704 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-5c1f590758/model-chainer-6b979c876d-j6srw" event={"ID":"5bbf5852-c16d-41a1-b2c1-b9a6ecddd898","Type":"ContainerStarted","Data":"35c97a945244c779e203422ed96e7af76fbd26966515c1d9c3f878d2e1e9b828"} Oct 31 07:41:40.005928 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:40.005782 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-5c1f590758/model-chainer-6b979c876d-j6srw" Oct 31 07:41:40.023531 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:40.023490 4935 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-5c1f590758/model-chainer-6b979c876d-j6srw" podStartSLOduration=2.023479849 podStartE2EDuration="2.023479849s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:41:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:41:40.022708747 +0000 UTC m=+673.893240285" watchObservedRunningTime="2026-10-31 07:41:40.023479849 +0000 UTC m=+673.894011385" Oct 31 07:41:46.015736 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:46.015706 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-5c1f590758/model-chainer-6b979c876d-j6srw" Oct 31 07:41:47.939736 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:47.939696 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/error-404-isvc-490af-predictor-778f84ffcc-44dgp" podUID="9f56af77-23b5-45e9-aa93-cdfcb8919917" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused" Oct 31 07:41:50.369113 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:50.369079 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:41:50.570451 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:50.570421 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:41:57.836041 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:57.836010 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:41:57.939307 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:57.939259 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/error-404-isvc-490af-predictor-778f84ffcc-44dgp" podUID="9f56af77-23b5-45e9-aa93-cdfcb8919917" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused" Oct 31 07:41:57.990678 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:57.990643 4935 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-5c1f590758/error-404-isvc-48632-predictor-7c7bc8bf7d-xj4tv"] Oct 31 07:41:57.995895 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:57.995875 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-5c1f590758/error-404-isvc-48632-predictor-7c7bc8bf7d-xj4tv" Oct 31 07:41:58.004479 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:58.004455 4935 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-5c1f590758/error-404-isvc-48632-predictor-7c7bc8bf7d-xj4tv"] Oct 31 07:41:58.005051 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:58.005029 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-5c1f590758\"/\"error-404-isvc-48632-predictor-serving-cert\"" Oct 31 07:41:58.005129 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:58.005053 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-5c1f590758\"/\"error-404-isvc-48632-kube-rbac-proxy-sar-config\"" Oct 31 07:41:58.005238 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:58.005218 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-5c1f590758\"/\"kube-root-ca.crt\"" Oct 31 07:41:58.042278 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:58.042249 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:41:58.134529 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:58.134439 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8cadcae9-d852-4529-9246-439f7f4ff955-proxy-tls\") pod \"error-404-isvc-48632-predictor-7c7bc8bf7d-xj4tv\" (UID: \"8cadcae9-d852-4529-9246-439f7f4ff955\") " pod="e2e-gw0-5c1f590758/error-404-isvc-48632-predictor-7c7bc8bf7d-xj4tv" Oct 31 07:41:58.134529 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:58.134491 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvkzc\" (UniqueName: \"kubernetes.io/projected/8cadcae9-d852-4529-9246-439f7f4ff955-kube-api-access-kvkzc\") pod \"error-404-isvc-48632-predictor-7c7bc8bf7d-xj4tv\" (UID: \"8cadcae9-d852-4529-9246-439f7f4ff955\") " pod="e2e-gw0-5c1f590758/error-404-isvc-48632-predictor-7c7bc8bf7d-xj4tv" Oct 31 07:41:58.134722 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:58.134592 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8cadcae9-d852-4529-9246-439f7f4ff955-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-48632-predictor-7c7bc8bf7d-xj4tv\" (UID: \"8cadcae9-d852-4529-9246-439f7f4ff955\") " pod="e2e-gw0-5c1f590758/error-404-isvc-48632-predictor-7c7bc8bf7d-xj4tv" Oct 31 07:41:58.235943 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:58.235905 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8cadcae9-d852-4529-9246-439f7f4ff955-proxy-tls\") pod \"error-404-isvc-48632-predictor-7c7bc8bf7d-xj4tv\" (UID: \"8cadcae9-d852-4529-9246-439f7f4ff955\") " pod="e2e-gw0-5c1f590758/error-404-isvc-48632-predictor-7c7bc8bf7d-xj4tv" Oct 31 07:41:58.235943 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:58.235961 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvkzc\" (UniqueName: \"kubernetes.io/projected/8cadcae9-d852-4529-9246-439f7f4ff955-kube-api-access-kvkzc\") pod \"error-404-isvc-48632-predictor-7c7bc8bf7d-xj4tv\" (UID: \"8cadcae9-d852-4529-9246-439f7f4ff955\") " pod="e2e-gw0-5c1f590758/error-404-isvc-48632-predictor-7c7bc8bf7d-xj4tv" Oct 31 07:41:58.236162 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:58.235995 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8cadcae9-d852-4529-9246-439f7f4ff955-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-48632-predictor-7c7bc8bf7d-xj4tv\" (UID: \"8cadcae9-d852-4529-9246-439f7f4ff955\") " pod="e2e-gw0-5c1f590758/error-404-isvc-48632-predictor-7c7bc8bf7d-xj4tv" Oct 31 07:41:58.236634 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:58.236615 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8cadcae9-d852-4529-9246-439f7f4ff955-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-48632-predictor-7c7bc8bf7d-xj4tv\" (UID: \"8cadcae9-d852-4529-9246-439f7f4ff955\") " pod="e2e-gw0-5c1f590758/error-404-isvc-48632-predictor-7c7bc8bf7d-xj4tv" Oct 31 07:41:58.238531 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:58.238499 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8cadcae9-d852-4529-9246-439f7f4ff955-proxy-tls\") pod \"error-404-isvc-48632-predictor-7c7bc8bf7d-xj4tv\" (UID: \"8cadcae9-d852-4529-9246-439f7f4ff955\") " pod="e2e-gw0-5c1f590758/error-404-isvc-48632-predictor-7c7bc8bf7d-xj4tv" Oct 31 07:41:58.240489 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:58.240467 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:41:58.247813 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:58.247792 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvkzc\" (UniqueName: \"kubernetes.io/projected/8cadcae9-d852-4529-9246-439f7f4ff955-kube-api-access-kvkzc\") pod \"error-404-isvc-48632-predictor-7c7bc8bf7d-xj4tv\" (UID: \"8cadcae9-d852-4529-9246-439f7f4ff955\") " pod="e2e-gw0-5c1f590758/error-404-isvc-48632-predictor-7c7bc8bf7d-xj4tv" Oct 31 07:41:58.311319 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:58.311295 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-5c1f590758/error-404-isvc-48632-predictor-7c7bc8bf7d-xj4tv" Oct 31 07:41:58.584983 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:58.584932 4935 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-5c1f590758/error-404-isvc-48632-predictor-7c7bc8bf7d-xj4tv"] Oct 31 07:41:58.588062 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:41:58.588038 4935 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cadcae9_d852_4529_9246_439f7f4ff955.slice/crio-6c5b82e5f3a353460d16bf2c4bbc804e5aa3a94c5bf3feba7b8c66d1907456cf WatchSource:0}: Error finding container 6c5b82e5f3a353460d16bf2c4bbc804e5aa3a94c5bf3feba7b8c66d1907456cf: Status 404 returned error can't find the container with id 6c5b82e5f3a353460d16bf2c4bbc804e5aa3a94c5bf3feba7b8c66d1907456cf Oct 31 07:41:59.089532 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:59.089497 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-5c1f590758/error-404-isvc-48632-predictor-7c7bc8bf7d-xj4tv" event={"ID":"8cadcae9-d852-4529-9246-439f7f4ff955","Type":"ContainerStarted","Data":"255874a6dfab10f08a7e37a89b2f7777841e07d07e801ca222fe0d2e8505fe9c"} Oct 31 07:41:59.090020 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:59.089559 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-5c1f590758/error-404-isvc-48632-predictor-7c7bc8bf7d-xj4tv" event={"ID":"8cadcae9-d852-4529-9246-439f7f4ff955","Type":"ContainerStarted","Data":"462e361112dd9076bd58225686be395f94a510621a15679f39047bc50cb129da"} Oct 31 07:41:59.090020 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:59.089579 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-5c1f590758/error-404-isvc-48632-predictor-7c7bc8bf7d-xj4tv" event={"ID":"8cadcae9-d852-4529-9246-439f7f4ff955","Type":"ContainerStarted","Data":"6c5b82e5f3a353460d16bf2c4bbc804e5aa3a94c5bf3feba7b8c66d1907456cf"} Oct 31 07:41:59.090020 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:59.089933 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-5c1f590758/error-404-isvc-48632-predictor-7c7bc8bf7d-xj4tv" Oct 31 07:41:59.090020 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:59.089970 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-5c1f590758/error-404-isvc-48632-predictor-7c7bc8bf7d-xj4tv" Oct 31 07:41:59.091102 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:59.091073 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-5c1f590758/error-404-isvc-48632-predictor-7c7bc8bf7d-xj4tv" podUID="8cadcae9-d852-4529-9246-439f7f4ff955" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused" Oct 31 07:41:59.109245 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:41:59.109141 4935 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-5c1f590758/error-404-isvc-48632-predictor-7c7bc8bf7d-xj4tv" podStartSLOduration=2.109124831 podStartE2EDuration="2.109124831s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:41:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:41:59.108629781 +0000 UTC m=+692.979161316" watchObservedRunningTime="2026-10-31 07:41:59.109124831 +0000 UTC m=+692.979656370" Oct 31 07:42:00.095303 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:42:00.095261 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-5c1f590758/error-404-isvc-48632-predictor-7c7bc8bf7d-xj4tv" podUID="8cadcae9-d852-4529-9246-439f7f4ff955" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused" Oct 31 07:42:01.454419 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:42:01.454382 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 07:42:05.100141 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:42:05.100114 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-5c1f590758/error-404-isvc-48632-predictor-7c7bc8bf7d-xj4tv" Oct 31 07:42:05.100617 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:42:05.100590 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-5c1f590758/error-404-isvc-48632-predictor-7c7bc8bf7d-xj4tv" podUID="8cadcae9-d852-4529-9246-439f7f4ff955" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused" Oct 31 07:42:07.939896 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:42:07.939857 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/error-404-isvc-490af-predictor-778f84ffcc-44dgp" podUID="9f56af77-23b5-45e9-aa93-cdfcb8919917" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused" Oct 31 07:42:15.100693 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:42:15.100656 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-5c1f590758/error-404-isvc-48632-predictor-7c7bc8bf7d-xj4tv" podUID="8cadcae9-d852-4529-9246-439f7f4ff955" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused" Oct 31 07:42:17.940664 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:42:17.940627 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-03ede73268/error-404-isvc-490af-predictor-778f84ffcc-44dgp" Oct 31 07:42:17.963071 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:42:17.963040 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:42:25.101330 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:42:25.101294 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-5c1f590758/error-404-isvc-48632-predictor-7c7bc8bf7d-xj4tv" podUID="8cadcae9-d852-4529-9246-439f7f4ff955" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused" Oct 31 07:42:29.609855 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:42:29.609819 4935 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-03ede73268/switch-graph-490af-5ccfdb64c7-mzjwg"] Oct 31 07:42:29.613406 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:42:29.613390 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-03ede73268/switch-graph-490af-5ccfdb64c7-mzjwg" Oct 31 07:42:29.616899 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:42:29.616873 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-03ede73268\"/\"switch-graph-490af-serving-cert\"" Oct 31 07:42:29.617012 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:42:29.616933 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-03ede73268\"/\"switch-graph-490af-kube-rbac-proxy-sar-config\"" Oct 31 07:42:29.621573 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:42:29.621552 4935 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-03ede73268/switch-graph-490af-5ccfdb64c7-mzjwg"] Oct 31 07:42:29.740774 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:42:29.740741 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:42:29.779850 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:42:29.779822 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f678a8c-1a16-41eb-93d1-45c89b51c98d-openshift-service-ca-bundle\") pod \"switch-graph-490af-5ccfdb64c7-mzjwg\" (UID: \"8f678a8c-1a16-41eb-93d1-45c89b51c98d\") " pod="e2e-gw1-03ede73268/switch-graph-490af-5ccfdb64c7-mzjwg" Oct 31 07:42:29.779974 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:42:29.779874 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8f678a8c-1a16-41eb-93d1-45c89b51c98d-proxy-tls\") pod \"switch-graph-490af-5ccfdb64c7-mzjwg\" (UID: \"8f678a8c-1a16-41eb-93d1-45c89b51c98d\") " pod="e2e-gw1-03ede73268/switch-graph-490af-5ccfdb64c7-mzjwg" Oct 31 07:42:29.881323 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:42:29.881262 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f678a8c-1a16-41eb-93d1-45c89b51c98d-openshift-service-ca-bundle\") pod \"switch-graph-490af-5ccfdb64c7-mzjwg\" (UID: \"8f678a8c-1a16-41eb-93d1-45c89b51c98d\") " pod="e2e-gw1-03ede73268/switch-graph-490af-5ccfdb64c7-mzjwg" Oct 31 07:42:29.881323 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:42:29.881313 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8f678a8c-1a16-41eb-93d1-45c89b51c98d-proxy-tls\") pod \"switch-graph-490af-5ccfdb64c7-mzjwg\" (UID: \"8f678a8c-1a16-41eb-93d1-45c89b51c98d\") " pod="e2e-gw1-03ede73268/switch-graph-490af-5ccfdb64c7-mzjwg" Oct 31 07:42:29.881891 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:42:29.881869 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f678a8c-1a16-41eb-93d1-45c89b51c98d-openshift-service-ca-bundle\") pod \"switch-graph-490af-5ccfdb64c7-mzjwg\" (UID: \"8f678a8c-1a16-41eb-93d1-45c89b51c98d\") " pod="e2e-gw1-03ede73268/switch-graph-490af-5ccfdb64c7-mzjwg" Oct 31 07:42:29.883794 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:42:29.883772 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8f678a8c-1a16-41eb-93d1-45c89b51c98d-proxy-tls\") pod \"switch-graph-490af-5ccfdb64c7-mzjwg\" (UID: \"8f678a8c-1a16-41eb-93d1-45c89b51c98d\") " pod="e2e-gw1-03ede73268/switch-graph-490af-5ccfdb64c7-mzjwg" Oct 31 07:42:29.926800 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:42:29.926775 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-03ede73268/switch-graph-490af-5ccfdb64c7-mzjwg" Oct 31 07:42:29.941581 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:42:29.941553 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:42:30.199329 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:42:30.199304 4935 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-03ede73268/switch-graph-490af-5ccfdb64c7-mzjwg"] Oct 31 07:42:30.201824 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:42:30.201796 4935 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f678a8c_1a16_41eb_93d1_45c89b51c98d.slice/crio-8c342a67dfcced990c23afe315b1d68ea919f6a6f9f6fe459e52656f4a136801 WatchSource:0}: Error finding container 8c342a67dfcced990c23afe315b1d68ea919f6a6f9f6fe459e52656f4a136801: Status 404 returned error can't find the container with id 8c342a67dfcced990c23afe315b1d68ea919f6a6f9f6fe459e52656f4a136801 Oct 31 07:42:30.226519 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:42:30.226492 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-03ede73268/switch-graph-490af-5ccfdb64c7-mzjwg" event={"ID":"8f678a8c-1a16-41eb-93d1-45c89b51c98d","Type":"ContainerStarted","Data":"8c342a67dfcced990c23afe315b1d68ea919f6a6f9f6fe459e52656f4a136801"} Oct 31 07:42:31.232843 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:42:31.232805 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-03ede73268/switch-graph-490af-5ccfdb64c7-mzjwg" event={"ID":"8f678a8c-1a16-41eb-93d1-45c89b51c98d","Type":"ContainerStarted","Data":"e07751231ecd2c1bac7ab9dc1c3119b408b4a7492ce747cab103a356af9d0fe5"} Oct 31 07:42:31.233249 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:42:31.232945 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-03ede73268/switch-graph-490af-5ccfdb64c7-mzjwg" Oct 31 07:42:31.251102 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:42:31.251057 4935 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-03ede73268/switch-graph-490af-5ccfdb64c7-mzjwg" podStartSLOduration=2.251046526 podStartE2EDuration="2.251046526s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:42:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:42:31.249899856 +0000 UTC m=+725.120431393" watchObservedRunningTime="2026-10-31 07:42:31.251046526 +0000 UTC m=+725.121578089" Oct 31 07:42:31.454553 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:42:31.454519 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 07:42:35.100654 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:42:35.100608 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-5c1f590758/error-404-isvc-48632-predictor-7c7bc8bf7d-xj4tv" podUID="8cadcae9-d852-4529-9246-439f7f4ff955" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused" Oct 31 07:42:37.242854 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:42:37.242825 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-03ede73268/switch-graph-490af-5ccfdb64c7-mzjwg" Oct 31 07:42:45.100725 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:42:45.100687 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-5c1f590758/error-404-isvc-48632-predictor-7c7bc8bf7d-xj4tv" podUID="8cadcae9-d852-4529-9246-439f7f4ff955" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused" Oct 31 07:42:55.101503 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:42:55.101474 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-5c1f590758/error-404-isvc-48632-predictor-7c7bc8bf7d-xj4tv" Oct 31 07:43:01.454589 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:43:01.454551 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 07:43:08.009289 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:43:08.009212 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:43:08.210836 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:43:08.210807 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:43:16.967870 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:43:16.967839 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:43:26.572147 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:43:26.572116 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:43:26.774512 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:43:26.774481 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:43:31.454987 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:43:31.454943 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 07:44:01.454503 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:44:01.454463 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 07:44:17.020453 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:44:17.020419 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:44:31.455412 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:44:31.455375 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 07:45:01.453826 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:45:01.453793 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 07:45:11.306673 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:45:11.306636 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:45:16.976488 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:45:16.976453 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:45:26.302167 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:45:26.302141 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jztcz_396e1c5e-3700-4fe6-b367-54ad2a244225/console-operator/2.log" Oct 31 07:45:26.305008 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:45:26.304984 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jztcz_396e1c5e-3700-4fe6-b367-54ad2a244225/console-operator/2.log" Oct 31 07:45:31.455087 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:45:31.455047 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 07:46:01.454634 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:46:01.454596 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 07:46:17.181140 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:46:17.181055 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:46:31.455007 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:46:31.454963 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 07:46:32.439574 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:46:32.439541 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:47:01.454716 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:47:01.454678 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 07:47:17.201811 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:47:17.201780 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:47:31.454744 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:47:31.454709 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 07:48:01.454116 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:48:01.454081 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 07:48:17.237026 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:48:17.236988 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:48:31.455102 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:48:31.455063 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 07:49:01.454876 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:49:01.454833 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 07:49:17.293997 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:49:17.293911 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:49:31.454756 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:49:31.454720 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 07:50:01.454393 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:50:01.454274 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 07:50:17.342075 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:17.342045 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:50:26.330211 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:26.330181 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jztcz_396e1c5e-3700-4fe6-b367-54ad2a244225/console-operator/2.log" Oct 31 07:50:26.332869 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:26.332846 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jztcz_396e1c5e-3700-4fe6-b367-54ad2a244225/console-operator/2.log" Oct 31 07:50:31.454461 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:50:31.454407 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 07:50:45.319256 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:45.319217 4935 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-03ede73268/error-404-isvc-490af-predictor-778f84ffcc-44dgp"] Oct 31 07:50:45.319876 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:45.319509 4935 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-03ede73268/error-404-isvc-490af-predictor-778f84ffcc-44dgp" podUID="9f56af77-23b5-45e9-aa93-cdfcb8919917" containerName="kserve-container" containerID="cri-o://6b7d4081f9022e77020ce5bb9eb274f97ab66ec4632254eed5f0e2baf5665d97" gracePeriod=30 Oct 31 07:50:45.319876 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:45.319562 4935 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-03ede73268/error-404-isvc-490af-predictor-778f84ffcc-44dgp" podUID="9f56af77-23b5-45e9-aa93-cdfcb8919917" containerName="kube-rbac-proxy" containerID="cri-o://ade88ef5151a97600abab52cf46422c9b52e2a26d192efa075ec91f6ab8b9d19" gracePeriod=30 Oct 31 07:50:45.388039 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:45.388004 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:50:45.595704 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:45.595622 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:50:46.276989 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:46.276959 4935 generic.go:356] "Generic (PLEG): container finished" podID="9f56af77-23b5-45e9-aa93-cdfcb8919917" containerID="ade88ef5151a97600abab52cf46422c9b52e2a26d192efa075ec91f6ab8b9d19" exitCode=0 Oct 31 07:50:46.277160 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:46.277028 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-03ede73268/error-404-isvc-490af-predictor-778f84ffcc-44dgp" event={"ID":"9f56af77-23b5-45e9-aa93-cdfcb8919917","Type":"ContainerDied","Data":"ade88ef5151a97600abab52cf46422c9b52e2a26d192efa075ec91f6ab8b9d19"} Oct 31 07:50:47.934153 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:47.934110 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/error-404-isvc-490af-predictor-778f84ffcc-44dgp" podUID="9f56af77-23b5-45e9-aa93-cdfcb8919917" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.31:8643/healthz\": dial tcp 10.132.0.31:8643: connect: connection refused" Oct 31 07:50:47.939473 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:47.939439 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/error-404-isvc-490af-predictor-778f84ffcc-44dgp" podUID="9f56af77-23b5-45e9-aa93-cdfcb8919917" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused" Oct 31 07:50:48.289395 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:48.289333 4935 generic.go:356] "Generic (PLEG): container finished" podID="9f56af77-23b5-45e9-aa93-cdfcb8919917" containerID="6b7d4081f9022e77020ce5bb9eb274f97ab66ec4632254eed5f0e2baf5665d97" exitCode=0 Oct 31 07:50:48.289553 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:48.289423 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-03ede73268/error-404-isvc-490af-predictor-778f84ffcc-44dgp" event={"ID":"9f56af77-23b5-45e9-aa93-cdfcb8919917","Type":"ContainerDied","Data":"6b7d4081f9022e77020ce5bb9eb274f97ab66ec4632254eed5f0e2baf5665d97"} Oct 31 07:50:48.368140 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:48.368116 4935 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-03ede73268/error-404-isvc-490af-predictor-778f84ffcc-44dgp" Oct 31 07:50:48.437909 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:48.437881 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9f56af77-23b5-45e9-aa93-cdfcb8919917-proxy-tls\" (UniqueName: \"kubernetes.io/secret/9f56af77-23b5-45e9-aa93-cdfcb8919917-proxy-tls\") pod \"9f56af77-23b5-45e9-aa93-cdfcb8919917\" (UID: \"9f56af77-23b5-45e9-aa93-cdfcb8919917\") " Oct 31 07:50:48.438079 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:48.437922 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9f56af77-23b5-45e9-aa93-cdfcb8919917-kube-api-access-kt6sd\" (UniqueName: \"kubernetes.io/projected/9f56af77-23b5-45e9-aa93-cdfcb8919917-kube-api-access-kt6sd\") pod \"9f56af77-23b5-45e9-aa93-cdfcb8919917\" (UID: \"9f56af77-23b5-45e9-aa93-cdfcb8919917\") " Oct 31 07:50:48.438079 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:48.437946 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9f56af77-23b5-45e9-aa93-cdfcb8919917-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9f56af77-23b5-45e9-aa93-cdfcb8919917-kube-rbac-proxy-sar-config\") pod \"9f56af77-23b5-45e9-aa93-cdfcb8919917\" (UID: \"9f56af77-23b5-45e9-aa93-cdfcb8919917\") " Oct 31 07:50:48.438333 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:48.438306 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f56af77-23b5-45e9-aa93-cdfcb8919917-kube-rbac-proxy-sar-config" pod "9f56af77-23b5-45e9-aa93-cdfcb8919917" (UID: "9f56af77-23b5-45e9-aa93-cdfcb8919917"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 31 07:50:48.440121 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:48.440098 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f56af77-23b5-45e9-aa93-cdfcb8919917-proxy-tls" pod "9f56af77-23b5-45e9-aa93-cdfcb8919917" (UID: "9f56af77-23b5-45e9-aa93-cdfcb8919917"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 31 07:50:48.440190 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:48.440145 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f56af77-23b5-45e9-aa93-cdfcb8919917-kube-api-access-kt6sd" pod "9f56af77-23b5-45e9-aa93-cdfcb8919917" (UID: "9f56af77-23b5-45e9-aa93-cdfcb8919917"). InnerVolumeSpecName "kube-api-access-kt6sd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 31 07:50:48.538605 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:48.538569 4935 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9f56af77-23b5-45e9-aa93-cdfcb8919917-proxy-tls\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 07:50:48.538605 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:48.538603 4935 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kt6sd\" (UniqueName: \"kubernetes.io/projected/9f56af77-23b5-45e9-aa93-cdfcb8919917-kube-api-access-kt6sd\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 07:50:48.538605 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:48.538614 4935 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9f56af77-23b5-45e9-aa93-cdfcb8919917-kube-rbac-proxy-sar-config\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 07:50:49.295249 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:49.295221 4935 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-03ede73268/error-404-isvc-490af-predictor-778f84ffcc-44dgp" Oct 31 07:50:49.295249 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:49.295227 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-03ede73268/error-404-isvc-490af-predictor-778f84ffcc-44dgp" event={"ID":"9f56af77-23b5-45e9-aa93-cdfcb8919917","Type":"ContainerDied","Data":"b0049f716749a922e78ab0e61071896e2fb65814f4ebdd4c807b5adffc262a46"} Oct 31 07:50:49.295772 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:49.295295 4935 scope.go:122] "RemoveContainer" containerID="ade88ef5151a97600abab52cf46422c9b52e2a26d192efa075ec91f6ab8b9d19" Oct 31 07:50:49.303917 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:49.303900 4935 scope.go:122] "RemoveContainer" containerID="6b7d4081f9022e77020ce5bb9eb274f97ab66ec4632254eed5f0e2baf5665d97" Oct 31 07:50:49.316377 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:49.316338 4935 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-03ede73268/error-404-isvc-490af-predictor-778f84ffcc-44dgp"] Oct 31 07:50:49.320136 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:49.320117 4935 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-03ede73268/error-404-isvc-490af-predictor-778f84ffcc-44dgp"] Oct 31 07:50:50.242419 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:50.242384 4935 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9f56af77-23b5-45e9-aa93-cdfcb8919917" path="/var/lib/kubelet/pods/9f56af77-23b5-45e9-aa93-cdfcb8919917/volumes" Oct 31 07:50:50.805641 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:50.805607 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:50:50.907584 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:50.907552 4935 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-03ede73268/error-404-isvc-e6a49-predictor-56df4bc975-jd92x"] Oct 31 07:50:50.907969 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:50.907954 4935 cpu_manager.go:414] "containerMap: removing container" podUID="9f56af77-23b5-45e9-aa93-cdfcb8919917" containerName="kube-rbac-proxy" Oct 31 07:50:50.908012 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:50.907970 4935 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9f56af77-23b5-45e9-aa93-cdfcb8919917" containerName="kube-rbac-proxy" Oct 31 07:50:50.908012 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:50.907985 4935 cpu_manager.go:414] "containerMap: removing container" podUID="9f56af77-23b5-45e9-aa93-cdfcb8919917" containerName="kserve-container" Oct 31 07:50:50.908012 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:50.907990 4935 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9f56af77-23b5-45e9-aa93-cdfcb8919917" containerName="kserve-container" Oct 31 07:50:50.908116 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:50.908103 4935 memory_manager.go:365] "RemoveStaleState removing state" podUID="9f56af77-23b5-45e9-aa93-cdfcb8919917" containerName="kube-rbac-proxy" Oct 31 07:50:50.908153 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:50.908122 4935 memory_manager.go:365] "RemoveStaleState removing state" podUID="9f56af77-23b5-45e9-aa93-cdfcb8919917" containerName="kserve-container" Oct 31 07:50:50.912456 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:50.912436 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-03ede73268/error-404-isvc-e6a49-predictor-56df4bc975-jd92x" Oct 31 07:50:50.916676 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:50.916650 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-03ede73268\"/\"error-404-isvc-e6a49-kube-rbac-proxy-sar-config\"" Oct 31 07:50:50.916676 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:50.916679 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-03ede73268\"/\"kube-root-ca.crt\"" Oct 31 07:50:50.921162 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:50.921140 4935 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-03ede73268/error-404-isvc-e6a49-predictor-56df4bc975-jd92x"] Oct 31 07:50:50.958002 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:50.957976 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lctjz\" (UniqueName: \"kubernetes.io/projected/57190b16-3213-44a0-8dab-e2fa9e22ef88-kube-api-access-lctjz\") pod \"error-404-isvc-e6a49-predictor-56df4bc975-jd92x\" (UID: \"57190b16-3213-44a0-8dab-e2fa9e22ef88\") " pod="e2e-gw1-03ede73268/error-404-isvc-e6a49-predictor-56df4bc975-jd92x" Oct 31 07:50:50.958123 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:50.958014 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/57190b16-3213-44a0-8dab-e2fa9e22ef88-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-e6a49-predictor-56df4bc975-jd92x\" (UID: \"57190b16-3213-44a0-8dab-e2fa9e22ef88\") " pod="e2e-gw1-03ede73268/error-404-isvc-e6a49-predictor-56df4bc975-jd92x" Oct 31 07:50:50.958123 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:50.958036 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/57190b16-3213-44a0-8dab-e2fa9e22ef88-proxy-tls\") pod \"error-404-isvc-e6a49-predictor-56df4bc975-jd92x\" (UID: \"57190b16-3213-44a0-8dab-e2fa9e22ef88\") " pod="e2e-gw1-03ede73268/error-404-isvc-e6a49-predictor-56df4bc975-jd92x" Oct 31 07:50:50.992536 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:50.992515 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-03ede73268\"/\"error-404-isvc-e6a49-predictor-serving-cert\"" Oct 31 07:50:51.010543 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:51.010511 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:50:51.058874 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:51.058821 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lctjz\" (UniqueName: \"kubernetes.io/projected/57190b16-3213-44a0-8dab-e2fa9e22ef88-kube-api-access-lctjz\") pod \"error-404-isvc-e6a49-predictor-56df4bc975-jd92x\" (UID: \"57190b16-3213-44a0-8dab-e2fa9e22ef88\") " pod="e2e-gw1-03ede73268/error-404-isvc-e6a49-predictor-56df4bc975-jd92x" Oct 31 07:50:51.058874 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:51.058859 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/57190b16-3213-44a0-8dab-e2fa9e22ef88-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-e6a49-predictor-56df4bc975-jd92x\" (UID: \"57190b16-3213-44a0-8dab-e2fa9e22ef88\") " pod="e2e-gw1-03ede73268/error-404-isvc-e6a49-predictor-56df4bc975-jd92x" Oct 31 07:50:51.059031 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:51.058879 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/57190b16-3213-44a0-8dab-e2fa9e22ef88-proxy-tls\") pod \"error-404-isvc-e6a49-predictor-56df4bc975-jd92x\" (UID: \"57190b16-3213-44a0-8dab-e2fa9e22ef88\") " pod="e2e-gw1-03ede73268/error-404-isvc-e6a49-predictor-56df4bc975-jd92x" Oct 31 07:50:51.059492 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:51.059475 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/57190b16-3213-44a0-8dab-e2fa9e22ef88-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-e6a49-predictor-56df4bc975-jd92x\" (UID: \"57190b16-3213-44a0-8dab-e2fa9e22ef88\") " pod="e2e-gw1-03ede73268/error-404-isvc-e6a49-predictor-56df4bc975-jd92x" Oct 31 07:50:51.061379 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:51.061339 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/57190b16-3213-44a0-8dab-e2fa9e22ef88-proxy-tls\") pod \"error-404-isvc-e6a49-predictor-56df4bc975-jd92x\" (UID: \"57190b16-3213-44a0-8dab-e2fa9e22ef88\") " pod="e2e-gw1-03ede73268/error-404-isvc-e6a49-predictor-56df4bc975-jd92x" Oct 31 07:50:51.070979 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:51.070950 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lctjz\" (UniqueName: \"kubernetes.io/projected/57190b16-3213-44a0-8dab-e2fa9e22ef88-kube-api-access-lctjz\") pod \"error-404-isvc-e6a49-predictor-56df4bc975-jd92x\" (UID: \"57190b16-3213-44a0-8dab-e2fa9e22ef88\") " pod="e2e-gw1-03ede73268/error-404-isvc-e6a49-predictor-56df4bc975-jd92x" Oct 31 07:50:51.213483 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:51.213454 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:50:51.228415 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:51.228392 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-03ede73268/error-404-isvc-e6a49-predictor-56df4bc975-jd92x" Oct 31 07:50:51.504491 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:51.504264 4935 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-03ede73268/error-404-isvc-e6a49-predictor-56df4bc975-jd92x"] Oct 31 07:50:51.506667 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:50:51.506644 4935 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57190b16_3213_44a0_8dab_e2fa9e22ef88.slice/crio-0eb6eda88cb9ef2b073521d7e1de2eba9fafa7cc718b9608e7d61a6d1d7429b1 WatchSource:0}: Error finding container 0eb6eda88cb9ef2b073521d7e1de2eba9fafa7cc718b9608e7d61a6d1d7429b1: Status 404 returned error can't find the container with id 0eb6eda88cb9ef2b073521d7e1de2eba9fafa7cc718b9608e7d61a6d1d7429b1 Oct 31 07:50:52.311847 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:52.311814 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-03ede73268/error-404-isvc-e6a49-predictor-56df4bc975-jd92x" event={"ID":"57190b16-3213-44a0-8dab-e2fa9e22ef88","Type":"ContainerStarted","Data":"0fc61928f2b46064fce4b263b7479fe75511d4646a2c7d2b1bd812face7b5f6c"} Oct 31 07:50:52.312338 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:52.311873 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-03ede73268/error-404-isvc-e6a49-predictor-56df4bc975-jd92x" event={"ID":"57190b16-3213-44a0-8dab-e2fa9e22ef88","Type":"ContainerStarted","Data":"df02866b59d808e70644a1240202924f00a870c0f72b3b4a48e89a2111a086f3"} Oct 31 07:50:52.312338 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:52.311905 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-03ede73268/error-404-isvc-e6a49-predictor-56df4bc975-jd92x" event={"ID":"57190b16-3213-44a0-8dab-e2fa9e22ef88","Type":"ContainerStarted","Data":"0eb6eda88cb9ef2b073521d7e1de2eba9fafa7cc718b9608e7d61a6d1d7429b1"} Oct 31 07:50:52.312338 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:52.311942 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-03ede73268/error-404-isvc-e6a49-predictor-56df4bc975-jd92x" Oct 31 07:50:52.312338 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:52.311972 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-03ede73268/error-404-isvc-e6a49-predictor-56df4bc975-jd92x" Oct 31 07:50:52.313205 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:52.313180 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/error-404-isvc-e6a49-predictor-56df4bc975-jd92x" podUID="57190b16-3213-44a0-8dab-e2fa9e22ef88" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused" Oct 31 07:50:52.332564 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:52.332518 4935 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-03ede73268/error-404-isvc-e6a49-predictor-56df4bc975-jd92x" podStartSLOduration=2.332505789 podStartE2EDuration="2.332505789s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:50:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:50:52.330715823 +0000 UTC m=+1226.201247361" watchObservedRunningTime="2026-10-31 07:50:52.332505789 +0000 UTC m=+1226.203037325" Oct 31 07:50:53.315787 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:53.315746 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/error-404-isvc-e6a49-predictor-56df4bc975-jd92x" podUID="57190b16-3213-44a0-8dab-e2fa9e22ef88" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused" Oct 31 07:50:58.320078 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:58.320051 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-03ede73268/error-404-isvc-e6a49-predictor-56df4bc975-jd92x" Oct 31 07:50:58.320533 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:50:58.320426 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/error-404-isvc-e6a49-predictor-56df4bc975-jd92x" podUID="57190b16-3213-44a0-8dab-e2fa9e22ef88" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused" Oct 31 07:51:01.454801 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:51:01.454759 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 07:51:08.320517 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:08.320480 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/error-404-isvc-e6a49-predictor-56df4bc975-jd92x" podUID="57190b16-3213-44a0-8dab-e2fa9e22ef88" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused" Oct 31 07:51:13.343981 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:13.343946 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:51:17.438809 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:17.438775 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:51:18.321288 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:18.321253 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/error-404-isvc-e6a49-predictor-56df4bc975-jd92x" podUID="57190b16-3213-44a0-8dab-e2fa9e22ef88" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused" Oct 31 07:51:24.087268 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:24.087231 4935 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-5c1f590758/error-404-isvc-48632-predictor-7c7bc8bf7d-xj4tv"] Oct 31 07:51:24.087766 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:24.087570 4935 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-5c1f590758/error-404-isvc-48632-predictor-7c7bc8bf7d-xj4tv" podUID="8cadcae9-d852-4529-9246-439f7f4ff955" containerName="kserve-container" containerID="cri-o://462e361112dd9076bd58225686be395f94a510621a15679f39047bc50cb129da" gracePeriod=30 Oct 31 07:51:24.087766 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:24.087639 4935 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-5c1f590758/error-404-isvc-48632-predictor-7c7bc8bf7d-xj4tv" podUID="8cadcae9-d852-4529-9246-439f7f4ff955" containerName="kube-rbac-proxy" containerID="cri-o://255874a6dfab10f08a7e37a89b2f7777841e07d07e801ca222fe0d2e8505fe9c" gracePeriod=30 Oct 31 07:51:24.233828 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:24.233775 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:51:24.433020 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:24.432924 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:51:24.463548 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:24.463519 4935 generic.go:356] "Generic (PLEG): container finished" podID="8cadcae9-d852-4529-9246-439f7f4ff955" containerID="255874a6dfab10f08a7e37a89b2f7777841e07d07e801ca222fe0d2e8505fe9c" exitCode=0 Oct 31 07:51:24.463711 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:24.463589 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-5c1f590758/error-404-isvc-48632-predictor-7c7bc8bf7d-xj4tv" event={"ID":"8cadcae9-d852-4529-9246-439f7f4ff955","Type":"ContainerDied","Data":"255874a6dfab10f08a7e37a89b2f7777841e07d07e801ca222fe0d2e8505fe9c"} Oct 31 07:51:25.096133 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:25.096085 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-5c1f590758/error-404-isvc-48632-predictor-7c7bc8bf7d-xj4tv" podUID="8cadcae9-d852-4529-9246-439f7f4ff955" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.33:8643/healthz\": dial tcp 10.132.0.33:8643: connect: connection refused" Oct 31 07:51:25.101076 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:25.101047 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-5c1f590758/error-404-isvc-48632-predictor-7c7bc8bf7d-xj4tv" podUID="8cadcae9-d852-4529-9246-439f7f4ff955" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.33:8080: connect: connection refused" Oct 31 07:51:27.141927 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:27.141898 4935 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-5c1f590758/error-404-isvc-48632-predictor-7c7bc8bf7d-xj4tv" Oct 31 07:51:27.232515 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:27.232488 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8cadcae9-d852-4529-9246-439f7f4ff955-proxy-tls\" (UniqueName: \"kubernetes.io/secret/8cadcae9-d852-4529-9246-439f7f4ff955-proxy-tls\") pod \"8cadcae9-d852-4529-9246-439f7f4ff955\" (UID: \"8cadcae9-d852-4529-9246-439f7f4ff955\") " Oct 31 07:51:27.232653 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:27.232528 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8cadcae9-d852-4529-9246-439f7f4ff955-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8cadcae9-d852-4529-9246-439f7f4ff955-kube-rbac-proxy-sar-config\") pod \"8cadcae9-d852-4529-9246-439f7f4ff955\" (UID: \"8cadcae9-d852-4529-9246-439f7f4ff955\") " Oct 31 07:51:27.232701 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:27.232647 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8cadcae9-d852-4529-9246-439f7f4ff955-kube-api-access-kvkzc\" (UniqueName: \"kubernetes.io/projected/8cadcae9-d852-4529-9246-439f7f4ff955-kube-api-access-kvkzc\") pod \"8cadcae9-d852-4529-9246-439f7f4ff955\" (UID: \"8cadcae9-d852-4529-9246-439f7f4ff955\") " Oct 31 07:51:27.232904 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:27.232868 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8cadcae9-d852-4529-9246-439f7f4ff955-kube-rbac-proxy-sar-config" pod "8cadcae9-d852-4529-9246-439f7f4ff955" (UID: "8cadcae9-d852-4529-9246-439f7f4ff955"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 31 07:51:27.234721 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:27.234689 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cadcae9-d852-4529-9246-439f7f4ff955-kube-api-access-kvkzc" pod "8cadcae9-d852-4529-9246-439f7f4ff955" (UID: "8cadcae9-d852-4529-9246-439f7f4ff955"). InnerVolumeSpecName "kube-api-access-kvkzc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 31 07:51:27.234806 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:27.234763 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8cadcae9-d852-4529-9246-439f7f4ff955-proxy-tls" pod "8cadcae9-d852-4529-9246-439f7f4ff955" (UID: "8cadcae9-d852-4529-9246-439f7f4ff955"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 31 07:51:27.333907 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:27.333885 4935 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8cadcae9-d852-4529-9246-439f7f4ff955-proxy-tls\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 07:51:27.334009 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:27.333912 4935 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8cadcae9-d852-4529-9246-439f7f4ff955-kube-rbac-proxy-sar-config\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 07:51:27.334009 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:27.333926 4935 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kvkzc\" (UniqueName: \"kubernetes.io/projected/8cadcae9-d852-4529-9246-439f7f4ff955-kube-api-access-kvkzc\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 07:51:27.477333 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:27.477296 4935 generic.go:356] "Generic (PLEG): container finished" podID="8cadcae9-d852-4529-9246-439f7f4ff955" containerID="462e361112dd9076bd58225686be395f94a510621a15679f39047bc50cb129da" exitCode=0 Oct 31 07:51:27.477520 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:27.477366 4935 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-5c1f590758/error-404-isvc-48632-predictor-7c7bc8bf7d-xj4tv" Oct 31 07:51:27.477520 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:27.477387 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-5c1f590758/error-404-isvc-48632-predictor-7c7bc8bf7d-xj4tv" event={"ID":"8cadcae9-d852-4529-9246-439f7f4ff955","Type":"ContainerDied","Data":"462e361112dd9076bd58225686be395f94a510621a15679f39047bc50cb129da"} Oct 31 07:51:27.477520 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:27.477449 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-5c1f590758/error-404-isvc-48632-predictor-7c7bc8bf7d-xj4tv" event={"ID":"8cadcae9-d852-4529-9246-439f7f4ff955","Type":"ContainerDied","Data":"6c5b82e5f3a353460d16bf2c4bbc804e5aa3a94c5bf3feba7b8c66d1907456cf"} Oct 31 07:51:27.477656 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:27.477537 4935 scope.go:122] "RemoveContainer" containerID="255874a6dfab10f08a7e37a89b2f7777841e07d07e801ca222fe0d2e8505fe9c" Oct 31 07:51:27.486671 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:27.486655 4935 scope.go:122] "RemoveContainer" containerID="462e361112dd9076bd58225686be395f94a510621a15679f39047bc50cb129da" Oct 31 07:51:27.494956 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:27.494940 4935 scope.go:122] "RemoveContainer" containerID="255874a6dfab10f08a7e37a89b2f7777841e07d07e801ca222fe0d2e8505fe9c" Oct 31 07:51:27.495207 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:51:27.495191 4935 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"255874a6dfab10f08a7e37a89b2f7777841e07d07e801ca222fe0d2e8505fe9c\": container with ID starting with 255874a6dfab10f08a7e37a89b2f7777841e07d07e801ca222fe0d2e8505fe9c not found: ID does not exist" containerID="255874a6dfab10f08a7e37a89b2f7777841e07d07e801ca222fe0d2e8505fe9c" Oct 31 07:51:27.495265 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:27.495215 4935 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"255874a6dfab10f08a7e37a89b2f7777841e07d07e801ca222fe0d2e8505fe9c"} err="failed to get container status \"255874a6dfab10f08a7e37a89b2f7777841e07d07e801ca222fe0d2e8505fe9c\": rpc error: code = NotFound desc = could not find container \"255874a6dfab10f08a7e37a89b2f7777841e07d07e801ca222fe0d2e8505fe9c\": container with ID starting with 255874a6dfab10f08a7e37a89b2f7777841e07d07e801ca222fe0d2e8505fe9c not found: ID does not exist" Oct 31 07:51:27.495265 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:27.495229 4935 scope.go:122] "RemoveContainer" containerID="462e361112dd9076bd58225686be395f94a510621a15679f39047bc50cb129da" Oct 31 07:51:27.495496 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:51:27.495477 4935 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"462e361112dd9076bd58225686be395f94a510621a15679f39047bc50cb129da\": container with ID starting with 462e361112dd9076bd58225686be395f94a510621a15679f39047bc50cb129da not found: ID does not exist" containerID="462e361112dd9076bd58225686be395f94a510621a15679f39047bc50cb129da" Oct 31 07:51:27.495548 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:27.495499 4935 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"462e361112dd9076bd58225686be395f94a510621a15679f39047bc50cb129da"} err="failed to get container status \"462e361112dd9076bd58225686be395f94a510621a15679f39047bc50cb129da\": rpc error: code = NotFound desc = could not find container \"462e361112dd9076bd58225686be395f94a510621a15679f39047bc50cb129da\": container with ID starting with 462e361112dd9076bd58225686be395f94a510621a15679f39047bc50cb129da not found: ID does not exist" Oct 31 07:51:27.498792 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:27.498772 4935 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-5c1f590758/error-404-isvc-48632-predictor-7c7bc8bf7d-xj4tv"] Oct 31 07:51:27.502554 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:27.502536 4935 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-5c1f590758/error-404-isvc-48632-predictor-7c7bc8bf7d-xj4tv"] Oct 31 07:51:28.242884 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:28.242848 4935 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8cadcae9-d852-4529-9246-439f7f4ff955" path="/var/lib/kubelet/pods/8cadcae9-d852-4529-9246-439f7f4ff955/volumes" Oct 31 07:51:28.321009 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:28.320981 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/error-404-isvc-e6a49-predictor-56df4bc975-jd92x" podUID="57190b16-3213-44a0-8dab-e2fa9e22ef88" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused" Oct 31 07:51:29.609049 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:29.609013 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:51:29.709195 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:29.709160 4935 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-5c1f590758/error-404-isvc-b7c3b-predictor-54b5879974-6vrkz"] Oct 31 07:51:29.709627 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:29.709610 4935 cpu_manager.go:414] "containerMap: removing container" podUID="8cadcae9-d852-4529-9246-439f7f4ff955" containerName="kube-rbac-proxy" Oct 31 07:51:29.709734 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:29.709629 4935 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8cadcae9-d852-4529-9246-439f7f4ff955" containerName="kube-rbac-proxy" Oct 31 07:51:29.709734 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:29.709646 4935 cpu_manager.go:414] "containerMap: removing container" podUID="8cadcae9-d852-4529-9246-439f7f4ff955" containerName="kserve-container" Oct 31 07:51:29.709734 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:29.709654 4935 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8cadcae9-d852-4529-9246-439f7f4ff955" containerName="kserve-container" Oct 31 07:51:29.709734 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:29.709725 4935 memory_manager.go:365] "RemoveStaleState removing state" podUID="8cadcae9-d852-4529-9246-439f7f4ff955" containerName="kserve-container" Oct 31 07:51:29.709936 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:29.709744 4935 memory_manager.go:365] "RemoveStaleState removing state" podUID="8cadcae9-d852-4529-9246-439f7f4ff955" containerName="kube-rbac-proxy" Oct 31 07:51:29.713984 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:29.713964 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-5c1f590758/error-404-isvc-b7c3b-predictor-54b5879974-6vrkz" Oct 31 07:51:29.722584 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:29.722563 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-5c1f590758\"/\"error-404-isvc-b7c3b-kube-rbac-proxy-sar-config\"" Oct 31 07:51:29.722757 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:29.722562 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-5c1f590758\"/\"kube-root-ca.crt\"" Oct 31 07:51:29.722757 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:29.722591 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-5c1f590758\"/\"error-404-isvc-b7c3b-predictor-serving-cert\"" Oct 31 07:51:29.724388 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:29.724370 4935 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-5c1f590758/error-404-isvc-b7c3b-predictor-54b5879974-6vrkz"] Oct 31 07:51:29.811446 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:29.811422 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:51:29.854524 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:29.854502 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fv69v\" (UniqueName: \"kubernetes.io/projected/ae10d26f-85c6-4f6c-a66f-b61903bedeef-kube-api-access-fv69v\") pod \"error-404-isvc-b7c3b-predictor-54b5879974-6vrkz\" (UID: \"ae10d26f-85c6-4f6c-a66f-b61903bedeef\") " pod="e2e-gw0-5c1f590758/error-404-isvc-b7c3b-predictor-54b5879974-6vrkz" Oct 31 07:51:29.854611 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:29.854549 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ae10d26f-85c6-4f6c-a66f-b61903bedeef-proxy-tls\") pod \"error-404-isvc-b7c3b-predictor-54b5879974-6vrkz\" (UID: \"ae10d26f-85c6-4f6c-a66f-b61903bedeef\") " pod="e2e-gw0-5c1f590758/error-404-isvc-b7c3b-predictor-54b5879974-6vrkz" Oct 31 07:51:29.854679 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:29.854665 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ae10d26f-85c6-4f6c-a66f-b61903bedeef-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-b7c3b-predictor-54b5879974-6vrkz\" (UID: \"ae10d26f-85c6-4f6c-a66f-b61903bedeef\") " pod="e2e-gw0-5c1f590758/error-404-isvc-b7c3b-predictor-54b5879974-6vrkz" Oct 31 07:51:29.955595 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:29.955575 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ae10d26f-85c6-4f6c-a66f-b61903bedeef-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-b7c3b-predictor-54b5879974-6vrkz\" (UID: \"ae10d26f-85c6-4f6c-a66f-b61903bedeef\") " pod="e2e-gw0-5c1f590758/error-404-isvc-b7c3b-predictor-54b5879974-6vrkz" Oct 31 07:51:29.955703 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:29.955610 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fv69v\" (UniqueName: \"kubernetes.io/projected/ae10d26f-85c6-4f6c-a66f-b61903bedeef-kube-api-access-fv69v\") pod \"error-404-isvc-b7c3b-predictor-54b5879974-6vrkz\" (UID: \"ae10d26f-85c6-4f6c-a66f-b61903bedeef\") " pod="e2e-gw0-5c1f590758/error-404-isvc-b7c3b-predictor-54b5879974-6vrkz" Oct 31 07:51:29.955749 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:29.955734 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ae10d26f-85c6-4f6c-a66f-b61903bedeef-proxy-tls\") pod \"error-404-isvc-b7c3b-predictor-54b5879974-6vrkz\" (UID: \"ae10d26f-85c6-4f6c-a66f-b61903bedeef\") " pod="e2e-gw0-5c1f590758/error-404-isvc-b7c3b-predictor-54b5879974-6vrkz" Oct 31 07:51:29.956196 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:29.956177 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ae10d26f-85c6-4f6c-a66f-b61903bedeef-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-b7c3b-predictor-54b5879974-6vrkz\" (UID: \"ae10d26f-85c6-4f6c-a66f-b61903bedeef\") " pod="e2e-gw0-5c1f590758/error-404-isvc-b7c3b-predictor-54b5879974-6vrkz" Oct 31 07:51:29.958209 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:29.958189 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ae10d26f-85c6-4f6c-a66f-b61903bedeef-proxy-tls\") pod \"error-404-isvc-b7c3b-predictor-54b5879974-6vrkz\" (UID: \"ae10d26f-85c6-4f6c-a66f-b61903bedeef\") " pod="e2e-gw0-5c1f590758/error-404-isvc-b7c3b-predictor-54b5879974-6vrkz" Oct 31 07:51:29.967480 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:29.967456 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fv69v\" (UniqueName: \"kubernetes.io/projected/ae10d26f-85c6-4f6c-a66f-b61903bedeef-kube-api-access-fv69v\") pod \"error-404-isvc-b7c3b-predictor-54b5879974-6vrkz\" (UID: \"ae10d26f-85c6-4f6c-a66f-b61903bedeef\") " pod="e2e-gw0-5c1f590758/error-404-isvc-b7c3b-predictor-54b5879974-6vrkz" Oct 31 07:51:30.011829 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:30.011804 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:51:30.029457 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:30.029437 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-5c1f590758/error-404-isvc-b7c3b-predictor-54b5879974-6vrkz" Oct 31 07:51:30.309178 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:30.309010 4935 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-5c1f590758/error-404-isvc-b7c3b-predictor-54b5879974-6vrkz"] Oct 31 07:51:30.311827 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:51:30.311804 4935 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae10d26f_85c6_4f6c_a66f_b61903bedeef.slice/crio-8e36760d2d32769cf432f851feffd8a43db48d1fbae674c0ccf22ee6b1b59f9f WatchSource:0}: Error finding container 8e36760d2d32769cf432f851feffd8a43db48d1fbae674c0ccf22ee6b1b59f9f: Status 404 returned error can't find the container with id 8e36760d2d32769cf432f851feffd8a43db48d1fbae674c0ccf22ee6b1b59f9f Oct 31 07:51:30.494596 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:30.494558 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-5c1f590758/error-404-isvc-b7c3b-predictor-54b5879974-6vrkz" event={"ID":"ae10d26f-85c6-4f6c-a66f-b61903bedeef","Type":"ContainerStarted","Data":"8555ffaf3462aed2fd44b8e1cd1f4591618c0b95dc9ec4775992a90db25a97c8"} Oct 31 07:51:30.494821 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:30.494780 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-5c1f590758/error-404-isvc-b7c3b-predictor-54b5879974-6vrkz" Oct 31 07:51:30.494882 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:30.494838 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-5c1f590758/error-404-isvc-b7c3b-predictor-54b5879974-6vrkz" event={"ID":"ae10d26f-85c6-4f6c-a66f-b61903bedeef","Type":"ContainerStarted","Data":"2936795bdf9a43dcc7130638879c545199389af5a3244e36c853221663371fe8"} Oct 31 07:51:30.494917 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:30.494881 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-5c1f590758/error-404-isvc-b7c3b-predictor-54b5879974-6vrkz" event={"ID":"ae10d26f-85c6-4f6c-a66f-b61903bedeef","Type":"ContainerStarted","Data":"8e36760d2d32769cf432f851feffd8a43db48d1fbae674c0ccf22ee6b1b59f9f"} Oct 31 07:51:30.513466 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:30.513418 4935 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-5c1f590758/error-404-isvc-b7c3b-predictor-54b5879974-6vrkz" podStartSLOduration=1.513399787 podStartE2EDuration="1.513399787s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:51:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:51:30.512909786 +0000 UTC m=+1264.383441323" watchObservedRunningTime="2026-10-31 07:51:30.513399787 +0000 UTC m=+1264.383931324" Oct 31 07:51:31.454580 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:51:31.454544 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 07:51:31.498503 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:31.498467 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-5c1f590758/error-404-isvc-b7c3b-predictor-54b5879974-6vrkz" Oct 31 07:51:31.499555 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:31.499531 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-5c1f590758/error-404-isvc-b7c3b-predictor-54b5879974-6vrkz" podUID="ae10d26f-85c6-4f6c-a66f-b61903bedeef" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused" Oct 31 07:51:32.502215 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:32.502172 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-5c1f590758/error-404-isvc-b7c3b-predictor-54b5879974-6vrkz" podUID="ae10d26f-85c6-4f6c-a66f-b61903bedeef" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused" Oct 31 07:51:37.507314 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:37.507278 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-5c1f590758/error-404-isvc-b7c3b-predictor-54b5879974-6vrkz" Oct 31 07:51:37.507848 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:37.507824 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-5c1f590758/error-404-isvc-b7c3b-predictor-54b5879974-6vrkz" podUID="ae10d26f-85c6-4f6c-a66f-b61903bedeef" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused" Oct 31 07:51:38.321300 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:38.321263 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/error-404-isvc-e6a49-predictor-56df4bc975-jd92x" podUID="57190b16-3213-44a0-8dab-e2fa9e22ef88" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused" Oct 31 07:51:47.508227 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:47.508190 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-5c1f590758/error-404-isvc-b7c3b-predictor-54b5879974-6vrkz" podUID="ae10d26f-85c6-4f6c-a66f-b61903bedeef" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused" Oct 31 07:51:48.321247 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:48.321219 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-03ede73268/error-404-isvc-e6a49-predictor-56df4bc975-jd92x" Oct 31 07:51:57.508649 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:51:57.508613 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-5c1f590758/error-404-isvc-b7c3b-predictor-54b5879974-6vrkz" podUID="ae10d26f-85c6-4f6c-a66f-b61903bedeef" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused" Oct 31 07:52:01.075155 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:01.075125 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:52:01.276178 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:01.276144 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:52:01.455248 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:52:01.455208 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 07:52:07.508241 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:07.508164 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-5c1f590758/error-404-isvc-b7c3b-predictor-54b5879974-6vrkz" podUID="ae10d26f-85c6-4f6c-a66f-b61903bedeef" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused" Oct 31 07:52:11.763047 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:11.763013 4935 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-03ede73268/error-404-isvc-e6a49-predictor-56df4bc975-jd92x"] Oct 31 07:52:11.763469 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:11.763242 4935 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-03ede73268/error-404-isvc-e6a49-predictor-56df4bc975-jd92x" podUID="57190b16-3213-44a0-8dab-e2fa9e22ef88" containerName="kserve-container" containerID="cri-o://df02866b59d808e70644a1240202924f00a870c0f72b3b4a48e89a2111a086f3" gracePeriod=30 Oct 31 07:52:11.763469 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:11.763314 4935 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-03ede73268/error-404-isvc-e6a49-predictor-56df4bc975-jd92x" podUID="57190b16-3213-44a0-8dab-e2fa9e22ef88" containerName="kube-rbac-proxy" containerID="cri-o://0fc61928f2b46064fce4b263b7479fe75511d4646a2c7d2b1bd812face7b5f6c" gracePeriod=30 Oct 31 07:52:11.918214 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:11.918183 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:52:12.120517 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:12.120439 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:52:12.671732 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:12.671698 4935 generic.go:356] "Generic (PLEG): container finished" podID="57190b16-3213-44a0-8dab-e2fa9e22ef88" containerID="0fc61928f2b46064fce4b263b7479fe75511d4646a2c7d2b1bd812face7b5f6c" exitCode=0 Oct 31 07:52:12.671925 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:12.671776 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-03ede73268/error-404-isvc-e6a49-predictor-56df4bc975-jd92x" event={"ID":"57190b16-3213-44a0-8dab-e2fa9e22ef88","Type":"ContainerDied","Data":"0fc61928f2b46064fce4b263b7479fe75511d4646a2c7d2b1bd812face7b5f6c"} Oct 31 07:52:13.316435 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:13.316395 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/error-404-isvc-e6a49-predictor-56df4bc975-jd92x" podUID="57190b16-3213-44a0-8dab-e2fa9e22ef88" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.35:8643/healthz\": dial tcp 10.132.0.35:8643: connect: connection refused" Oct 31 07:52:15.013106 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:15.013086 4935 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-03ede73268/error-404-isvc-e6a49-predictor-56df4bc975-jd92x" Oct 31 07:52:15.108303 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:15.108229 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/57190b16-3213-44a0-8dab-e2fa9e22ef88-proxy-tls\" (UniqueName: \"kubernetes.io/secret/57190b16-3213-44a0-8dab-e2fa9e22ef88-proxy-tls\") pod \"57190b16-3213-44a0-8dab-e2fa9e22ef88\" (UID: \"57190b16-3213-44a0-8dab-e2fa9e22ef88\") " Oct 31 07:52:15.108478 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:15.108331 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/57190b16-3213-44a0-8dab-e2fa9e22ef88-kube-api-access-lctjz\" (UniqueName: \"kubernetes.io/projected/57190b16-3213-44a0-8dab-e2fa9e22ef88-kube-api-access-lctjz\") pod \"57190b16-3213-44a0-8dab-e2fa9e22ef88\" (UID: \"57190b16-3213-44a0-8dab-e2fa9e22ef88\") " Oct 31 07:52:15.108478 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:15.108398 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/57190b16-3213-44a0-8dab-e2fa9e22ef88-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/57190b16-3213-44a0-8dab-e2fa9e22ef88-kube-rbac-proxy-sar-config\") pod \"57190b16-3213-44a0-8dab-e2fa9e22ef88\" (UID: \"57190b16-3213-44a0-8dab-e2fa9e22ef88\") " Oct 31 07:52:15.108700 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:15.108677 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57190b16-3213-44a0-8dab-e2fa9e22ef88-kube-rbac-proxy-sar-config" pod "57190b16-3213-44a0-8dab-e2fa9e22ef88" (UID: "57190b16-3213-44a0-8dab-e2fa9e22ef88"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 31 07:52:15.110539 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:15.110517 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57190b16-3213-44a0-8dab-e2fa9e22ef88-kube-api-access-lctjz" pod "57190b16-3213-44a0-8dab-e2fa9e22ef88" (UID: "57190b16-3213-44a0-8dab-e2fa9e22ef88"). InnerVolumeSpecName "kube-api-access-lctjz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 31 07:52:15.110627 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:15.110566 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57190b16-3213-44a0-8dab-e2fa9e22ef88-proxy-tls" pod "57190b16-3213-44a0-8dab-e2fa9e22ef88" (UID: "57190b16-3213-44a0-8dab-e2fa9e22ef88"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 31 07:52:15.209045 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:15.209021 4935 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/57190b16-3213-44a0-8dab-e2fa9e22ef88-proxy-tls\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 07:52:15.209045 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:15.209046 4935 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lctjz\" (UniqueName: \"kubernetes.io/projected/57190b16-3213-44a0-8dab-e2fa9e22ef88-kube-api-access-lctjz\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 07:52:15.209198 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:15.209056 4935 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/57190b16-3213-44a0-8dab-e2fa9e22ef88-kube-rbac-proxy-sar-config\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 07:52:15.691944 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:15.691906 4935 generic.go:356] "Generic (PLEG): container finished" podID="57190b16-3213-44a0-8dab-e2fa9e22ef88" containerID="df02866b59d808e70644a1240202924f00a870c0f72b3b4a48e89a2111a086f3" exitCode=0 Oct 31 07:52:15.692154 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:15.691971 4935 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-03ede73268/error-404-isvc-e6a49-predictor-56df4bc975-jd92x" Oct 31 07:52:15.692154 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:15.691997 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-03ede73268/error-404-isvc-e6a49-predictor-56df4bc975-jd92x" event={"ID":"57190b16-3213-44a0-8dab-e2fa9e22ef88","Type":"ContainerDied","Data":"df02866b59d808e70644a1240202924f00a870c0f72b3b4a48e89a2111a086f3"} Oct 31 07:52:15.692154 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:15.692061 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-03ede73268/error-404-isvc-e6a49-predictor-56df4bc975-jd92x" event={"ID":"57190b16-3213-44a0-8dab-e2fa9e22ef88","Type":"ContainerDied","Data":"0eb6eda88cb9ef2b073521d7e1de2eba9fafa7cc718b9608e7d61a6d1d7429b1"} Oct 31 07:52:15.692154 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:15.692087 4935 scope.go:122] "RemoveContainer" containerID="0fc61928f2b46064fce4b263b7479fe75511d4646a2c7d2b1bd812face7b5f6c" Oct 31 07:52:15.701413 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:15.701396 4935 scope.go:122] "RemoveContainer" containerID="df02866b59d808e70644a1240202924f00a870c0f72b3b4a48e89a2111a086f3" Oct 31 07:52:15.709003 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:15.708983 4935 scope.go:122] "RemoveContainer" containerID="0fc61928f2b46064fce4b263b7479fe75511d4646a2c7d2b1bd812face7b5f6c" Oct 31 07:52:15.709245 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:52:15.709226 4935 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0fc61928f2b46064fce4b263b7479fe75511d4646a2c7d2b1bd812face7b5f6c\": container with ID starting with 0fc61928f2b46064fce4b263b7479fe75511d4646a2c7d2b1bd812face7b5f6c not found: ID does not exist" containerID="0fc61928f2b46064fce4b263b7479fe75511d4646a2c7d2b1bd812face7b5f6c" Oct 31 07:52:15.709299 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:15.709254 4935 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fc61928f2b46064fce4b263b7479fe75511d4646a2c7d2b1bd812face7b5f6c"} err="failed to get container status \"0fc61928f2b46064fce4b263b7479fe75511d4646a2c7d2b1bd812face7b5f6c\": rpc error: code = NotFound desc = could not find container \"0fc61928f2b46064fce4b263b7479fe75511d4646a2c7d2b1bd812face7b5f6c\": container with ID starting with 0fc61928f2b46064fce4b263b7479fe75511d4646a2c7d2b1bd812face7b5f6c not found: ID does not exist" Oct 31 07:52:15.709299 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:15.709269 4935 scope.go:122] "RemoveContainer" containerID="df02866b59d808e70644a1240202924f00a870c0f72b3b4a48e89a2111a086f3" Oct 31 07:52:15.709513 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:52:15.709495 4935 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df02866b59d808e70644a1240202924f00a870c0f72b3b4a48e89a2111a086f3\": container with ID starting with df02866b59d808e70644a1240202924f00a870c0f72b3b4a48e89a2111a086f3 not found: ID does not exist" containerID="df02866b59d808e70644a1240202924f00a870c0f72b3b4a48e89a2111a086f3" Oct 31 07:52:15.709567 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:15.709523 4935 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df02866b59d808e70644a1240202924f00a870c0f72b3b4a48e89a2111a086f3"} err="failed to get container status \"df02866b59d808e70644a1240202924f00a870c0f72b3b4a48e89a2111a086f3\": rpc error: code = NotFound desc = could not find container \"df02866b59d808e70644a1240202924f00a870c0f72b3b4a48e89a2111a086f3\": container with ID starting with df02866b59d808e70644a1240202924f00a870c0f72b3b4a48e89a2111a086f3 not found: ID does not exist" Oct 31 07:52:15.714071 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:15.714050 4935 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-03ede73268/error-404-isvc-e6a49-predictor-56df4bc975-jd92x"] Oct 31 07:52:15.718103 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:15.718083 4935 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-03ede73268/error-404-isvc-e6a49-predictor-56df4bc975-jd92x"] Oct 31 07:52:16.243336 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:16.243305 4935 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="57190b16-3213-44a0-8dab-e2fa9e22ef88" path="/var/lib/kubelet/pods/57190b16-3213-44a0-8dab-e2fa9e22ef88/volumes" Oct 31 07:52:17.338160 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:17.338130 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:52:17.400641 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:17.400613 4935 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-03ede73268/error-404-isvc-bc1fb-predictor-5f96d49d6f-c86vj"] Oct 31 07:52:17.400960 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:17.400947 4935 cpu_manager.go:414] "containerMap: removing container" podUID="57190b16-3213-44a0-8dab-e2fa9e22ef88" containerName="kube-rbac-proxy" Oct 31 07:52:17.401008 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:17.400961 4935 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="57190b16-3213-44a0-8dab-e2fa9e22ef88" containerName="kube-rbac-proxy" Oct 31 07:52:17.401008 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:17.400984 4935 cpu_manager.go:414] "containerMap: removing container" podUID="57190b16-3213-44a0-8dab-e2fa9e22ef88" containerName="kserve-container" Oct 31 07:52:17.401008 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:17.400989 4935 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="57190b16-3213-44a0-8dab-e2fa9e22ef88" containerName="kserve-container" Oct 31 07:52:17.401110 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:17.401042 4935 memory_manager.go:365] "RemoveStaleState removing state" podUID="57190b16-3213-44a0-8dab-e2fa9e22ef88" containerName="kube-rbac-proxy" Oct 31 07:52:17.401110 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:17.401051 4935 memory_manager.go:365] "RemoveStaleState removing state" podUID="57190b16-3213-44a0-8dab-e2fa9e22ef88" containerName="kserve-container" Oct 31 07:52:17.405445 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:17.405424 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-03ede73268/error-404-isvc-bc1fb-predictor-5f96d49d6f-c86vj" Oct 31 07:52:17.413174 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:17.413154 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-03ede73268\"/\"error-404-isvc-bc1fb-kube-rbac-proxy-sar-config\"" Oct 31 07:52:17.413174 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:17.413169 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-03ede73268\"/\"error-404-isvc-bc1fb-predictor-serving-cert\"" Oct 31 07:52:17.413315 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:17.413270 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-03ede73268\"/\"kube-root-ca.crt\"" Oct 31 07:52:17.415838 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:17.415819 4935 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-03ede73268/error-404-isvc-bc1fb-predictor-5f96d49d6f-c86vj"] Oct 31 07:52:17.509237 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:17.509213 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-5c1f590758/error-404-isvc-b7c3b-predictor-54b5879974-6vrkz" Oct 31 07:52:17.525207 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:17.525184 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/12d27978-9126-407f-87a4-dcd7866bfc90-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-bc1fb-predictor-5f96d49d6f-c86vj\" (UID: \"12d27978-9126-407f-87a4-dcd7866bfc90\") " pod="e2e-gw1-03ede73268/error-404-isvc-bc1fb-predictor-5f96d49d6f-c86vj" Oct 31 07:52:17.525311 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:17.525215 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/12d27978-9126-407f-87a4-dcd7866bfc90-proxy-tls\") pod \"error-404-isvc-bc1fb-predictor-5f96d49d6f-c86vj\" (UID: \"12d27978-9126-407f-87a4-dcd7866bfc90\") " pod="e2e-gw1-03ede73268/error-404-isvc-bc1fb-predictor-5f96d49d6f-c86vj" Oct 31 07:52:17.525381 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:17.525317 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5g7vg\" (UniqueName: \"kubernetes.io/projected/12d27978-9126-407f-87a4-dcd7866bfc90-kube-api-access-5g7vg\") pod \"error-404-isvc-bc1fb-predictor-5f96d49d6f-c86vj\" (UID: \"12d27978-9126-407f-87a4-dcd7866bfc90\") " pod="e2e-gw1-03ede73268/error-404-isvc-bc1fb-predictor-5f96d49d6f-c86vj" Oct 31 07:52:17.541641 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:17.541614 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:52:17.626000 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:17.625917 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5g7vg\" (UniqueName: \"kubernetes.io/projected/12d27978-9126-407f-87a4-dcd7866bfc90-kube-api-access-5g7vg\") pod \"error-404-isvc-bc1fb-predictor-5f96d49d6f-c86vj\" (UID: \"12d27978-9126-407f-87a4-dcd7866bfc90\") " pod="e2e-gw1-03ede73268/error-404-isvc-bc1fb-predictor-5f96d49d6f-c86vj" Oct 31 07:52:17.626000 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:17.625980 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/12d27978-9126-407f-87a4-dcd7866bfc90-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-bc1fb-predictor-5f96d49d6f-c86vj\" (UID: \"12d27978-9126-407f-87a4-dcd7866bfc90\") " pod="e2e-gw1-03ede73268/error-404-isvc-bc1fb-predictor-5f96d49d6f-c86vj" Oct 31 07:52:17.626249 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:17.626013 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/12d27978-9126-407f-87a4-dcd7866bfc90-proxy-tls\") pod \"error-404-isvc-bc1fb-predictor-5f96d49d6f-c86vj\" (UID: \"12d27978-9126-407f-87a4-dcd7866bfc90\") " pod="e2e-gw1-03ede73268/error-404-isvc-bc1fb-predictor-5f96d49d6f-c86vj" Oct 31 07:52:17.626628 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:17.626601 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/12d27978-9126-407f-87a4-dcd7866bfc90-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-bc1fb-predictor-5f96d49d6f-c86vj\" (UID: \"12d27978-9126-407f-87a4-dcd7866bfc90\") " pod="e2e-gw1-03ede73268/error-404-isvc-bc1fb-predictor-5f96d49d6f-c86vj" Oct 31 07:52:17.628465 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:17.628447 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/12d27978-9126-407f-87a4-dcd7866bfc90-proxy-tls\") pod \"error-404-isvc-bc1fb-predictor-5f96d49d6f-c86vj\" (UID: \"12d27978-9126-407f-87a4-dcd7866bfc90\") " pod="e2e-gw1-03ede73268/error-404-isvc-bc1fb-predictor-5f96d49d6f-c86vj" Oct 31 07:52:17.638027 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:17.638000 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5g7vg\" (UniqueName: \"kubernetes.io/projected/12d27978-9126-407f-87a4-dcd7866bfc90-kube-api-access-5g7vg\") pod \"error-404-isvc-bc1fb-predictor-5f96d49d6f-c86vj\" (UID: \"12d27978-9126-407f-87a4-dcd7866bfc90\") " pod="e2e-gw1-03ede73268/error-404-isvc-bc1fb-predictor-5f96d49d6f-c86vj" Oct 31 07:52:17.721090 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:17.721059 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-03ede73268/error-404-isvc-bc1fb-predictor-5f96d49d6f-c86vj" Oct 31 07:52:17.743644 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:17.743609 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:52:17.995094 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:17.995070 4935 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-03ede73268/error-404-isvc-bc1fb-predictor-5f96d49d6f-c86vj"] Oct 31 07:52:17.997808 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:52:17.997767 4935 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12d27978_9126_407f_87a4_dcd7866bfc90.slice/crio-727bc4c3bc951d526305fa828cc037e9bd724460575c6873c1de13ca40db0393 WatchSource:0}: Error finding container 727bc4c3bc951d526305fa828cc037e9bd724460575c6873c1de13ca40db0393: Status 404 returned error can't find the container with id 727bc4c3bc951d526305fa828cc037e9bd724460575c6873c1de13ca40db0393 Oct 31 07:52:18.706443 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:18.706407 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-03ede73268/error-404-isvc-bc1fb-predictor-5f96d49d6f-c86vj" event={"ID":"12d27978-9126-407f-87a4-dcd7866bfc90","Type":"ContainerStarted","Data":"eaf6943a6c985d25c424292d84249a52b99fc10d1726c16931cc39ccf9e661ef"} Oct 31 07:52:18.706892 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:18.706478 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-03ede73268/error-404-isvc-bc1fb-predictor-5f96d49d6f-c86vj" event={"ID":"12d27978-9126-407f-87a4-dcd7866bfc90","Type":"ContainerStarted","Data":"74e3b5b9272888051a0ad33c6eb2475c44ed45091306bf1b263886cca6e13ede"} Oct 31 07:52:18.706892 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:18.706510 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-03ede73268/error-404-isvc-bc1fb-predictor-5f96d49d6f-c86vj" event={"ID":"12d27978-9126-407f-87a4-dcd7866bfc90","Type":"ContainerStarted","Data":"727bc4c3bc951d526305fa828cc037e9bd724460575c6873c1de13ca40db0393"} Oct 31 07:52:18.706892 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:18.706541 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-03ede73268/error-404-isvc-bc1fb-predictor-5f96d49d6f-c86vj" Oct 31 07:52:18.706892 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:18.706570 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-03ede73268/error-404-isvc-bc1fb-predictor-5f96d49d6f-c86vj" Oct 31 07:52:18.708127 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:18.708103 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/error-404-isvc-bc1fb-predictor-5f96d49d6f-c86vj" podUID="12d27978-9126-407f-87a4-dcd7866bfc90" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused" Oct 31 07:52:18.727119 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:18.727075 4935 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-03ede73268/error-404-isvc-bc1fb-predictor-5f96d49d6f-c86vj" podStartSLOduration=1.72706252 podStartE2EDuration="1.72706252s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:52:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:52:18.725325404 +0000 UTC m=+1312.595856940" watchObservedRunningTime="2026-10-31 07:52:18.72706252 +0000 UTC m=+1312.597594057" Oct 31 07:52:19.710855 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:19.710820 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/error-404-isvc-bc1fb-predictor-5f96d49d6f-c86vj" podUID="12d27978-9126-407f-87a4-dcd7866bfc90" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused" Oct 31 07:52:24.715900 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:24.715866 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-03ede73268/error-404-isvc-bc1fb-predictor-5f96d49d6f-c86vj" Oct 31 07:52:24.716283 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:24.716263 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/error-404-isvc-bc1fb-predictor-5f96d49d6f-c86vj" podUID="12d27978-9126-407f-87a4-dcd7866bfc90" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused" Oct 31 07:52:31.455085 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:52:31.455047 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 07:52:34.717104 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:34.717069 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/error-404-isvc-bc1fb-predictor-5f96d49d6f-c86vj" podUID="12d27978-9126-407f-87a4-dcd7866bfc90" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused" Oct 31 07:52:39.759260 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:39.759221 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:52:39.962598 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:39.962568 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:52:44.716286 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:44.716250 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/error-404-isvc-bc1fb-predictor-5f96d49d6f-c86vj" podUID="12d27978-9126-407f-87a4-dcd7866bfc90" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused" Oct 31 07:52:50.871631 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:50.871601 4935 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-5c1f590758/error-404-isvc-b7c3b-predictor-54b5879974-6vrkz"] Oct 31 07:52:50.872072 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:50.871866 4935 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-5c1f590758/error-404-isvc-b7c3b-predictor-54b5879974-6vrkz" podUID="ae10d26f-85c6-4f6c-a66f-b61903bedeef" containerName="kserve-container" containerID="cri-o://2936795bdf9a43dcc7130638879c545199389af5a3244e36c853221663371fe8" gracePeriod=30 Oct 31 07:52:50.872072 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:50.871916 4935 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-5c1f590758/error-404-isvc-b7c3b-predictor-54b5879974-6vrkz" podUID="ae10d26f-85c6-4f6c-a66f-b61903bedeef" containerName="kube-rbac-proxy" containerID="cri-o://8555ffaf3462aed2fd44b8e1cd1f4591618c0b95dc9ec4775992a90db25a97c8" gracePeriod=30 Oct 31 07:52:51.024300 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:51.024261 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:52:51.225138 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:51.225095 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:52:51.850422 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:51.850391 4935 generic.go:356] "Generic (PLEG): container finished" podID="ae10d26f-85c6-4f6c-a66f-b61903bedeef" containerID="8555ffaf3462aed2fd44b8e1cd1f4591618c0b95dc9ec4775992a90db25a97c8" exitCode=0 Oct 31 07:52:51.850605 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:51.850467 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-5c1f590758/error-404-isvc-b7c3b-predictor-54b5879974-6vrkz" event={"ID":"ae10d26f-85c6-4f6c-a66f-b61903bedeef","Type":"ContainerDied","Data":"8555ffaf3462aed2fd44b8e1cd1f4591618c0b95dc9ec4775992a90db25a97c8"} Oct 31 07:52:52.503338 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:52.503286 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-5c1f590758/error-404-isvc-b7c3b-predictor-54b5879974-6vrkz" podUID="ae10d26f-85c6-4f6c-a66f-b61903bedeef" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.36:8643/healthz\": dial tcp 10.132.0.36:8643: connect: connection refused" Oct 31 07:52:54.322957 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:54.322935 4935 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-5c1f590758/error-404-isvc-b7c3b-predictor-54b5879974-6vrkz" Oct 31 07:52:54.430829 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:54.430765 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ae10d26f-85c6-4f6c-a66f-b61903bedeef-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ae10d26f-85c6-4f6c-a66f-b61903bedeef-kube-rbac-proxy-sar-config\") pod \"ae10d26f-85c6-4f6c-a66f-b61903bedeef\" (UID: \"ae10d26f-85c6-4f6c-a66f-b61903bedeef\") " Oct 31 07:52:54.430829 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:54.430815 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ae10d26f-85c6-4f6c-a66f-b61903bedeef-kube-api-access-fv69v\" (UniqueName: \"kubernetes.io/projected/ae10d26f-85c6-4f6c-a66f-b61903bedeef-kube-api-access-fv69v\") pod \"ae10d26f-85c6-4f6c-a66f-b61903bedeef\" (UID: \"ae10d26f-85c6-4f6c-a66f-b61903bedeef\") " Oct 31 07:52:54.431009 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:54.430843 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ae10d26f-85c6-4f6c-a66f-b61903bedeef-proxy-tls\" (UniqueName: \"kubernetes.io/secret/ae10d26f-85c6-4f6c-a66f-b61903bedeef-proxy-tls\") pod \"ae10d26f-85c6-4f6c-a66f-b61903bedeef\" (UID: \"ae10d26f-85c6-4f6c-a66f-b61903bedeef\") " Oct 31 07:52:54.431115 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:54.431093 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae10d26f-85c6-4f6c-a66f-b61903bedeef-kube-rbac-proxy-sar-config" pod "ae10d26f-85c6-4f6c-a66f-b61903bedeef" (UID: "ae10d26f-85c6-4f6c-a66f-b61903bedeef"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 31 07:52:54.433024 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:54.432997 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae10d26f-85c6-4f6c-a66f-b61903bedeef-proxy-tls" pod "ae10d26f-85c6-4f6c-a66f-b61903bedeef" (UID: "ae10d26f-85c6-4f6c-a66f-b61903bedeef"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 31 07:52:54.433133 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:54.433076 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae10d26f-85c6-4f6c-a66f-b61903bedeef-kube-api-access-fv69v" pod "ae10d26f-85c6-4f6c-a66f-b61903bedeef" (UID: "ae10d26f-85c6-4f6c-a66f-b61903bedeef"). InnerVolumeSpecName "kube-api-access-fv69v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 31 07:52:54.532268 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:54.532238 4935 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ae10d26f-85c6-4f6c-a66f-b61903bedeef-kube-rbac-proxy-sar-config\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 07:52:54.532268 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:54.532263 4935 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fv69v\" (UniqueName: \"kubernetes.io/projected/ae10d26f-85c6-4f6c-a66f-b61903bedeef-kube-api-access-fv69v\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 07:52:54.532268 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:54.532273 4935 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ae10d26f-85c6-4f6c-a66f-b61903bedeef-proxy-tls\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 07:52:54.716773 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:54.716737 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/error-404-isvc-bc1fb-predictor-5f96d49d6f-c86vj" podUID="12d27978-9126-407f-87a4-dcd7866bfc90" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused" Oct 31 07:52:54.865092 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:54.865063 4935 generic.go:356] "Generic (PLEG): container finished" podID="ae10d26f-85c6-4f6c-a66f-b61903bedeef" containerID="2936795bdf9a43dcc7130638879c545199389af5a3244e36c853221663371fe8" exitCode=0 Oct 31 07:52:54.865253 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:54.865120 4935 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-5c1f590758/error-404-isvc-b7c3b-predictor-54b5879974-6vrkz" Oct 31 07:52:54.865253 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:54.865135 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-5c1f590758/error-404-isvc-b7c3b-predictor-54b5879974-6vrkz" event={"ID":"ae10d26f-85c6-4f6c-a66f-b61903bedeef","Type":"ContainerDied","Data":"2936795bdf9a43dcc7130638879c545199389af5a3244e36c853221663371fe8"} Oct 31 07:52:54.865253 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:54.865182 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-5c1f590758/error-404-isvc-b7c3b-predictor-54b5879974-6vrkz" event={"ID":"ae10d26f-85c6-4f6c-a66f-b61903bedeef","Type":"ContainerDied","Data":"8e36760d2d32769cf432f851feffd8a43db48d1fbae674c0ccf22ee6b1b59f9f"} Oct 31 07:52:54.865253 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:54.865206 4935 scope.go:122] "RemoveContainer" containerID="8555ffaf3462aed2fd44b8e1cd1f4591618c0b95dc9ec4775992a90db25a97c8" Oct 31 07:52:54.874060 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:54.874044 4935 scope.go:122] "RemoveContainer" containerID="2936795bdf9a43dcc7130638879c545199389af5a3244e36c853221663371fe8" Oct 31 07:52:54.881661 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:54.881643 4935 scope.go:122] "RemoveContainer" containerID="8555ffaf3462aed2fd44b8e1cd1f4591618c0b95dc9ec4775992a90db25a97c8" Oct 31 07:52:54.881922 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:52:54.881905 4935 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8555ffaf3462aed2fd44b8e1cd1f4591618c0b95dc9ec4775992a90db25a97c8\": container with ID starting with 8555ffaf3462aed2fd44b8e1cd1f4591618c0b95dc9ec4775992a90db25a97c8 not found: ID does not exist" containerID="8555ffaf3462aed2fd44b8e1cd1f4591618c0b95dc9ec4775992a90db25a97c8" Oct 31 07:52:54.881971 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:54.881932 4935 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8555ffaf3462aed2fd44b8e1cd1f4591618c0b95dc9ec4775992a90db25a97c8"} err="failed to get container status \"8555ffaf3462aed2fd44b8e1cd1f4591618c0b95dc9ec4775992a90db25a97c8\": rpc error: code = NotFound desc = could not find container \"8555ffaf3462aed2fd44b8e1cd1f4591618c0b95dc9ec4775992a90db25a97c8\": container with ID starting with 8555ffaf3462aed2fd44b8e1cd1f4591618c0b95dc9ec4775992a90db25a97c8 not found: ID does not exist" Oct 31 07:52:54.881971 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:54.881949 4935 scope.go:122] "RemoveContainer" containerID="2936795bdf9a43dcc7130638879c545199389af5a3244e36c853221663371fe8" Oct 31 07:52:54.882175 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:52:54.882158 4935 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2936795bdf9a43dcc7130638879c545199389af5a3244e36c853221663371fe8\": container with ID starting with 2936795bdf9a43dcc7130638879c545199389af5a3244e36c853221663371fe8 not found: ID does not exist" containerID="2936795bdf9a43dcc7130638879c545199389af5a3244e36c853221663371fe8" Oct 31 07:52:54.882218 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:54.882183 4935 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2936795bdf9a43dcc7130638879c545199389af5a3244e36c853221663371fe8"} err="failed to get container status \"2936795bdf9a43dcc7130638879c545199389af5a3244e36c853221663371fe8\": rpc error: code = NotFound desc = could not find container \"2936795bdf9a43dcc7130638879c545199389af5a3244e36c853221663371fe8\": container with ID starting with 2936795bdf9a43dcc7130638879c545199389af5a3244e36c853221663371fe8 not found: ID does not exist" Oct 31 07:52:54.886424 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:54.886404 4935 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-5c1f590758/error-404-isvc-b7c3b-predictor-54b5879974-6vrkz"] Oct 31 07:52:54.892016 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:54.891995 4935 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-5c1f590758/error-404-isvc-b7c3b-predictor-54b5879974-6vrkz"] Oct 31 07:52:56.243113 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:56.243085 4935 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ae10d26f-85c6-4f6c-a66f-b61903bedeef" path="/var/lib/kubelet/pods/ae10d26f-85c6-4f6c-a66f-b61903bedeef/volumes" Oct 31 07:52:56.423869 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:56.423836 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:52:56.528799 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:56.528723 4935 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-5c1f590758/error-404-isvc-45944-predictor-7f69cd48-45twp"] Oct 31 07:52:56.529075 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:56.529063 4935 cpu_manager.go:414] "containerMap: removing container" podUID="ae10d26f-85c6-4f6c-a66f-b61903bedeef" containerName="kube-rbac-proxy" Oct 31 07:52:56.529115 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:56.529089 4935 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ae10d26f-85c6-4f6c-a66f-b61903bedeef" containerName="kube-rbac-proxy" Oct 31 07:52:56.529115 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:56.529112 4935 cpu_manager.go:414] "containerMap: removing container" podUID="ae10d26f-85c6-4f6c-a66f-b61903bedeef" containerName="kserve-container" Oct 31 07:52:56.529177 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:56.529116 4935 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ae10d26f-85c6-4f6c-a66f-b61903bedeef" containerName="kserve-container" Oct 31 07:52:56.529177 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:56.529173 4935 memory_manager.go:365] "RemoveStaleState removing state" podUID="ae10d26f-85c6-4f6c-a66f-b61903bedeef" containerName="kube-rbac-proxy" Oct 31 07:52:56.529244 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:56.529184 4935 memory_manager.go:365] "RemoveStaleState removing state" podUID="ae10d26f-85c6-4f6c-a66f-b61903bedeef" containerName="kserve-container" Oct 31 07:52:56.533578 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:56.533562 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-5c1f590758/error-404-isvc-45944-predictor-7f69cd48-45twp" Oct 31 07:52:56.542801 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:56.542760 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-5c1f590758\"/\"error-404-isvc-45944-predictor-serving-cert\"" Oct 31 07:52:56.542801 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:56.542764 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-5c1f590758\"/\"kube-root-ca.crt\"" Oct 31 07:52:56.542801 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:56.542792 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-5c1f590758\"/\"error-404-isvc-45944-kube-rbac-proxy-sar-config\"" Oct 31 07:52:56.544197 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:56.544177 4935 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-5c1f590758/error-404-isvc-45944-predictor-7f69cd48-45twp"] Oct 31 07:52:56.629196 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:56.629169 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:52:56.650986 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:56.650965 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d42c1eae-03e7-4ca5-9c85-70a0574a1dea-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-45944-predictor-7f69cd48-45twp\" (UID: \"d42c1eae-03e7-4ca5-9c85-70a0574a1dea\") " pod="e2e-gw0-5c1f590758/error-404-isvc-45944-predictor-7f69cd48-45twp" Oct 31 07:52:56.651116 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:56.651000 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d42c1eae-03e7-4ca5-9c85-70a0574a1dea-proxy-tls\") pod \"error-404-isvc-45944-predictor-7f69cd48-45twp\" (UID: \"d42c1eae-03e7-4ca5-9c85-70a0574a1dea\") " pod="e2e-gw0-5c1f590758/error-404-isvc-45944-predictor-7f69cd48-45twp" Oct 31 07:52:56.651116 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:56.651098 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2kjb\" (UniqueName: \"kubernetes.io/projected/d42c1eae-03e7-4ca5-9c85-70a0574a1dea-kube-api-access-l2kjb\") pod \"error-404-isvc-45944-predictor-7f69cd48-45twp\" (UID: \"d42c1eae-03e7-4ca5-9c85-70a0574a1dea\") " pod="e2e-gw0-5c1f590758/error-404-isvc-45944-predictor-7f69cd48-45twp" Oct 31 07:52:56.751501 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:56.751479 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2kjb\" (UniqueName: \"kubernetes.io/projected/d42c1eae-03e7-4ca5-9c85-70a0574a1dea-kube-api-access-l2kjb\") pod \"error-404-isvc-45944-predictor-7f69cd48-45twp\" (UID: \"d42c1eae-03e7-4ca5-9c85-70a0574a1dea\") " pod="e2e-gw0-5c1f590758/error-404-isvc-45944-predictor-7f69cd48-45twp" Oct 31 07:52:56.751624 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:56.751511 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d42c1eae-03e7-4ca5-9c85-70a0574a1dea-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-45944-predictor-7f69cd48-45twp\" (UID: \"d42c1eae-03e7-4ca5-9c85-70a0574a1dea\") " pod="e2e-gw0-5c1f590758/error-404-isvc-45944-predictor-7f69cd48-45twp" Oct 31 07:52:56.751624 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:56.751541 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d42c1eae-03e7-4ca5-9c85-70a0574a1dea-proxy-tls\") pod \"error-404-isvc-45944-predictor-7f69cd48-45twp\" (UID: \"d42c1eae-03e7-4ca5-9c85-70a0574a1dea\") " pod="e2e-gw0-5c1f590758/error-404-isvc-45944-predictor-7f69cd48-45twp" Oct 31 07:52:56.752101 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:56.752083 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d42c1eae-03e7-4ca5-9c85-70a0574a1dea-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-45944-predictor-7f69cd48-45twp\" (UID: \"d42c1eae-03e7-4ca5-9c85-70a0574a1dea\") " pod="e2e-gw0-5c1f590758/error-404-isvc-45944-predictor-7f69cd48-45twp" Oct 31 07:52:56.754090 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:56.754071 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d42c1eae-03e7-4ca5-9c85-70a0574a1dea-proxy-tls\") pod \"error-404-isvc-45944-predictor-7f69cd48-45twp\" (UID: \"d42c1eae-03e7-4ca5-9c85-70a0574a1dea\") " pod="e2e-gw0-5c1f590758/error-404-isvc-45944-predictor-7f69cd48-45twp" Oct 31 07:52:56.764317 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:56.764298 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2kjb\" (UniqueName: \"kubernetes.io/projected/d42c1eae-03e7-4ca5-9c85-70a0574a1dea-kube-api-access-l2kjb\") pod \"error-404-isvc-45944-predictor-7f69cd48-45twp\" (UID: \"d42c1eae-03e7-4ca5-9c85-70a0574a1dea\") " pod="e2e-gw0-5c1f590758/error-404-isvc-45944-predictor-7f69cd48-45twp" Oct 31 07:52:56.825743 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:56.825677 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:52:56.848677 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:56.848656 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-5c1f590758/error-404-isvc-45944-predictor-7f69cd48-45twp" Oct 31 07:52:57.124649 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:57.124624 4935 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-5c1f590758/error-404-isvc-45944-predictor-7f69cd48-45twp"] Oct 31 07:52:57.127125 ip-10-0-130-48 kubenswrapper[4935]: W1031 07:52:57.127089 4935 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd42c1eae_03e7_4ca5_9c85_70a0574a1dea.slice/crio-4dfff0230b32b3f67e517ec2298e3f026367f436abba229b72d1fd66fa8fd6c4 WatchSource:0}: Error finding container 4dfff0230b32b3f67e517ec2298e3f026367f436abba229b72d1fd66fa8fd6c4: Status 404 returned error can't find the container with id 4dfff0230b32b3f67e517ec2298e3f026367f436abba229b72d1fd66fa8fd6c4 Oct 31 07:52:57.880546 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:57.880508 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-5c1f590758/error-404-isvc-45944-predictor-7f69cd48-45twp" event={"ID":"d42c1eae-03e7-4ca5-9c85-70a0574a1dea","Type":"ContainerStarted","Data":"d19719a4a7e3b3faee9124b5f326b4d8fbd4b614834a4caefbf59996112edf10"} Oct 31 07:52:57.880957 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:57.880573 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-5c1f590758/error-404-isvc-45944-predictor-7f69cd48-45twp" event={"ID":"d42c1eae-03e7-4ca5-9c85-70a0574a1dea","Type":"ContainerStarted","Data":"63d1e8eb9cee66b16c4445ec79484040c15bc0ddeeeffccd7a3c1a180d5e7b25"} Oct 31 07:52:57.880957 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:57.880608 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-5c1f590758/error-404-isvc-45944-predictor-7f69cd48-45twp" Oct 31 07:52:57.880957 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:57.880633 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-5c1f590758/error-404-isvc-45944-predictor-7f69cd48-45twp" event={"ID":"d42c1eae-03e7-4ca5-9c85-70a0574a1dea","Type":"ContainerStarted","Data":"4dfff0230b32b3f67e517ec2298e3f026367f436abba229b72d1fd66fa8fd6c4"} Oct 31 07:52:57.880957 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:57.880651 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-5c1f590758/error-404-isvc-45944-predictor-7f69cd48-45twp" Oct 31 07:52:57.882111 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:57.882089 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-5c1f590758/error-404-isvc-45944-predictor-7f69cd48-45twp" podUID="d42c1eae-03e7-4ca5-9c85-70a0574a1dea" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused" Oct 31 07:52:57.900808 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:57.900765 4935 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-5c1f590758/error-404-isvc-45944-predictor-7f69cd48-45twp" podStartSLOduration=1.900751068 podStartE2EDuration="1.900751068s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 07:52:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 07:52:57.899661987 +0000 UTC m=+1351.770193521" watchObservedRunningTime="2026-10-31 07:52:57.900751068 +0000 UTC m=+1351.771282604" Oct 31 07:52:58.884482 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:52:58.884442 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-5c1f590758/error-404-isvc-45944-predictor-7f69cd48-45twp" podUID="d42c1eae-03e7-4ca5-9c85-70a0574a1dea" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused" Oct 31 07:53:01.454587 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:53:01.454554 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 07:53:03.889071 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:53:03.889044 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-5c1f590758/error-404-isvc-45944-predictor-7f69cd48-45twp" Oct 31 07:53:03.889614 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:53:03.889582 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-5c1f590758/error-404-isvc-45944-predictor-7f69cd48-45twp" podUID="d42c1eae-03e7-4ca5-9c85-70a0574a1dea" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused" Oct 31 07:53:04.716603 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:53:04.716557 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/error-404-isvc-bc1fb-predictor-5f96d49d6f-c86vj" podUID="12d27978-9126-407f-87a4-dcd7866bfc90" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused" Oct 31 07:53:13.889928 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:53:13.889887 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-5c1f590758/error-404-isvc-45944-predictor-7f69cd48-45twp" podUID="d42c1eae-03e7-4ca5-9c85-70a0574a1dea" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused" Oct 31 07:53:14.717171 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:53:14.717144 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-03ede73268/error-404-isvc-bc1fb-predictor-5f96d49d6f-c86vj" Oct 31 07:53:17.495647 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:53:17.495617 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:53:23.890237 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:53:23.890195 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-5c1f590758/error-404-isvc-45944-predictor-7f69cd48-45twp" podUID="d42c1eae-03e7-4ca5-9c85-70a0574a1dea" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused" Oct 31 07:53:27.528949 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:53:27.528909 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:53:27.729240 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:53:27.729212 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:53:31.454859 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:53:31.454814 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 07:53:33.889771 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:53:33.889732 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-5c1f590758/error-404-isvc-45944-predictor-7f69cd48-45twp" podUID="d42c1eae-03e7-4ca5-9c85-70a0574a1dea" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused" Oct 31 07:53:43.890187 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:53:43.890154 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-5c1f590758/error-404-isvc-45944-predictor-7f69cd48-45twp" podUID="d42c1eae-03e7-4ca5-9c85-70a0574a1dea" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused" Oct 31 07:53:53.891019 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:53:53.890992 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-5c1f590758/error-404-isvc-45944-predictor-7f69cd48-45twp" Oct 31 07:54:01.455091 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:54:01.455047 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 07:54:08.971318 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:54:08.971278 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:54:09.172109 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:54:09.172062 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:54:17.556344 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:54:17.556289 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:54:31.454614 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:54:31.454576 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 07:54:42.046805 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:54:42.046762 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:54:45.916031 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:54:45.915994 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:54:46.119044 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:54:46.119012 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:55:01.455072 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:55:01.454968 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 07:55:17.598729 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:55:17.598643 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:55:26.355027 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:55:26.355002 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jztcz_396e1c5e-3700-4fe6-b367-54ad2a244225/console-operator/2.log" Oct 31 07:55:26.359090 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:55:26.359073 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jztcz_396e1c5e-3700-4fe6-b367-54ad2a244225/console-operator/2.log" Oct 31 07:55:31.455249 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:55:31.455211 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 07:56:01.454796 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:56:01.454746 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 07:56:17.656375 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:56:17.656326 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:56:31.453946 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:56:31.453907 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 07:57:01.455205 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:57:01.455170 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 07:57:17.615958 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:57:17.615926 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:57:31.454519 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:57:31.454484 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 07:58:01.454448 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:58:01.454402 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 07:58:17.757911 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:58:17.757836 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:58:31.456116 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:58:31.456071 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 07:58:54.324648 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:58:54.324618 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:59:01.454687 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:59:01.454653 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 07:59:17.812199 ip-10-0-130-48 kubenswrapper[4935]: I1031 07:59:17.812166 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 07:59:31.455232 ip-10-0-130-48 kubenswrapper[4935]: E1031 07:59:31.455198 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 08:00:01.455574 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:00:01.455469 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 08:00:17.867219 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:00:17.867184 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:00:26.380491 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:00:26.380462 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jztcz_396e1c5e-3700-4fe6-b367-54ad2a244225/console-operator/2.log" Oct 31 08:00:26.384663 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:00:26.384638 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jztcz_396e1c5e-3700-4fe6-b367-54ad2a244225/console-operator/2.log" Oct 31 08:00:31.454484 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:00:31.454443 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 08:01:01.455252 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:01:01.455210 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 08:01:17.927319 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:17.927180 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:01:31.455909 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:01:31.455873 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 08:01:39.341299 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:39.341268 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:01:40.690305 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:40.690262 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:01:43.268293 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:43.268258 4935 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-03ede73268/error-404-isvc-bc1fb-predictor-5f96d49d6f-c86vj"] Oct 31 08:01:43.268788 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:43.268572 4935 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-03ede73268/error-404-isvc-bc1fb-predictor-5f96d49d6f-c86vj" podUID="12d27978-9126-407f-87a4-dcd7866bfc90" containerName="kserve-container" containerID="cri-o://74e3b5b9272888051a0ad33c6eb2475c44ed45091306bf1b263886cca6e13ede" gracePeriod=30 Oct 31 08:01:43.268788 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:43.268622 4935 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-03ede73268/error-404-isvc-bc1fb-predictor-5f96d49d6f-c86vj" podUID="12d27978-9126-407f-87a4-dcd7866bfc90" containerName="kube-rbac-proxy" containerID="cri-o://eaf6943a6c985d25c424292d84249a52b99fc10d1726c16931cc39ccf9e661ef" gracePeriod=30 Oct 31 08:01:43.412863 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:43.412829 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:01:43.612686 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:43.612600 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:01:44.048500 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:44.048465 4935 generic.go:356] "Generic (PLEG): container finished" podID="12d27978-9126-407f-87a4-dcd7866bfc90" containerID="eaf6943a6c985d25c424292d84249a52b99fc10d1726c16931cc39ccf9e661ef" exitCode=0 Oct 31 08:01:44.048672 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:44.048524 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-03ede73268/error-404-isvc-bc1fb-predictor-5f96d49d6f-c86vj" event={"ID":"12d27978-9126-407f-87a4-dcd7866bfc90","Type":"ContainerDied","Data":"eaf6943a6c985d25c424292d84249a52b99fc10d1726c16931cc39ccf9e661ef"} Oct 31 08:01:44.711254 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:44.711211 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/error-404-isvc-bc1fb-predictor-5f96d49d6f-c86vj" podUID="12d27978-9126-407f-87a4-dcd7866bfc90" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.37:8643/healthz\": dial tcp 10.132.0.37:8643: connect: connection refused" Oct 31 08:01:44.716560 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:44.716537 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/error-404-isvc-bc1fb-predictor-5f96d49d6f-c86vj" podUID="12d27978-9126-407f-87a4-dcd7866bfc90" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.37:8080: connect: connection refused" Oct 31 08:01:46.816296 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:46.816271 4935 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-03ede73268/error-404-isvc-bc1fb-predictor-5f96d49d6f-c86vj" Oct 31 08:01:46.918743 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:46.918644 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/12d27978-9126-407f-87a4-dcd7866bfc90-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/12d27978-9126-407f-87a4-dcd7866bfc90-kube-rbac-proxy-sar-config\") pod \"12d27978-9126-407f-87a4-dcd7866bfc90\" (UID: \"12d27978-9126-407f-87a4-dcd7866bfc90\") " Oct 31 08:01:46.918743 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:46.918733 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/12d27978-9126-407f-87a4-dcd7866bfc90-proxy-tls\" (UniqueName: \"kubernetes.io/secret/12d27978-9126-407f-87a4-dcd7866bfc90-proxy-tls\") pod \"12d27978-9126-407f-87a4-dcd7866bfc90\" (UID: \"12d27978-9126-407f-87a4-dcd7866bfc90\") " Oct 31 08:01:46.918982 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:46.918801 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/12d27978-9126-407f-87a4-dcd7866bfc90-kube-api-access-5g7vg\" (UniqueName: \"kubernetes.io/projected/12d27978-9126-407f-87a4-dcd7866bfc90-kube-api-access-5g7vg\") pod \"12d27978-9126-407f-87a4-dcd7866bfc90\" (UID: \"12d27978-9126-407f-87a4-dcd7866bfc90\") " Oct 31 08:01:46.919031 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:46.918991 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12d27978-9126-407f-87a4-dcd7866bfc90-kube-rbac-proxy-sar-config" pod "12d27978-9126-407f-87a4-dcd7866bfc90" (UID: "12d27978-9126-407f-87a4-dcd7866bfc90"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 31 08:01:46.921047 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:46.921025 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12d27978-9126-407f-87a4-dcd7866bfc90-kube-api-access-5g7vg" pod "12d27978-9126-407f-87a4-dcd7866bfc90" (UID: "12d27978-9126-407f-87a4-dcd7866bfc90"). InnerVolumeSpecName "kube-api-access-5g7vg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 31 08:01:46.921112 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:46.921030 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/12d27978-9126-407f-87a4-dcd7866bfc90-proxy-tls" pod "12d27978-9126-407f-87a4-dcd7866bfc90" (UID: "12d27978-9126-407f-87a4-dcd7866bfc90"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 31 08:01:47.019606 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:47.019571 4935 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5g7vg\" (UniqueName: \"kubernetes.io/projected/12d27978-9126-407f-87a4-dcd7866bfc90-kube-api-access-5g7vg\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 08:01:47.019606 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:47.019600 4935 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/12d27978-9126-407f-87a4-dcd7866bfc90-kube-rbac-proxy-sar-config\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 08:01:47.019606 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:47.019611 4935 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/12d27978-9126-407f-87a4-dcd7866bfc90-proxy-tls\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 08:01:47.061600 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:47.061569 4935 generic.go:356] "Generic (PLEG): container finished" podID="12d27978-9126-407f-87a4-dcd7866bfc90" containerID="74e3b5b9272888051a0ad33c6eb2475c44ed45091306bf1b263886cca6e13ede" exitCode=0 Oct 31 08:01:47.061782 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:47.061620 4935 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-03ede73268/error-404-isvc-bc1fb-predictor-5f96d49d6f-c86vj" Oct 31 08:01:47.061782 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:47.061624 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-03ede73268/error-404-isvc-bc1fb-predictor-5f96d49d6f-c86vj" event={"ID":"12d27978-9126-407f-87a4-dcd7866bfc90","Type":"ContainerDied","Data":"74e3b5b9272888051a0ad33c6eb2475c44ed45091306bf1b263886cca6e13ede"} Oct 31 08:01:47.061782 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:47.061685 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-03ede73268/error-404-isvc-bc1fb-predictor-5f96d49d6f-c86vj" event={"ID":"12d27978-9126-407f-87a4-dcd7866bfc90","Type":"ContainerDied","Data":"727bc4c3bc951d526305fa828cc037e9bd724460575c6873c1de13ca40db0393"} Oct 31 08:01:47.061782 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:47.061718 4935 scope.go:122] "RemoveContainer" containerID="eaf6943a6c985d25c424292d84249a52b99fc10d1726c16931cc39ccf9e661ef" Oct 31 08:01:47.070769 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:47.070752 4935 scope.go:122] "RemoveContainer" containerID="74e3b5b9272888051a0ad33c6eb2475c44ed45091306bf1b263886cca6e13ede" Oct 31 08:01:47.078552 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:47.078527 4935 scope.go:122] "RemoveContainer" containerID="eaf6943a6c985d25c424292d84249a52b99fc10d1726c16931cc39ccf9e661ef" Oct 31 08:01:47.078860 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:01:47.078840 4935 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eaf6943a6c985d25c424292d84249a52b99fc10d1726c16931cc39ccf9e661ef\": container with ID starting with eaf6943a6c985d25c424292d84249a52b99fc10d1726c16931cc39ccf9e661ef not found: ID does not exist" containerID="eaf6943a6c985d25c424292d84249a52b99fc10d1726c16931cc39ccf9e661ef" Oct 31 08:01:47.078922 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:47.078872 4935 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eaf6943a6c985d25c424292d84249a52b99fc10d1726c16931cc39ccf9e661ef"} err="failed to get container status \"eaf6943a6c985d25c424292d84249a52b99fc10d1726c16931cc39ccf9e661ef\": rpc error: code = NotFound desc = could not find container \"eaf6943a6c985d25c424292d84249a52b99fc10d1726c16931cc39ccf9e661ef\": container with ID starting with eaf6943a6c985d25c424292d84249a52b99fc10d1726c16931cc39ccf9e661ef not found: ID does not exist" Oct 31 08:01:47.078922 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:47.078890 4935 scope.go:122] "RemoveContainer" containerID="74e3b5b9272888051a0ad33c6eb2475c44ed45091306bf1b263886cca6e13ede" Oct 31 08:01:47.079153 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:01:47.079139 4935 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74e3b5b9272888051a0ad33c6eb2475c44ed45091306bf1b263886cca6e13ede\": container with ID starting with 74e3b5b9272888051a0ad33c6eb2475c44ed45091306bf1b263886cca6e13ede not found: ID does not exist" containerID="74e3b5b9272888051a0ad33c6eb2475c44ed45091306bf1b263886cca6e13ede" Oct 31 08:01:47.079194 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:47.079157 4935 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74e3b5b9272888051a0ad33c6eb2475c44ed45091306bf1b263886cca6e13ede"} err="failed to get container status \"74e3b5b9272888051a0ad33c6eb2475c44ed45091306bf1b263886cca6e13ede\": rpc error: code = NotFound desc = could not find container \"74e3b5b9272888051a0ad33c6eb2475c44ed45091306bf1b263886cca6e13ede\": container with ID starting with 74e3b5b9272888051a0ad33c6eb2475c44ed45091306bf1b263886cca6e13ede not found: ID does not exist" Oct 31 08:01:47.089000 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:47.083722 4935 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-03ede73268/error-404-isvc-bc1fb-predictor-5f96d49d6f-c86vj"] Oct 31 08:01:47.089000 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:47.086601 4935 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-03ede73268/error-404-isvc-bc1fb-predictor-5f96d49d6f-c86vj"] Oct 31 08:01:48.243143 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:48.243109 4935 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="12d27978-9126-407f-87a4-dcd7866bfc90" path="/var/lib/kubelet/pods/12d27978-9126-407f-87a4-dcd7866bfc90/volumes" Oct 31 08:01:48.812439 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:48.812403 4935 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-03ede73268/error-404-isvc-d859c-predictor-6689cddc7f-9fnqq"] Oct 31 08:01:48.812741 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:48.812730 4935 cpu_manager.go:414] "containerMap: removing container" podUID="12d27978-9126-407f-87a4-dcd7866bfc90" containerName="kube-rbac-proxy" Oct 31 08:01:48.812790 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:48.812743 4935 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="12d27978-9126-407f-87a4-dcd7866bfc90" containerName="kube-rbac-proxy" Oct 31 08:01:48.812790 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:48.812767 4935 cpu_manager.go:414] "containerMap: removing container" podUID="12d27978-9126-407f-87a4-dcd7866bfc90" containerName="kserve-container" Oct 31 08:01:48.812790 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:48.812772 4935 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="12d27978-9126-407f-87a4-dcd7866bfc90" containerName="kserve-container" Oct 31 08:01:48.812901 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:48.812829 4935 memory_manager.go:365] "RemoveStaleState removing state" podUID="12d27978-9126-407f-87a4-dcd7866bfc90" containerName="kserve-container" Oct 31 08:01:48.812901 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:48.812840 4935 memory_manager.go:365] "RemoveStaleState removing state" podUID="12d27978-9126-407f-87a4-dcd7866bfc90" containerName="kube-rbac-proxy" Oct 31 08:01:48.815709 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:48.815694 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-03ede73268/error-404-isvc-d859c-predictor-6689cddc7f-9fnqq" Oct 31 08:01:48.826240 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:48.826222 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-03ede73268\"/\"error-404-isvc-d859c-predictor-serving-cert\"" Oct 31 08:01:48.827449 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:48.827430 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-03ede73268\"/\"error-404-isvc-d859c-kube-rbac-proxy-sar-config\"" Oct 31 08:01:48.827532 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:48.827480 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-03ede73268\"/\"kube-root-ca.crt\"" Oct 31 08:01:48.838249 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:48.838232 4935 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-03ede73268/error-404-isvc-d859c-predictor-6689cddc7f-9fnqq"] Oct 31 08:01:48.935691 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:48.935656 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fc816c92-a312-4cb4-a566-397d4609625a-proxy-tls\") pod \"error-404-isvc-d859c-predictor-6689cddc7f-9fnqq\" (UID: \"fc816c92-a312-4cb4-a566-397d4609625a\") " pod="e2e-gw1-03ede73268/error-404-isvc-d859c-predictor-6689cddc7f-9fnqq" Oct 31 08:01:48.935691 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:48.935695 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mt74c\" (UniqueName: \"kubernetes.io/projected/fc816c92-a312-4cb4-a566-397d4609625a-kube-api-access-mt74c\") pod \"error-404-isvc-d859c-predictor-6689cddc7f-9fnqq\" (UID: \"fc816c92-a312-4cb4-a566-397d4609625a\") " pod="e2e-gw1-03ede73268/error-404-isvc-d859c-predictor-6689cddc7f-9fnqq" Oct 31 08:01:48.935908 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:48.935790 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fc816c92-a312-4cb4-a566-397d4609625a-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-d859c-predictor-6689cddc7f-9fnqq\" (UID: \"fc816c92-a312-4cb4-a566-397d4609625a\") " pod="e2e-gw1-03ede73268/error-404-isvc-d859c-predictor-6689cddc7f-9fnqq" Oct 31 08:01:49.037199 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:49.037161 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fc816c92-a312-4cb4-a566-397d4609625a-proxy-tls\") pod \"error-404-isvc-d859c-predictor-6689cddc7f-9fnqq\" (UID: \"fc816c92-a312-4cb4-a566-397d4609625a\") " pod="e2e-gw1-03ede73268/error-404-isvc-d859c-predictor-6689cddc7f-9fnqq" Oct 31 08:01:49.037427 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:49.037206 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mt74c\" (UniqueName: \"kubernetes.io/projected/fc816c92-a312-4cb4-a566-397d4609625a-kube-api-access-mt74c\") pod \"error-404-isvc-d859c-predictor-6689cddc7f-9fnqq\" (UID: \"fc816c92-a312-4cb4-a566-397d4609625a\") " pod="e2e-gw1-03ede73268/error-404-isvc-d859c-predictor-6689cddc7f-9fnqq" Oct 31 08:01:49.037427 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:49.037341 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fc816c92-a312-4cb4-a566-397d4609625a-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-d859c-predictor-6689cddc7f-9fnqq\" (UID: \"fc816c92-a312-4cb4-a566-397d4609625a\") " pod="e2e-gw1-03ede73268/error-404-isvc-d859c-predictor-6689cddc7f-9fnqq" Oct 31 08:01:49.037942 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:49.037922 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fc816c92-a312-4cb4-a566-397d4609625a-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-d859c-predictor-6689cddc7f-9fnqq\" (UID: \"fc816c92-a312-4cb4-a566-397d4609625a\") " pod="e2e-gw1-03ede73268/error-404-isvc-d859c-predictor-6689cddc7f-9fnqq" Oct 31 08:01:49.039786 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:49.039769 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fc816c92-a312-4cb4-a566-397d4609625a-proxy-tls\") pod \"error-404-isvc-d859c-predictor-6689cddc7f-9fnqq\" (UID: \"fc816c92-a312-4cb4-a566-397d4609625a\") " pod="e2e-gw1-03ede73268/error-404-isvc-d859c-predictor-6689cddc7f-9fnqq" Oct 31 08:01:49.042700 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:49.042676 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:01:49.049457 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:49.049438 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mt74c\" (UniqueName: \"kubernetes.io/projected/fc816c92-a312-4cb4-a566-397d4609625a-kube-api-access-mt74c\") pod \"error-404-isvc-d859c-predictor-6689cddc7f-9fnqq\" (UID: \"fc816c92-a312-4cb4-a566-397d4609625a\") " pod="e2e-gw1-03ede73268/error-404-isvc-d859c-predictor-6689cddc7f-9fnqq" Oct 31 08:01:49.130826 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:49.130741 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-03ede73268/error-404-isvc-d859c-predictor-6689cddc7f-9fnqq" Oct 31 08:01:49.311757 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:49.311718 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:01:49.406341 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:49.406265 4935 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-03ede73268/error-404-isvc-d859c-predictor-6689cddc7f-9fnqq"] Oct 31 08:01:49.409747 ip-10-0-130-48 kubenswrapper[4935]: W1031 08:01:49.409710 4935 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc816c92_a312_4cb4_a566_397d4609625a.slice/crio-c68a58889a40e7e61b4406761dc110eac9955471691a8bb400cc753adcaf62c8 WatchSource:0}: Error finding container c68a58889a40e7e61b4406761dc110eac9955471691a8bb400cc753adcaf62c8: Status 404 returned error can't find the container with id c68a58889a40e7e61b4406761dc110eac9955471691a8bb400cc753adcaf62c8 Oct 31 08:01:50.076865 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:50.076824 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-03ede73268/error-404-isvc-d859c-predictor-6689cddc7f-9fnqq" event={"ID":"fc816c92-a312-4cb4-a566-397d4609625a","Type":"ContainerStarted","Data":"beeb54b24be320e2eb199a7bf7bbb99e3d04fe4957ef7bf98d921cc98d3e7762"} Oct 31 08:01:50.077052 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:50.076895 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-03ede73268/error-404-isvc-d859c-predictor-6689cddc7f-9fnqq" event={"ID":"fc816c92-a312-4cb4-a566-397d4609625a","Type":"ContainerStarted","Data":"ed7834ca1effab404ba879b58c55e73df1b0531f7575361c036c383d7bd9addd"} Oct 31 08:01:50.077052 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:50.076928 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-03ede73268/error-404-isvc-d859c-predictor-6689cddc7f-9fnqq" event={"ID":"fc816c92-a312-4cb4-a566-397d4609625a","Type":"ContainerStarted","Data":"c68a58889a40e7e61b4406761dc110eac9955471691a8bb400cc753adcaf62c8"} Oct 31 08:01:50.077153 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:50.077067 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-03ede73268/error-404-isvc-d859c-predictor-6689cddc7f-9fnqq" Oct 31 08:01:50.097473 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:50.097422 4935 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-03ede73268/error-404-isvc-d859c-predictor-6689cddc7f-9fnqq" podStartSLOduration=2.097406604 podStartE2EDuration="2.097406604s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 08:01:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 08:01:50.096703627 +0000 UTC m=+1883.967235166" watchObservedRunningTime="2026-10-31 08:01:50.097406604 +0000 UTC m=+1883.967938140" Oct 31 08:01:51.080555 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:51.080522 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-03ede73268/error-404-isvc-d859c-predictor-6689cddc7f-9fnqq" Oct 31 08:01:51.081965 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:51.081934 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/error-404-isvc-d859c-predictor-6689cddc7f-9fnqq" podUID="fc816c92-a312-4cb4-a566-397d4609625a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused" Oct 31 08:01:52.085232 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:52.085183 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/error-404-isvc-d859c-predictor-6689cddc7f-9fnqq" podUID="fc816c92-a312-4cb4-a566-397d4609625a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused" Oct 31 08:01:57.089846 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:57.089819 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-03ede73268/error-404-isvc-d859c-predictor-6689cddc7f-9fnqq" Oct 31 08:01:57.090453 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:01:57.090426 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/error-404-isvc-d859c-predictor-6689cddc7f-9fnqq" podUID="fc816c92-a312-4cb4-a566-397d4609625a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused" Oct 31 08:02:01.454777 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:02:01.454740 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 08:02:07.090460 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:07.090417 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/error-404-isvc-d859c-predictor-6689cddc7f-9fnqq" podUID="fc816c92-a312-4cb4-a566-397d4609625a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused" Oct 31 08:02:17.090440 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:17.090402 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/error-404-isvc-d859c-predictor-6689cddc7f-9fnqq" podUID="fc816c92-a312-4cb4-a566-397d4609625a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused" Oct 31 08:02:17.976567 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:17.976532 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:02:22.269989 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:22.269950 4935 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-5c1f590758/error-404-isvc-45944-predictor-7f69cd48-45twp"] Oct 31 08:02:22.270497 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:22.270163 4935 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-5c1f590758/error-404-isvc-45944-predictor-7f69cd48-45twp" podUID="d42c1eae-03e7-4ca5-9c85-70a0574a1dea" containerName="kserve-container" containerID="cri-o://63d1e8eb9cee66b16c4445ec79484040c15bc0ddeeeffccd7a3c1a180d5e7b25" gracePeriod=30 Oct 31 08:02:22.270497 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:22.270205 4935 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-5c1f590758/error-404-isvc-45944-predictor-7f69cd48-45twp" podUID="d42c1eae-03e7-4ca5-9c85-70a0574a1dea" containerName="kube-rbac-proxy" containerID="cri-o://d19719a4a7e3b3faee9124b5f326b4d8fbd4b614834a4caefbf59996112edf10" gracePeriod=30 Oct 31 08:02:22.418600 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:22.418573 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:02:22.619601 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:22.619519 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:02:23.213593 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:23.213562 4935 generic.go:356] "Generic (PLEG): container finished" podID="d42c1eae-03e7-4ca5-9c85-70a0574a1dea" containerID="d19719a4a7e3b3faee9124b5f326b4d8fbd4b614834a4caefbf59996112edf10" exitCode=0 Oct 31 08:02:23.213771 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:23.213639 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-5c1f590758/error-404-isvc-45944-predictor-7f69cd48-45twp" event={"ID":"d42c1eae-03e7-4ca5-9c85-70a0574a1dea","Type":"ContainerDied","Data":"d19719a4a7e3b3faee9124b5f326b4d8fbd4b614834a4caefbf59996112edf10"} Oct 31 08:02:23.885070 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:23.885028 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-5c1f590758/error-404-isvc-45944-predictor-7f69cd48-45twp" podUID="d42c1eae-03e7-4ca5-9c85-70a0574a1dea" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.38:8643/healthz\": dial tcp 10.132.0.38:8643: connect: connection refused" Oct 31 08:02:23.890420 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:23.890391 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-5c1f590758/error-404-isvc-45944-predictor-7f69cd48-45twp" podUID="d42c1eae-03e7-4ca5-9c85-70a0574a1dea" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused" Oct 31 08:02:25.517246 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:25.517224 4935 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-5c1f590758/error-404-isvc-45944-predictor-7f69cd48-45twp" Oct 31 08:02:25.629377 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:25.629292 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d42c1eae-03e7-4ca5-9c85-70a0574a1dea-proxy-tls\" (UniqueName: \"kubernetes.io/secret/d42c1eae-03e7-4ca5-9c85-70a0574a1dea-proxy-tls\") pod \"d42c1eae-03e7-4ca5-9c85-70a0574a1dea\" (UID: \"d42c1eae-03e7-4ca5-9c85-70a0574a1dea\") " Oct 31 08:02:25.629377 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:25.629334 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d42c1eae-03e7-4ca5-9c85-70a0574a1dea-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d42c1eae-03e7-4ca5-9c85-70a0574a1dea-kube-rbac-proxy-sar-config\") pod \"d42c1eae-03e7-4ca5-9c85-70a0574a1dea\" (UID: \"d42c1eae-03e7-4ca5-9c85-70a0574a1dea\") " Oct 31 08:02:25.629599 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:25.629401 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d42c1eae-03e7-4ca5-9c85-70a0574a1dea-kube-api-access-l2kjb\" (UniqueName: \"kubernetes.io/projected/d42c1eae-03e7-4ca5-9c85-70a0574a1dea-kube-api-access-l2kjb\") pod \"d42c1eae-03e7-4ca5-9c85-70a0574a1dea\" (UID: \"d42c1eae-03e7-4ca5-9c85-70a0574a1dea\") " Oct 31 08:02:25.629745 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:25.629725 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d42c1eae-03e7-4ca5-9c85-70a0574a1dea-kube-rbac-proxy-sar-config" pod "d42c1eae-03e7-4ca5-9c85-70a0574a1dea" (UID: "d42c1eae-03e7-4ca5-9c85-70a0574a1dea"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 31 08:02:25.631519 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:25.631488 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d42c1eae-03e7-4ca5-9c85-70a0574a1dea-proxy-tls" pod "d42c1eae-03e7-4ca5-9c85-70a0574a1dea" (UID: "d42c1eae-03e7-4ca5-9c85-70a0574a1dea"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 31 08:02:25.631680 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:25.631655 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d42c1eae-03e7-4ca5-9c85-70a0574a1dea-kube-api-access-l2kjb" pod "d42c1eae-03e7-4ca5-9c85-70a0574a1dea" (UID: "d42c1eae-03e7-4ca5-9c85-70a0574a1dea"). InnerVolumeSpecName "kube-api-access-l2kjb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 31 08:02:25.730667 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:25.730642 4935 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d42c1eae-03e7-4ca5-9c85-70a0574a1dea-proxy-tls\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 08:02:25.730667 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:25.730665 4935 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d42c1eae-03e7-4ca5-9c85-70a0574a1dea-kube-rbac-proxy-sar-config\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 08:02:25.730821 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:25.730676 4935 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l2kjb\" (UniqueName: \"kubernetes.io/projected/d42c1eae-03e7-4ca5-9c85-70a0574a1dea-kube-api-access-l2kjb\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 08:02:26.229633 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:26.229598 4935 generic.go:356] "Generic (PLEG): container finished" podID="d42c1eae-03e7-4ca5-9c85-70a0574a1dea" containerID="63d1e8eb9cee66b16c4445ec79484040c15bc0ddeeeffccd7a3c1a180d5e7b25" exitCode=0 Oct 31 08:02:26.229833 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:26.229645 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-5c1f590758/error-404-isvc-45944-predictor-7f69cd48-45twp" event={"ID":"d42c1eae-03e7-4ca5-9c85-70a0574a1dea","Type":"ContainerDied","Data":"63d1e8eb9cee66b16c4445ec79484040c15bc0ddeeeffccd7a3c1a180d5e7b25"} Oct 31 08:02:26.229833 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:26.229651 4935 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-5c1f590758/error-404-isvc-45944-predictor-7f69cd48-45twp" Oct 31 08:02:26.229833 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:26.229704 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-5c1f590758/error-404-isvc-45944-predictor-7f69cd48-45twp" event={"ID":"d42c1eae-03e7-4ca5-9c85-70a0574a1dea","Type":"ContainerDied","Data":"4dfff0230b32b3f67e517ec2298e3f026367f436abba229b72d1fd66fa8fd6c4"} Oct 31 08:02:26.229833 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:26.229740 4935 scope.go:122] "RemoveContainer" containerID="d19719a4a7e3b3faee9124b5f326b4d8fbd4b614834a4caefbf59996112edf10" Oct 31 08:02:26.238649 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:26.238620 4935 scope.go:122] "RemoveContainer" containerID="63d1e8eb9cee66b16c4445ec79484040c15bc0ddeeeffccd7a3c1a180d5e7b25" Oct 31 08:02:26.246606 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:26.246591 4935 scope.go:122] "RemoveContainer" containerID="d19719a4a7e3b3faee9124b5f326b4d8fbd4b614834a4caefbf59996112edf10" Oct 31 08:02:26.246810 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:02:26.246793 4935 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d19719a4a7e3b3faee9124b5f326b4d8fbd4b614834a4caefbf59996112edf10\": container with ID starting with d19719a4a7e3b3faee9124b5f326b4d8fbd4b614834a4caefbf59996112edf10 not found: ID does not exist" containerID="d19719a4a7e3b3faee9124b5f326b4d8fbd4b614834a4caefbf59996112edf10" Oct 31 08:02:26.246860 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:26.246819 4935 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d19719a4a7e3b3faee9124b5f326b4d8fbd4b614834a4caefbf59996112edf10"} err="failed to get container status \"d19719a4a7e3b3faee9124b5f326b4d8fbd4b614834a4caefbf59996112edf10\": rpc error: code = NotFound desc = could not find container \"d19719a4a7e3b3faee9124b5f326b4d8fbd4b614834a4caefbf59996112edf10\": container with ID starting with d19719a4a7e3b3faee9124b5f326b4d8fbd4b614834a4caefbf59996112edf10 not found: ID does not exist" Oct 31 08:02:26.246860 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:26.246834 4935 scope.go:122] "RemoveContainer" containerID="63d1e8eb9cee66b16c4445ec79484040c15bc0ddeeeffccd7a3c1a180d5e7b25" Oct 31 08:02:26.247089 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:02:26.247071 4935 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63d1e8eb9cee66b16c4445ec79484040c15bc0ddeeeffccd7a3c1a180d5e7b25\": container with ID starting with 63d1e8eb9cee66b16c4445ec79484040c15bc0ddeeeffccd7a3c1a180d5e7b25 not found: ID does not exist" containerID="63d1e8eb9cee66b16c4445ec79484040c15bc0ddeeeffccd7a3c1a180d5e7b25" Oct 31 08:02:26.247138 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:26.247091 4935 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63d1e8eb9cee66b16c4445ec79484040c15bc0ddeeeffccd7a3c1a180d5e7b25"} err="failed to get container status \"63d1e8eb9cee66b16c4445ec79484040c15bc0ddeeeffccd7a3c1a180d5e7b25\": rpc error: code = NotFound desc = could not find container \"63d1e8eb9cee66b16c4445ec79484040c15bc0ddeeeffccd7a3c1a180d5e7b25\": container with ID starting with 63d1e8eb9cee66b16c4445ec79484040c15bc0ddeeeffccd7a3c1a180d5e7b25 not found: ID does not exist" Oct 31 08:02:26.252979 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:26.252959 4935 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-5c1f590758/error-404-isvc-45944-predictor-7f69cd48-45twp"] Oct 31 08:02:26.259460 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:26.259437 4935 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-5c1f590758/error-404-isvc-45944-predictor-7f69cd48-45twp"] Oct 31 08:02:27.090514 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:27.090478 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/error-404-isvc-d859c-predictor-6689cddc7f-9fnqq" podUID="fc816c92-a312-4cb4-a566-397d4609625a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused" Oct 31 08:02:27.803765 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:27.803725 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:02:27.899255 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:27.899226 4935 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-5c1f590758/error-404-isvc-8ae00-predictor-7f849b78d-rjwg7"] Oct 31 08:02:27.899635 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:27.899621 4935 cpu_manager.go:414] "containerMap: removing container" podUID="d42c1eae-03e7-4ca5-9c85-70a0574a1dea" containerName="kserve-container" Oct 31 08:02:27.899687 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:27.899637 4935 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d42c1eae-03e7-4ca5-9c85-70a0574a1dea" containerName="kserve-container" Oct 31 08:02:27.899687 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:27.899658 4935 cpu_manager.go:414] "containerMap: removing container" podUID="d42c1eae-03e7-4ca5-9c85-70a0574a1dea" containerName="kube-rbac-proxy" Oct 31 08:02:27.899687 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:27.899663 4935 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d42c1eae-03e7-4ca5-9c85-70a0574a1dea" containerName="kube-rbac-proxy" Oct 31 08:02:27.899783 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:27.899714 4935 memory_manager.go:365] "RemoveStaleState removing state" podUID="d42c1eae-03e7-4ca5-9c85-70a0574a1dea" containerName="kube-rbac-proxy" Oct 31 08:02:27.899783 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:27.899726 4935 memory_manager.go:365] "RemoveStaleState removing state" podUID="d42c1eae-03e7-4ca5-9c85-70a0574a1dea" containerName="kserve-container" Oct 31 08:02:27.904043 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:27.904028 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-5c1f590758/error-404-isvc-8ae00-predictor-7f849b78d-rjwg7" Oct 31 08:02:27.911591 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:27.911569 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-5c1f590758\"/\"error-404-isvc-8ae00-predictor-serving-cert\"" Oct 31 08:02:27.911741 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:27.911569 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-5c1f590758\"/\"kube-root-ca.crt\"" Oct 31 08:02:27.911741 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:27.911626 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-5c1f590758\"/\"error-404-isvc-8ae00-kube-rbac-proxy-sar-config\"" Oct 31 08:02:27.912999 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:27.912981 4935 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-5c1f590758/error-404-isvc-8ae00-predictor-7f849b78d-rjwg7"] Oct 31 08:02:28.003404 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:28.003373 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:02:28.047207 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:28.047188 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/364dfb0e-d451-4015-b577-785b0ae174b4-proxy-tls\") pod \"error-404-isvc-8ae00-predictor-7f849b78d-rjwg7\" (UID: \"364dfb0e-d451-4015-b577-785b0ae174b4\") " pod="e2e-gw0-5c1f590758/error-404-isvc-8ae00-predictor-7f849b78d-rjwg7" Oct 31 08:02:28.047321 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:28.047218 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmprj\" (UniqueName: \"kubernetes.io/projected/364dfb0e-d451-4015-b577-785b0ae174b4-kube-api-access-rmprj\") pod \"error-404-isvc-8ae00-predictor-7f849b78d-rjwg7\" (UID: \"364dfb0e-d451-4015-b577-785b0ae174b4\") " pod="e2e-gw0-5c1f590758/error-404-isvc-8ae00-predictor-7f849b78d-rjwg7" Oct 31 08:02:28.047321 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:28.047239 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/364dfb0e-d451-4015-b577-785b0ae174b4-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-8ae00-predictor-7f849b78d-rjwg7\" (UID: \"364dfb0e-d451-4015-b577-785b0ae174b4\") " pod="e2e-gw0-5c1f590758/error-404-isvc-8ae00-predictor-7f849b78d-rjwg7" Oct 31 08:02:28.147718 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:28.147655 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmprj\" (UniqueName: \"kubernetes.io/projected/364dfb0e-d451-4015-b577-785b0ae174b4-kube-api-access-rmprj\") pod \"error-404-isvc-8ae00-predictor-7f849b78d-rjwg7\" (UID: \"364dfb0e-d451-4015-b577-785b0ae174b4\") " pod="e2e-gw0-5c1f590758/error-404-isvc-8ae00-predictor-7f849b78d-rjwg7" Oct 31 08:02:28.147718 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:28.147689 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/364dfb0e-d451-4015-b577-785b0ae174b4-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-8ae00-predictor-7f849b78d-rjwg7\" (UID: \"364dfb0e-d451-4015-b577-785b0ae174b4\") " pod="e2e-gw0-5c1f590758/error-404-isvc-8ae00-predictor-7f849b78d-rjwg7" Oct 31 08:02:28.148119 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:28.147760 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/364dfb0e-d451-4015-b577-785b0ae174b4-proxy-tls\") pod \"error-404-isvc-8ae00-predictor-7f849b78d-rjwg7\" (UID: \"364dfb0e-d451-4015-b577-785b0ae174b4\") " pod="e2e-gw0-5c1f590758/error-404-isvc-8ae00-predictor-7f849b78d-rjwg7" Oct 31 08:02:28.148468 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:28.148452 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/364dfb0e-d451-4015-b577-785b0ae174b4-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-8ae00-predictor-7f849b78d-rjwg7\" (UID: \"364dfb0e-d451-4015-b577-785b0ae174b4\") " pod="e2e-gw0-5c1f590758/error-404-isvc-8ae00-predictor-7f849b78d-rjwg7" Oct 31 08:02:28.150302 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:28.150286 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/364dfb0e-d451-4015-b577-785b0ae174b4-proxy-tls\") pod \"error-404-isvc-8ae00-predictor-7f849b78d-rjwg7\" (UID: \"364dfb0e-d451-4015-b577-785b0ae174b4\") " pod="e2e-gw0-5c1f590758/error-404-isvc-8ae00-predictor-7f849b78d-rjwg7" Oct 31 08:02:28.160428 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:28.160404 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmprj\" (UniqueName: \"kubernetes.io/projected/364dfb0e-d451-4015-b577-785b0ae174b4-kube-api-access-rmprj\") pod \"error-404-isvc-8ae00-predictor-7f849b78d-rjwg7\" (UID: \"364dfb0e-d451-4015-b577-785b0ae174b4\") " pod="e2e-gw0-5c1f590758/error-404-isvc-8ae00-predictor-7f849b78d-rjwg7" Oct 31 08:02:28.202186 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:28.202157 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:02:28.219613 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:28.219589 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-5c1f590758/error-404-isvc-8ae00-predictor-7f849b78d-rjwg7" Oct 31 08:02:28.244046 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:28.244019 4935 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d42c1eae-03e7-4ca5-9c85-70a0574a1dea" path="/var/lib/kubelet/pods/d42c1eae-03e7-4ca5-9c85-70a0574a1dea/volumes" Oct 31 08:02:28.498295 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:28.498118 4935 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-5c1f590758/error-404-isvc-8ae00-predictor-7f849b78d-rjwg7"] Oct 31 08:02:28.500988 ip-10-0-130-48 kubenswrapper[4935]: W1031 08:02:28.500958 4935 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod364dfb0e_d451_4015_b577_785b0ae174b4.slice/crio-32e30d452d7f26ed9ecf9737b1d7619484c1e3024ed8e07bbf36acd5fc60b715 WatchSource:0}: Error finding container 32e30d452d7f26ed9ecf9737b1d7619484c1e3024ed8e07bbf36acd5fc60b715: Status 404 returned error can't find the container with id 32e30d452d7f26ed9ecf9737b1d7619484c1e3024ed8e07bbf36acd5fc60b715 Oct 31 08:02:29.250881 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:29.250848 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-5c1f590758/error-404-isvc-8ae00-predictor-7f849b78d-rjwg7" event={"ID":"364dfb0e-d451-4015-b577-785b0ae174b4","Type":"ContainerStarted","Data":"58243a78d18ef50a032cf3149324285c64bde508a152a6f4722661d5e8a2a66e"} Oct 31 08:02:29.251273 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:29.250911 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-5c1f590758/error-404-isvc-8ae00-predictor-7f849b78d-rjwg7" event={"ID":"364dfb0e-d451-4015-b577-785b0ae174b4","Type":"ContainerStarted","Data":"a5e7f2c10a96a42de3eaec7705f1f9fae5ff95456a708c1a270820bc74e1d757"} Oct 31 08:02:29.251273 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:29.250933 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-5c1f590758/error-404-isvc-8ae00-predictor-7f849b78d-rjwg7" event={"ID":"364dfb0e-d451-4015-b577-785b0ae174b4","Type":"ContainerStarted","Data":"32e30d452d7f26ed9ecf9737b1d7619484c1e3024ed8e07bbf36acd5fc60b715"} Oct 31 08:02:29.251273 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:29.250987 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-5c1f590758/error-404-isvc-8ae00-predictor-7f849b78d-rjwg7" Oct 31 08:02:29.272703 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:29.272654 4935 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-5c1f590758/error-404-isvc-8ae00-predictor-7f849b78d-rjwg7" podStartSLOduration=2.272640933 podStartE2EDuration="2.272640933s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 08:02:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 08:02:29.270925455 +0000 UTC m=+1923.141456993" watchObservedRunningTime="2026-10-31 08:02:29.272640933 +0000 UTC m=+1923.143172469" Oct 31 08:02:30.254273 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:30.254244 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-5c1f590758/error-404-isvc-8ae00-predictor-7f849b78d-rjwg7" Oct 31 08:02:30.255578 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:30.255549 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-5c1f590758/error-404-isvc-8ae00-predictor-7f849b78d-rjwg7" podUID="364dfb0e-d451-4015-b577-785b0ae174b4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused" Oct 31 08:02:31.258573 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:31.258532 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-5c1f590758/error-404-isvc-8ae00-predictor-7f849b78d-rjwg7" podUID="364dfb0e-d451-4015-b577-785b0ae174b4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused" Oct 31 08:02:31.454477 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:02:31.454442 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 08:02:36.263185 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:36.263156 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-5c1f590758/error-404-isvc-8ae00-predictor-7f849b78d-rjwg7" Oct 31 08:02:36.263625 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:36.263563 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-5c1f590758/error-404-isvc-8ae00-predictor-7f849b78d-rjwg7" podUID="364dfb0e-d451-4015-b577-785b0ae174b4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused" Oct 31 08:02:37.091593 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:37.091507 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-03ede73268/error-404-isvc-d859c-predictor-6689cddc7f-9fnqq" Oct 31 08:02:46.263847 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:46.263812 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-5c1f590758/error-404-isvc-8ae00-predictor-7f849b78d-rjwg7" podUID="364dfb0e-d451-4015-b577-785b0ae174b4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused" Oct 31 08:02:56.264592 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:56.264555 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-5c1f590758/error-404-isvc-8ae00-predictor-7f849b78d-rjwg7" podUID="364dfb0e-d451-4015-b577-785b0ae174b4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused" Oct 31 08:02:58.856030 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:58.855999 4935 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-03ede73268/splitter-graph-d859c-75c5cd656-ffflh"] Oct 31 08:02:58.859325 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:58.859309 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-03ede73268/splitter-graph-d859c-75c5cd656-ffflh" Oct 31 08:02:58.868435 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:58.868409 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-03ede73268\"/\"splitter-graph-d859c-kube-rbac-proxy-sar-config\"" Oct 31 08:02:58.868435 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:58.868432 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-03ede73268\"/\"splitter-graph-d859c-serving-cert\"" Oct 31 08:02:58.871076 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:58.871050 4935 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-03ede73268/splitter-graph-d859c-75c5cd656-ffflh"] Oct 31 08:02:58.977552 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:58.977490 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:02:58.984301 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:58.984277 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62d26bfc-7aed-4be2-a24c-33afc28acf05-openshift-service-ca-bundle\") pod \"splitter-graph-d859c-75c5cd656-ffflh\" (UID: \"62d26bfc-7aed-4be2-a24c-33afc28acf05\") " pod="e2e-gw1-03ede73268/splitter-graph-d859c-75c5cd656-ffflh" Oct 31 08:02:58.984418 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:58.984316 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/62d26bfc-7aed-4be2-a24c-33afc28acf05-proxy-tls\") pod \"splitter-graph-d859c-75c5cd656-ffflh\" (UID: \"62d26bfc-7aed-4be2-a24c-33afc28acf05\") " pod="e2e-gw1-03ede73268/splitter-graph-d859c-75c5cd656-ffflh" Oct 31 08:02:59.085520 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:59.085497 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62d26bfc-7aed-4be2-a24c-33afc28acf05-openshift-service-ca-bundle\") pod \"splitter-graph-d859c-75c5cd656-ffflh\" (UID: \"62d26bfc-7aed-4be2-a24c-33afc28acf05\") " pod="e2e-gw1-03ede73268/splitter-graph-d859c-75c5cd656-ffflh" Oct 31 08:02:59.085623 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:59.085532 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/62d26bfc-7aed-4be2-a24c-33afc28acf05-proxy-tls\") pod \"splitter-graph-d859c-75c5cd656-ffflh\" (UID: \"62d26bfc-7aed-4be2-a24c-33afc28acf05\") " pod="e2e-gw1-03ede73268/splitter-graph-d859c-75c5cd656-ffflh" Oct 31 08:02:59.086123 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:59.086090 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62d26bfc-7aed-4be2-a24c-33afc28acf05-openshift-service-ca-bundle\") pod \"splitter-graph-d859c-75c5cd656-ffflh\" (UID: \"62d26bfc-7aed-4be2-a24c-33afc28acf05\") " pod="e2e-gw1-03ede73268/splitter-graph-d859c-75c5cd656-ffflh" Oct 31 08:02:59.088023 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:59.088004 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/62d26bfc-7aed-4be2-a24c-33afc28acf05-proxy-tls\") pod \"splitter-graph-d859c-75c5cd656-ffflh\" (UID: \"62d26bfc-7aed-4be2-a24c-33afc28acf05\") " pod="e2e-gw1-03ede73268/splitter-graph-d859c-75c5cd656-ffflh" Oct 31 08:02:59.172014 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:59.171957 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-03ede73268/splitter-graph-d859c-75c5cd656-ffflh" Oct 31 08:02:59.172695 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:59.172676 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:02:59.442913 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:02:59.442837 4935 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-03ede73268/splitter-graph-d859c-75c5cd656-ffflh"] Oct 31 08:02:59.445792 ip-10-0-130-48 kubenswrapper[4935]: W1031 08:02:59.445770 4935 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62d26bfc_7aed_4be2_a24c_33afc28acf05.slice/crio-6955dd3a6de482edbffaf5f466c48584db912df37a7bd1f46adca8f43a193f7f WatchSource:0}: Error finding container 6955dd3a6de482edbffaf5f466c48584db912df37a7bd1f46adca8f43a193f7f: Status 404 returned error can't find the container with id 6955dd3a6de482edbffaf5f466c48584db912df37a7bd1f46adca8f43a193f7f Oct 31 08:03:00.376204 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:00.376169 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-03ede73268/splitter-graph-d859c-75c5cd656-ffflh" event={"ID":"62d26bfc-7aed-4be2-a24c-33afc28acf05","Type":"ContainerStarted","Data":"5f595ac212c238a5e9d78671d9f0ab0f8a732768b77ffe8d3ff79bcd77e89a89"} Oct 31 08:03:00.376705 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:00.376225 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-03ede73268/splitter-graph-d859c-75c5cd656-ffflh" event={"ID":"62d26bfc-7aed-4be2-a24c-33afc28acf05","Type":"ContainerStarted","Data":"6955dd3a6de482edbffaf5f466c48584db912df37a7bd1f46adca8f43a193f7f"} Oct 31 08:03:00.376705 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:00.376291 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-03ede73268/splitter-graph-d859c-75c5cd656-ffflh" Oct 31 08:03:00.394642 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:00.394600 4935 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-03ede73268/splitter-graph-d859c-75c5cd656-ffflh" podStartSLOduration=2.39458817 podStartE2EDuration="2.39458817s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 08:02:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 08:03:00.393530196 +0000 UTC m=+1954.264061734" watchObservedRunningTime="2026-10-31 08:03:00.39458817 +0000 UTC m=+1954.265119706" Oct 31 08:03:01.454476 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:03:01.454443 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 08:03:06.264269 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:06.264237 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-5c1f590758/error-404-isvc-8ae00-predictor-7f849b78d-rjwg7" podUID="364dfb0e-d451-4015-b577-785b0ae174b4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused" Oct 31 08:03:06.385924 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:06.385895 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-03ede73268/splitter-graph-d859c-75c5cd656-ffflh" Oct 31 08:03:10.086051 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:10.085995 4935 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-03ede73268/error-404-isvc-d859c-predictor-6689cddc7f-9fnqq"] Oct 31 08:03:10.086565 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:10.086275 4935 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-03ede73268/error-404-isvc-d859c-predictor-6689cddc7f-9fnqq" podUID="fc816c92-a312-4cb4-a566-397d4609625a" containerName="kserve-container" containerID="cri-o://ed7834ca1effab404ba879b58c55e73df1b0531f7575361c036c383d7bd9addd" gracePeriod=30 Oct 31 08:03:10.086565 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:10.086322 4935 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-03ede73268/error-404-isvc-d859c-predictor-6689cddc7f-9fnqq" podUID="fc816c92-a312-4cb4-a566-397d4609625a" containerName="kube-rbac-proxy" containerID="cri-o://beeb54b24be320e2eb199a7bf7bbb99e3d04fe4957ef7bf98d921cc98d3e7762" gracePeriod=30 Oct 31 08:03:10.265911 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:10.265873 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:03:10.419917 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:10.419833 4935 generic.go:356] "Generic (PLEG): container finished" podID="fc816c92-a312-4cb4-a566-397d4609625a" containerID="beeb54b24be320e2eb199a7bf7bbb99e3d04fe4957ef7bf98d921cc98d3e7762" exitCode=0 Oct 31 08:03:10.419917 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:10.419893 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-03ede73268/error-404-isvc-d859c-predictor-6689cddc7f-9fnqq" event={"ID":"fc816c92-a312-4cb4-a566-397d4609625a","Type":"ContainerDied","Data":"beeb54b24be320e2eb199a7bf7bbb99e3d04fe4957ef7bf98d921cc98d3e7762"} Oct 31 08:03:10.469238 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:10.469212 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:03:12.085786 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:12.085738 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/error-404-isvc-d859c-predictor-6689cddc7f-9fnqq" podUID="fc816c92-a312-4cb4-a566-397d4609625a" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.39:8643/healthz\": dial tcp 10.132.0.39:8643: connect: connection refused" Oct 31 08:03:13.434362 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:13.434317 4935 generic.go:356] "Generic (PLEG): container finished" podID="fc816c92-a312-4cb4-a566-397d4609625a" containerID="ed7834ca1effab404ba879b58c55e73df1b0531f7575361c036c383d7bd9addd" exitCode=0 Oct 31 08:03:13.434701 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:13.434373 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-03ede73268/error-404-isvc-d859c-predictor-6689cddc7f-9fnqq" event={"ID":"fc816c92-a312-4cb4-a566-397d4609625a","Type":"ContainerDied","Data":"ed7834ca1effab404ba879b58c55e73df1b0531f7575361c036c383d7bd9addd"} Oct 31 08:03:13.551070 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:13.551041 4935 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-03ede73268/error-404-isvc-d859c-predictor-6689cddc7f-9fnqq" Oct 31 08:03:13.601713 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:13.601686 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fc816c92-a312-4cb4-a566-397d4609625a-kube-api-access-mt74c\" (UniqueName: \"kubernetes.io/projected/fc816c92-a312-4cb4-a566-397d4609625a-kube-api-access-mt74c\") pod \"fc816c92-a312-4cb4-a566-397d4609625a\" (UID: \"fc816c92-a312-4cb4-a566-397d4609625a\") " Oct 31 08:03:13.601885 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:13.601736 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fc816c92-a312-4cb4-a566-397d4609625a-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fc816c92-a312-4cb4-a566-397d4609625a-kube-rbac-proxy-sar-config\") pod \"fc816c92-a312-4cb4-a566-397d4609625a\" (UID: \"fc816c92-a312-4cb4-a566-397d4609625a\") " Oct 31 08:03:13.601885 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:13.601774 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fc816c92-a312-4cb4-a566-397d4609625a-proxy-tls\" (UniqueName: \"kubernetes.io/secret/fc816c92-a312-4cb4-a566-397d4609625a-proxy-tls\") pod \"fc816c92-a312-4cb4-a566-397d4609625a\" (UID: \"fc816c92-a312-4cb4-a566-397d4609625a\") " Oct 31 08:03:13.602175 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:13.602138 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc816c92-a312-4cb4-a566-397d4609625a-kube-rbac-proxy-sar-config" pod "fc816c92-a312-4cb4-a566-397d4609625a" (UID: "fc816c92-a312-4cb4-a566-397d4609625a"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 31 08:03:13.604663 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:13.604640 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc816c92-a312-4cb4-a566-397d4609625a-proxy-tls" pod "fc816c92-a312-4cb4-a566-397d4609625a" (UID: "fc816c92-a312-4cb4-a566-397d4609625a"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 31 08:03:13.604663 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:13.604647 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc816c92-a312-4cb4-a566-397d4609625a-kube-api-access-mt74c" pod "fc816c92-a312-4cb4-a566-397d4609625a" (UID: "fc816c92-a312-4cb4-a566-397d4609625a"). InnerVolumeSpecName "kube-api-access-mt74c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 31 08:03:13.702407 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:13.702375 4935 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mt74c\" (UniqueName: \"kubernetes.io/projected/fc816c92-a312-4cb4-a566-397d4609625a-kube-api-access-mt74c\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 08:03:13.702407 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:13.702406 4935 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fc816c92-a312-4cb4-a566-397d4609625a-kube-rbac-proxy-sar-config\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 08:03:13.702566 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:13.702422 4935 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fc816c92-a312-4cb4-a566-397d4609625a-proxy-tls\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 08:03:14.441070 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:14.441037 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-03ede73268/error-404-isvc-d859c-predictor-6689cddc7f-9fnqq" event={"ID":"fc816c92-a312-4cb4-a566-397d4609625a","Type":"ContainerDied","Data":"c68a58889a40e7e61b4406761dc110eac9955471691a8bb400cc753adcaf62c8"} Oct 31 08:03:14.441070 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:14.441066 4935 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-03ede73268/error-404-isvc-d859c-predictor-6689cddc7f-9fnqq" Oct 31 08:03:14.441589 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:14.441101 4935 scope.go:122] "RemoveContainer" containerID="beeb54b24be320e2eb199a7bf7bbb99e3d04fe4957ef7bf98d921cc98d3e7762" Oct 31 08:03:14.449657 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:14.449638 4935 scope.go:122] "RemoveContainer" containerID="ed7834ca1effab404ba879b58c55e73df1b0531f7575361c036c383d7bd9addd" Oct 31 08:03:14.457954 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:14.457934 4935 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-03ede73268/error-404-isvc-d859c-predictor-6689cddc7f-9fnqq"] Oct 31 08:03:14.463530 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:14.463507 4935 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-03ede73268/error-404-isvc-d859c-predictor-6689cddc7f-9fnqq"] Oct 31 08:03:15.591388 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:15.591343 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:03:15.691904 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:15.691874 4935 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-03ede73268/error-404-isvc-e5af3-predictor-5bf79df77c-wknfx"] Oct 31 08:03:15.692205 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:15.692194 4935 cpu_manager.go:414] "containerMap: removing container" podUID="fc816c92-a312-4cb4-a566-397d4609625a" containerName="kube-rbac-proxy" Oct 31 08:03:15.692247 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:15.692207 4935 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fc816c92-a312-4cb4-a566-397d4609625a" containerName="kube-rbac-proxy" Oct 31 08:03:15.692247 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:15.692226 4935 cpu_manager.go:414] "containerMap: removing container" podUID="fc816c92-a312-4cb4-a566-397d4609625a" containerName="kserve-container" Oct 31 08:03:15.692247 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:15.692231 4935 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fc816c92-a312-4cb4-a566-397d4609625a" containerName="kserve-container" Oct 31 08:03:15.692341 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:15.692297 4935 memory_manager.go:365] "RemoveStaleState removing state" podUID="fc816c92-a312-4cb4-a566-397d4609625a" containerName="kserve-container" Oct 31 08:03:15.692341 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:15.692309 4935 memory_manager.go:365] "RemoveStaleState removing state" podUID="fc816c92-a312-4cb4-a566-397d4609625a" containerName="kube-rbac-proxy" Oct 31 08:03:15.696811 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:15.696795 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-03ede73268/error-404-isvc-e5af3-predictor-5bf79df77c-wknfx" Oct 31 08:03:15.700215 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:15.700195 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-03ede73268\"/\"error-404-isvc-e5af3-predictor-serving-cert\"" Oct 31 08:03:15.700334 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:15.700298 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-03ede73268\"/\"error-404-isvc-e5af3-kube-rbac-proxy-sar-config\"" Oct 31 08:03:15.702871 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:15.702854 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-03ede73268\"/\"kube-root-ca.crt\"" Oct 31 08:03:15.705208 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:15.705186 4935 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-03ede73268/error-404-isvc-e5af3-predictor-5bf79df77c-wknfx"] Oct 31 08:03:15.717301 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:15.717280 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-e5af3-predictor-5bf79df77c-wknfx\" (UID: \"9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71\") " pod="e2e-gw1-03ede73268/error-404-isvc-e5af3-predictor-5bf79df77c-wknfx" Oct 31 08:03:15.717460 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:15.717437 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j89ds\" (UniqueName: \"kubernetes.io/projected/9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71-kube-api-access-j89ds\") pod \"error-404-isvc-e5af3-predictor-5bf79df77c-wknfx\" (UID: \"9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71\") " pod="e2e-gw1-03ede73268/error-404-isvc-e5af3-predictor-5bf79df77c-wknfx" Oct 31 08:03:15.717578 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:15.717561 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71-proxy-tls\") pod \"error-404-isvc-e5af3-predictor-5bf79df77c-wknfx\" (UID: \"9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71\") " pod="e2e-gw1-03ede73268/error-404-isvc-e5af3-predictor-5bf79df77c-wknfx" Oct 31 08:03:15.793024 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:15.792990 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:03:15.818254 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:15.818228 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71-proxy-tls\") pod \"error-404-isvc-e5af3-predictor-5bf79df77c-wknfx\" (UID: \"9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71\") " pod="e2e-gw1-03ede73268/error-404-isvc-e5af3-predictor-5bf79df77c-wknfx" Oct 31 08:03:15.818405 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:15.818272 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-e5af3-predictor-5bf79df77c-wknfx\" (UID: \"9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71\") " pod="e2e-gw1-03ede73268/error-404-isvc-e5af3-predictor-5bf79df77c-wknfx" Oct 31 08:03:15.818405 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:15.818333 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j89ds\" (UniqueName: \"kubernetes.io/projected/9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71-kube-api-access-j89ds\") pod \"error-404-isvc-e5af3-predictor-5bf79df77c-wknfx\" (UID: \"9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71\") " pod="e2e-gw1-03ede73268/error-404-isvc-e5af3-predictor-5bf79df77c-wknfx" Oct 31 08:03:15.818996 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:15.818966 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-e5af3-predictor-5bf79df77c-wknfx\" (UID: \"9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71\") " pod="e2e-gw1-03ede73268/error-404-isvc-e5af3-predictor-5bf79df77c-wknfx" Oct 31 08:03:15.820916 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:15.820900 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71-proxy-tls\") pod \"error-404-isvc-e5af3-predictor-5bf79df77c-wknfx\" (UID: \"9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71\") " pod="e2e-gw1-03ede73268/error-404-isvc-e5af3-predictor-5bf79df77c-wknfx" Oct 31 08:03:15.830120 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:15.830092 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j89ds\" (UniqueName: \"kubernetes.io/projected/9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71-kube-api-access-j89ds\") pod \"error-404-isvc-e5af3-predictor-5bf79df77c-wknfx\" (UID: \"9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71\") " pod="e2e-gw1-03ede73268/error-404-isvc-e5af3-predictor-5bf79df77c-wknfx" Oct 31 08:03:15.992255 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:15.992224 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:03:16.012696 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:16.012676 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-03ede73268/error-404-isvc-e5af3-predictor-5bf79df77c-wknfx" Oct 31 08:03:16.243495 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:16.243420 4935 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fc816c92-a312-4cb4-a566-397d4609625a" path="/var/lib/kubelet/pods/fc816c92-a312-4cb4-a566-397d4609625a/volumes" Oct 31 08:03:16.264489 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:16.264465 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-5c1f590758/error-404-isvc-8ae00-predictor-7f849b78d-rjwg7" Oct 31 08:03:16.288744 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:16.288667 4935 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-03ede73268/error-404-isvc-e5af3-predictor-5bf79df77c-wknfx"] Oct 31 08:03:16.291728 ip-10-0-130-48 kubenswrapper[4935]: W1031 08:03:16.291702 4935 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e47b4c7_1cd9_4662_9fc2_500a5a9c1a71.slice/crio-1c708a93f1495bdeebd08c0df435444e475f1017d16a2acf5f2e10a5bb486bb0 WatchSource:0}: Error finding container 1c708a93f1495bdeebd08c0df435444e475f1017d16a2acf5f2e10a5bb486bb0: Status 404 returned error can't find the container with id 1c708a93f1495bdeebd08c0df435444e475f1017d16a2acf5f2e10a5bb486bb0 Oct 31 08:03:16.457762 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:16.457725 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-03ede73268/error-404-isvc-e5af3-predictor-5bf79df77c-wknfx" event={"ID":"9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71","Type":"ContainerStarted","Data":"7a7aca1736922c0b5e07fa33027265fa33562a6d683afb336715506a9c07a232"} Oct 31 08:03:16.457964 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:16.457945 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-03ede73268/error-404-isvc-e5af3-predictor-5bf79df77c-wknfx" Oct 31 08:03:16.458020 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:16.457992 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-03ede73268/error-404-isvc-e5af3-predictor-5bf79df77c-wknfx" Oct 31 08:03:16.458058 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:16.458020 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-03ede73268/error-404-isvc-e5af3-predictor-5bf79df77c-wknfx" event={"ID":"9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71","Type":"ContainerStarted","Data":"1d10a7dc7746c2b87217651a92748a53b21c349787e23fe1572e2582e35443a6"} Oct 31 08:03:16.458058 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:16.458051 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-03ede73268/error-404-isvc-e5af3-predictor-5bf79df77c-wknfx" event={"ID":"9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71","Type":"ContainerStarted","Data":"1c708a93f1495bdeebd08c0df435444e475f1017d16a2acf5f2e10a5bb486bb0"} Oct 31 08:03:16.459588 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:16.459563 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/error-404-isvc-e5af3-predictor-5bf79df77c-wknfx" podUID="9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused" Oct 31 08:03:16.478046 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:16.477992 4935 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-03ede73268/error-404-isvc-e5af3-predictor-5bf79df77c-wknfx" podStartSLOduration=1.477980699 podStartE2EDuration="1.477980699s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 08:03:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 08:03:16.476443668 +0000 UTC m=+1970.346975206" watchObservedRunningTime="2026-10-31 08:03:16.477980699 +0000 UTC m=+1970.348512236" Oct 31 08:03:17.461922 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:17.461878 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/error-404-isvc-e5af3-predictor-5bf79df77c-wknfx" podUID="9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused" Oct 31 08:03:18.026600 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:18.026572 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:03:18.465988 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:18.465952 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/error-404-isvc-e5af3-predictor-5bf79df77c-wknfx" podUID="9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused" Oct 31 08:03:23.470885 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:23.470858 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-03ede73268/error-404-isvc-e5af3-predictor-5bf79df77c-wknfx" Oct 31 08:03:23.471389 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:23.471274 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/error-404-isvc-e5af3-predictor-5bf79df77c-wknfx" podUID="9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused" Oct 31 08:03:29.932726 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:29.932696 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:03:30.134488 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:30.134459 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:03:31.454331 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:03:31.454290 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 08:03:33.471880 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:33.471816 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/error-404-isvc-e5af3-predictor-5bf79df77c-wknfx" podUID="9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused" Oct 31 08:03:37.901531 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:37.901493 4935 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-5c1f590758/switch-graph-8ae00-86769987cd-rhthk"] Oct 31 08:03:37.906315 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:37.906298 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-5c1f590758/switch-graph-8ae00-86769987cd-rhthk" Oct 31 08:03:37.913434 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:37.913396 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-5c1f590758\"/\"switch-graph-8ae00-serving-cert\"" Oct 31 08:03:37.913571 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:37.913442 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-5c1f590758\"/\"switch-graph-8ae00-kube-rbac-proxy-sar-config\"" Oct 31 08:03:37.915010 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:37.914989 4935 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-5c1f590758/switch-graph-8ae00-86769987cd-rhthk"] Oct 31 08:03:37.991702 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:37.991657 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c26d8ba-fd9e-424a-acbf-1836ba32033a-openshift-service-ca-bundle\") pod \"switch-graph-8ae00-86769987cd-rhthk\" (UID: \"9c26d8ba-fd9e-424a-acbf-1836ba32033a\") " pod="e2e-gw0-5c1f590758/switch-graph-8ae00-86769987cd-rhthk" Oct 31 08:03:37.991875 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:37.991711 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9c26d8ba-fd9e-424a-acbf-1836ba32033a-proxy-tls\") pod \"switch-graph-8ae00-86769987cd-rhthk\" (UID: \"9c26d8ba-fd9e-424a-acbf-1836ba32033a\") " pod="e2e-gw0-5c1f590758/switch-graph-8ae00-86769987cd-rhthk" Oct 31 08:03:38.022174 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:38.022134 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:03:38.092195 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:38.092161 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c26d8ba-fd9e-424a-acbf-1836ba32033a-openshift-service-ca-bundle\") pod \"switch-graph-8ae00-86769987cd-rhthk\" (UID: \"9c26d8ba-fd9e-424a-acbf-1836ba32033a\") " pod="e2e-gw0-5c1f590758/switch-graph-8ae00-86769987cd-rhthk" Oct 31 08:03:38.092376 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:38.092216 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9c26d8ba-fd9e-424a-acbf-1836ba32033a-proxy-tls\") pod \"switch-graph-8ae00-86769987cd-rhthk\" (UID: \"9c26d8ba-fd9e-424a-acbf-1836ba32033a\") " pod="e2e-gw0-5c1f590758/switch-graph-8ae00-86769987cd-rhthk" Oct 31 08:03:38.092792 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:38.092766 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c26d8ba-fd9e-424a-acbf-1836ba32033a-openshift-service-ca-bundle\") pod \"switch-graph-8ae00-86769987cd-rhthk\" (UID: \"9c26d8ba-fd9e-424a-acbf-1836ba32033a\") " pod="e2e-gw0-5c1f590758/switch-graph-8ae00-86769987cd-rhthk" Oct 31 08:03:38.094701 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:38.094679 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9c26d8ba-fd9e-424a-acbf-1836ba32033a-proxy-tls\") pod \"switch-graph-8ae00-86769987cd-rhthk\" (UID: \"9c26d8ba-fd9e-424a-acbf-1836ba32033a\") " pod="e2e-gw0-5c1f590758/switch-graph-8ae00-86769987cd-rhthk" Oct 31 08:03:38.219961 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:38.219934 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-5c1f590758/switch-graph-8ae00-86769987cd-rhthk" Oct 31 08:03:38.221978 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:38.221956 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:03:38.495420 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:38.495316 4935 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-5c1f590758/switch-graph-8ae00-86769987cd-rhthk"] Oct 31 08:03:38.498557 ip-10-0-130-48 kubenswrapper[4935]: W1031 08:03:38.498529 4935 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c26d8ba_fd9e_424a_acbf_1836ba32033a.slice/crio-7d1cd625406ecc057743f7b2a2f03265289dbd3e44c2ac8861844b0b8340fd07 WatchSource:0}: Error finding container 7d1cd625406ecc057743f7b2a2f03265289dbd3e44c2ac8861844b0b8340fd07: Status 404 returned error can't find the container with id 7d1cd625406ecc057743f7b2a2f03265289dbd3e44c2ac8861844b0b8340fd07 Oct 31 08:03:38.556480 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:38.556455 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-5c1f590758/switch-graph-8ae00-86769987cd-rhthk" event={"ID":"9c26d8ba-fd9e-424a-acbf-1836ba32033a","Type":"ContainerStarted","Data":"7d1cd625406ecc057743f7b2a2f03265289dbd3e44c2ac8861844b0b8340fd07"} Oct 31 08:03:39.562074 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:39.562038 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-5c1f590758/switch-graph-8ae00-86769987cd-rhthk" event={"ID":"9c26d8ba-fd9e-424a-acbf-1836ba32033a","Type":"ContainerStarted","Data":"d8ad9cab2c34e0db0bd76eb00814734f5331f6f8cf012a10d341b3ee4ee22118"} Oct 31 08:03:39.562500 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:39.562120 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-5c1f590758/switch-graph-8ae00-86769987cd-rhthk" Oct 31 08:03:39.581641 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:39.581588 4935 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-5c1f590758/switch-graph-8ae00-86769987cd-rhthk" podStartSLOduration=2.581577256 podStartE2EDuration="2.581577256s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 08:03:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 08:03:39.580666095 +0000 UTC m=+1993.451197642" watchObservedRunningTime="2026-10-31 08:03:39.581577256 +0000 UTC m=+1993.452108792" Oct 31 08:03:42.936229 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:42.936196 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:03:43.472017 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:43.471982 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/error-404-isvc-e5af3-predictor-5bf79df77c-wknfx" podUID="9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused" Oct 31 08:03:45.571997 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:45.571972 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-5c1f590758/switch-graph-8ae00-86769987cd-rhthk" Oct 31 08:03:53.472185 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:03:53.472146 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/error-404-isvc-e5af3-predictor-5bf79df77c-wknfx" podUID="9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused" Oct 31 08:04:01.455194 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:04:01.455155 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 08:04:03.472511 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:04:03.472478 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-03ede73268/error-404-isvc-e5af3-predictor-5bf79df77c-wknfx" Oct 31 08:04:18.085626 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:04:18.085549 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:04:25.809507 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:04:25.809473 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:04:26.009830 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:04:26.009784 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:04:31.455093 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:04:31.455057 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 08:05:01.454682 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:05:01.454647 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 08:05:18.130597 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:05:18.130561 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:05:26.407195 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:05:26.407165 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jztcz_396e1c5e-3700-4fe6-b367-54ad2a244225/console-operator/2.log" Oct 31 08:05:26.412016 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:05:26.411990 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jztcz_396e1c5e-3700-4fe6-b367-54ad2a244225/console-operator/2.log" Oct 31 08:05:31.454655 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:05:31.454620 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 08:06:01.454463 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:06:01.454413 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 08:06:18.191050 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:06:18.191019 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:06:31.455492 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:06:31.455448 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 08:07:01.454678 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:07:01.454646 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 08:07:18.240321 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:07:18.240246 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:07:31.454705 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:07:31.454658 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 08:08:01.455084 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:08:01.455042 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 08:08:18.202695 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:08:18.202657 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:08:19.625683 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:08:19.625647 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:08:31.455473 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:08:31.455436 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 08:09:01.455043 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:09:01.455009 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 08:09:18.367219 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:09:18.367177 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:09:31.454337 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:09:31.454299 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 08:10:01.454502 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:10:01.454464 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 08:10:18.399220 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:10:18.399190 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:10:26.433374 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:10:26.433325 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jztcz_396e1c5e-3700-4fe6-b367-54ad2a244225/console-operator/2.log" Oct 31 08:10:26.438715 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:10:26.438695 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jztcz_396e1c5e-3700-4fe6-b367-54ad2a244225/console-operator/2.log" Oct 31 08:10:31.454456 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:10:31.454418 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 08:11:01.456807 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:11:01.456768 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 08:11:18.451013 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:11:18.450980 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:11:31.455286 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:11:31.455252 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 08:12:01.455028 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:12:01.454991 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 08:12:18.503981 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:12:18.503941 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:12:31.454847 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:12:31.454810 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 08:12:41.485825 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:12:41.485792 4935 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-03ede73268/error-404-isvc-e5af3-predictor-5bf79df77c-wknfx"] Oct 31 08:12:41.486327 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:12:41.486009 4935 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-03ede73268/error-404-isvc-e5af3-predictor-5bf79df77c-wknfx" podUID="9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71" containerName="kserve-container" containerID="cri-o://1d10a7dc7746c2b87217651a92748a53b21c349787e23fe1572e2582e35443a6" gracePeriod=30 Oct 31 08:12:41.486327 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:12:41.486291 4935 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-03ede73268/error-404-isvc-e5af3-predictor-5bf79df77c-wknfx" podUID="9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71" containerName="kube-rbac-proxy" containerID="cri-o://7a7aca1736922c0b5e07fa33027265fa33562a6d683afb336715506a9c07a232" gracePeriod=30 Oct 31 08:12:41.636621 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:12:41.636593 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:12:41.820022 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:12:41.819940 4935 generic.go:356] "Generic (PLEG): container finished" podID="9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71" containerID="7a7aca1736922c0b5e07fa33027265fa33562a6d683afb336715506a9c07a232" exitCode=0 Oct 31 08:12:41.820174 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:12:41.820015 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-03ede73268/error-404-isvc-e5af3-predictor-5bf79df77c-wknfx" event={"ID":"9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71","Type":"ContainerDied","Data":"7a7aca1736922c0b5e07fa33027265fa33562a6d683afb336715506a9c07a232"} Oct 31 08:12:43.230366 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:12:43.230317 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:12:43.466865 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:12:43.466826 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/error-404-isvc-e5af3-predictor-5bf79df77c-wknfx" podUID="9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.42:8643/healthz\": dial tcp 10.132.0.42:8643: connect: connection refused" Oct 31 08:12:43.472224 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:12:43.472201 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/error-404-isvc-e5af3-predictor-5bf79df77c-wknfx" podUID="9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.42:8080: connect: connection refused" Oct 31 08:12:44.538025 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:12:44.538002 4935 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-03ede73268/error-404-isvc-e5af3-predictor-5bf79df77c-wknfx" Oct 31 08:12:44.628715 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:12:44.628636 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71-proxy-tls\" (UniqueName: \"kubernetes.io/secret/9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71-proxy-tls\") pod \"9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71\" (UID: \"9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71\") " Oct 31 08:12:44.628715 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:12:44.628695 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71-kube-rbac-proxy-sar-config\") pod \"9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71\" (UID: \"9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71\") " Oct 31 08:12:44.628918 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:12:44.628771 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71-kube-api-access-j89ds\" (UniqueName: \"kubernetes.io/projected/9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71-kube-api-access-j89ds\") pod \"9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71\" (UID: \"9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71\") " Oct 31 08:12:44.629131 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:12:44.629098 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71-kube-rbac-proxy-sar-config" pod "9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71" (UID: "9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 31 08:12:44.630950 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:12:44.630926 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71-proxy-tls" pod "9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71" (UID: "9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 31 08:12:44.631050 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:12:44.630956 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71-kube-api-access-j89ds" pod "9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71" (UID: "9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71"). InnerVolumeSpecName "kube-api-access-j89ds". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 31 08:12:44.729301 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:12:44.729273 4935 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71-kube-rbac-proxy-sar-config\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 08:12:44.729301 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:12:44.729298 4935 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j89ds\" (UniqueName: \"kubernetes.io/projected/9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71-kube-api-access-j89ds\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 08:12:44.729471 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:12:44.729308 4935 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71-proxy-tls\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 08:12:44.833949 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:12:44.833919 4935 generic.go:356] "Generic (PLEG): container finished" podID="9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71" containerID="1d10a7dc7746c2b87217651a92748a53b21c349787e23fe1572e2582e35443a6" exitCode=0 Oct 31 08:12:44.834075 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:12:44.833969 4935 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-03ede73268/error-404-isvc-e5af3-predictor-5bf79df77c-wknfx" Oct 31 08:12:44.834075 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:12:44.834004 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-03ede73268/error-404-isvc-e5af3-predictor-5bf79df77c-wknfx" event={"ID":"9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71","Type":"ContainerDied","Data":"1d10a7dc7746c2b87217651a92748a53b21c349787e23fe1572e2582e35443a6"} Oct 31 08:12:44.834075 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:12:44.834061 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-03ede73268/error-404-isvc-e5af3-predictor-5bf79df77c-wknfx" event={"ID":"9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71","Type":"ContainerDied","Data":"1c708a93f1495bdeebd08c0df435444e475f1017d16a2acf5f2e10a5bb486bb0"} Oct 31 08:12:44.834200 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:12:44.834088 4935 scope.go:122] "RemoveContainer" containerID="7a7aca1736922c0b5e07fa33027265fa33562a6d683afb336715506a9c07a232" Oct 31 08:12:44.843709 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:12:44.843694 4935 scope.go:122] "RemoveContainer" containerID="1d10a7dc7746c2b87217651a92748a53b21c349787e23fe1572e2582e35443a6" Oct 31 08:12:44.851524 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:12:44.851504 4935 scope.go:122] "RemoveContainer" containerID="7a7aca1736922c0b5e07fa33027265fa33562a6d683afb336715506a9c07a232" Oct 31 08:12:44.851776 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:12:44.851755 4935 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a7aca1736922c0b5e07fa33027265fa33562a6d683afb336715506a9c07a232\": container with ID starting with 7a7aca1736922c0b5e07fa33027265fa33562a6d683afb336715506a9c07a232 not found: ID does not exist" containerID="7a7aca1736922c0b5e07fa33027265fa33562a6d683afb336715506a9c07a232" Oct 31 08:12:44.851845 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:12:44.851789 4935 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a7aca1736922c0b5e07fa33027265fa33562a6d683afb336715506a9c07a232"} err="failed to get container status \"7a7aca1736922c0b5e07fa33027265fa33562a6d683afb336715506a9c07a232\": rpc error: code = NotFound desc = could not find container \"7a7aca1736922c0b5e07fa33027265fa33562a6d683afb336715506a9c07a232\": container with ID starting with 7a7aca1736922c0b5e07fa33027265fa33562a6d683afb336715506a9c07a232 not found: ID does not exist" Oct 31 08:12:44.851845 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:12:44.851814 4935 scope.go:122] "RemoveContainer" containerID="1d10a7dc7746c2b87217651a92748a53b21c349787e23fe1572e2582e35443a6" Oct 31 08:12:44.852050 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:12:44.852034 4935 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d10a7dc7746c2b87217651a92748a53b21c349787e23fe1572e2582e35443a6\": container with ID starting with 1d10a7dc7746c2b87217651a92748a53b21c349787e23fe1572e2582e35443a6 not found: ID does not exist" containerID="1d10a7dc7746c2b87217651a92748a53b21c349787e23fe1572e2582e35443a6" Oct 31 08:12:44.852088 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:12:44.852056 4935 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d10a7dc7746c2b87217651a92748a53b21c349787e23fe1572e2582e35443a6"} err="failed to get container status \"1d10a7dc7746c2b87217651a92748a53b21c349787e23fe1572e2582e35443a6\": rpc error: code = NotFound desc = could not find container \"1d10a7dc7746c2b87217651a92748a53b21c349787e23fe1572e2582e35443a6\": container with ID starting with 1d10a7dc7746c2b87217651a92748a53b21c349787e23fe1572e2582e35443a6 not found: ID does not exist" Oct 31 08:12:44.856538 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:12:44.856516 4935 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-03ede73268/error-404-isvc-e5af3-predictor-5bf79df77c-wknfx"] Oct 31 08:12:44.860490 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:12:44.860468 4935 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-03ede73268/error-404-isvc-e5af3-predictor-5bf79df77c-wknfx"] Oct 31 08:12:46.243243 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:12:46.243211 4935 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71" path="/var/lib/kubelet/pods/9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71/volumes" Oct 31 08:12:46.748289 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:12:46.748259 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:12:51.620219 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:12:51.620184 4935 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-03ede73268/splitter-graph-d859c-75c5cd656-ffflh"] Oct 31 08:12:51.620642 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:12:51.620412 4935 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-03ede73268/splitter-graph-d859c-75c5cd656-ffflh" podUID="62d26bfc-7aed-4be2-a24c-33afc28acf05" containerName="splitter-graph-d859c" containerID="cri-o://5f595ac212c238a5e9d78671d9f0ab0f8a732768b77ffe8d3ff79bcd77e89a89" gracePeriod=30 Oct 31 08:12:51.643086 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:12:51.643062 4935 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-03ede73268/switch-graph-43360-f8cd54dcb-4959p"] Oct 31 08:12:51.643255 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:12:51.643237 4935 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-03ede73268/switch-graph-43360-f8cd54dcb-4959p" podUID="6ada0c88-70ac-48e5-9e91-f6fd589b40cb" containerName="switch-graph-43360" containerID="cri-o://83ab399634f48fa661c5a3d7c2a75ec489fe3201223e5beb7bdb3a881ec08e68" gracePeriod=30 Oct 31 08:12:51.649838 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:12:51.649814 4935 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-03ede73268/switch-graph-490af-5ccfdb64c7-mzjwg"] Oct 31 08:12:51.649991 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:12:51.649974 4935 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-03ede73268/switch-graph-490af-5ccfdb64c7-mzjwg" podUID="8f678a8c-1a16-41eb-93d1-45c89b51c98d" containerName="switch-graph-490af" containerID="cri-o://e07751231ecd2c1bac7ab9dc1c3119b408b4a7492ce747cab103a356af9d0fe5" gracePeriod=30 Oct 31 08:12:51.840003 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:12:51.839917 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:12:52.241883 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:12:52.241850 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/switch-graph-490af-5ccfdb64c7-mzjwg" podUID="8f678a8c-1a16-41eb-93d1-45c89b51c98d" containerName="switch-graph-490af" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 31 08:12:54.860820 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:12:54.860783 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/switch-graph-43360-f8cd54dcb-4959p" podUID="6ada0c88-70ac-48e5-9e91-f6fd589b40cb" containerName="switch-graph-43360" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 31 08:12:56.383839 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:12:56.383800 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/splitter-graph-d859c-75c5cd656-ffflh" podUID="62d26bfc-7aed-4be2-a24c-33afc28acf05" containerName="splitter-graph-d859c" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 31 08:12:57.240792 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:12:57.240757 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/switch-graph-490af-5ccfdb64c7-mzjwg" podUID="8f678a8c-1a16-41eb-93d1-45c89b51c98d" containerName="switch-graph-490af" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 31 08:12:59.860970 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:12:59.860929 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/switch-graph-43360-f8cd54dcb-4959p" podUID="6ada0c88-70ac-48e5-9e91-f6fd589b40cb" containerName="switch-graph-43360" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 31 08:13:01.383881 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:01.383847 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/splitter-graph-d859c-75c5cd656-ffflh" podUID="62d26bfc-7aed-4be2-a24c-33afc28acf05" containerName="splitter-graph-d859c" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 31 08:13:01.454748 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:13:01.454716 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 08:13:02.242085 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:02.242033 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/switch-graph-490af-5ccfdb64c7-mzjwg" podUID="8f678a8c-1a16-41eb-93d1-45c89b51c98d" containerName="switch-graph-490af" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 31 08:13:02.246930 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:02.246902 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-03ede73268/switch-graph-490af-5ccfdb64c7-mzjwg" Oct 31 08:13:04.861236 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:04.861204 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/switch-graph-43360-f8cd54dcb-4959p" podUID="6ada0c88-70ac-48e5-9e91-f6fd589b40cb" containerName="switch-graph-43360" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 31 08:13:04.861634 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:04.861303 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-03ede73268/switch-graph-43360-f8cd54dcb-4959p" Oct 31 08:13:06.384621 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:06.384579 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/splitter-graph-d859c-75c5cd656-ffflh" podUID="62d26bfc-7aed-4be2-a24c-33afc28acf05" containerName="splitter-graph-d859c" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 31 08:13:06.385024 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:06.384678 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-03ede73268/splitter-graph-d859c-75c5cd656-ffflh" Oct 31 08:13:07.241524 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:07.241445 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/switch-graph-490af-5ccfdb64c7-mzjwg" podUID="8f678a8c-1a16-41eb-93d1-45c89b51c98d" containerName="switch-graph-490af" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 31 08:13:09.860891 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:09.860850 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/switch-graph-43360-f8cd54dcb-4959p" podUID="6ada0c88-70ac-48e5-9e91-f6fd589b40cb" containerName="switch-graph-43360" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 31 08:13:11.384890 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:11.384841 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/splitter-graph-d859c-75c5cd656-ffflh" podUID="62d26bfc-7aed-4be2-a24c-33afc28acf05" containerName="splitter-graph-d859c" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 31 08:13:12.242237 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:12.242197 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/switch-graph-490af-5ccfdb64c7-mzjwg" podUID="8f678a8c-1a16-41eb-93d1-45c89b51c98d" containerName="switch-graph-490af" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 31 08:13:14.860916 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:14.860879 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/switch-graph-43360-f8cd54dcb-4959p" podUID="6ada0c88-70ac-48e5-9e91-f6fd589b40cb" containerName="switch-graph-43360" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 31 08:13:16.384276 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:16.384240 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/splitter-graph-d859c-75c5cd656-ffflh" podUID="62d26bfc-7aed-4be2-a24c-33afc28acf05" containerName="splitter-graph-d859c" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 31 08:13:17.240794 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:17.240761 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/switch-graph-490af-5ccfdb64c7-mzjwg" podUID="8f678a8c-1a16-41eb-93d1-45c89b51c98d" containerName="switch-graph-490af" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 31 08:13:18.583197 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:18.583163 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:13:19.860857 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:19.860813 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/switch-graph-43360-f8cd54dcb-4959p" podUID="6ada0c88-70ac-48e5-9e91-f6fd589b40cb" containerName="switch-graph-43360" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 31 08:13:21.384145 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:21.384106 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-03ede73268/splitter-graph-d859c-75c5cd656-ffflh" podUID="62d26bfc-7aed-4be2-a24c-33afc28acf05" containerName="splitter-graph-d859c" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 31 08:13:21.792977 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:21.792940 4935 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-03ede73268/splitter-graph-d859c-75c5cd656-ffflh" Oct 31 08:13:21.835843 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:21.835823 4935 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-03ede73268/switch-graph-43360-f8cd54dcb-4959p" Oct 31 08:13:21.838942 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:21.838924 4935 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-03ede73268/switch-graph-490af-5ccfdb64c7-mzjwg" Oct 31 08:13:21.906073 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:21.906015 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6ada0c88-70ac-48e5-9e91-f6fd589b40cb-openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ada0c88-70ac-48e5-9e91-f6fd589b40cb-openshift-service-ca-bundle\") pod \"6ada0c88-70ac-48e5-9e91-f6fd589b40cb\" (UID: \"6ada0c88-70ac-48e5-9e91-f6fd589b40cb\") " Oct 31 08:13:21.906073 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:21.906051 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/62d26bfc-7aed-4be2-a24c-33afc28acf05-proxy-tls\" (UniqueName: \"kubernetes.io/secret/62d26bfc-7aed-4be2-a24c-33afc28acf05-proxy-tls\") pod \"62d26bfc-7aed-4be2-a24c-33afc28acf05\" (UID: \"62d26bfc-7aed-4be2-a24c-33afc28acf05\") " Oct 31 08:13:21.906258 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:21.906081 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6ada0c88-70ac-48e5-9e91-f6fd589b40cb-proxy-tls\" (UniqueName: \"kubernetes.io/secret/6ada0c88-70ac-48e5-9e91-f6fd589b40cb-proxy-tls\") pod \"6ada0c88-70ac-48e5-9e91-f6fd589b40cb\" (UID: \"6ada0c88-70ac-48e5-9e91-f6fd589b40cb\") " Oct 31 08:13:21.906258 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:21.906207 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/62d26bfc-7aed-4be2-a24c-33afc28acf05-openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62d26bfc-7aed-4be2-a24c-33afc28acf05-openshift-service-ca-bundle\") pod \"62d26bfc-7aed-4be2-a24c-33afc28acf05\" (UID: \"62d26bfc-7aed-4be2-a24c-33afc28acf05\") " Oct 31 08:13:21.906394 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:21.906253 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8f678a8c-1a16-41eb-93d1-45c89b51c98d-openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f678a8c-1a16-41eb-93d1-45c89b51c98d-openshift-service-ca-bundle\") pod \"8f678a8c-1a16-41eb-93d1-45c89b51c98d\" (UID: \"8f678a8c-1a16-41eb-93d1-45c89b51c98d\") " Oct 31 08:13:21.906394 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:21.906304 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8f678a8c-1a16-41eb-93d1-45c89b51c98d-proxy-tls\" (UniqueName: \"kubernetes.io/secret/8f678a8c-1a16-41eb-93d1-45c89b51c98d-proxy-tls\") pod \"8f678a8c-1a16-41eb-93d1-45c89b51c98d\" (UID: \"8f678a8c-1a16-41eb-93d1-45c89b51c98d\") " Oct 31 08:13:21.906559 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:21.906535 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ada0c88-70ac-48e5-9e91-f6fd589b40cb-openshift-service-ca-bundle" pod "6ada0c88-70ac-48e5-9e91-f6fd589b40cb" (UID: "6ada0c88-70ac-48e5-9e91-f6fd589b40cb"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 31 08:13:21.906617 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:21.906595 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62d26bfc-7aed-4be2-a24c-33afc28acf05-openshift-service-ca-bundle" pod "62d26bfc-7aed-4be2-a24c-33afc28acf05" (UID: "62d26bfc-7aed-4be2-a24c-33afc28acf05"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 31 08:13:21.906668 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:21.906653 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f678a8c-1a16-41eb-93d1-45c89b51c98d-openshift-service-ca-bundle" pod "8f678a8c-1a16-41eb-93d1-45c89b51c98d" (UID: "8f678a8c-1a16-41eb-93d1-45c89b51c98d"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 31 08:13:21.908368 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:21.908332 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ada0c88-70ac-48e5-9e91-f6fd589b40cb-proxy-tls" pod "6ada0c88-70ac-48e5-9e91-f6fd589b40cb" (UID: "6ada0c88-70ac-48e5-9e91-f6fd589b40cb"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 31 08:13:21.908483 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:21.908462 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62d26bfc-7aed-4be2-a24c-33afc28acf05-proxy-tls" pod "62d26bfc-7aed-4be2-a24c-33afc28acf05" (UID: "62d26bfc-7aed-4be2-a24c-33afc28acf05"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 31 08:13:21.908699 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:21.908674 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f678a8c-1a16-41eb-93d1-45c89b51c98d-proxy-tls" pod "8f678a8c-1a16-41eb-93d1-45c89b51c98d" (UID: "8f678a8c-1a16-41eb-93d1-45c89b51c98d"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 31 08:13:21.987039 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:21.987010 4935 generic.go:356] "Generic (PLEG): container finished" podID="8f678a8c-1a16-41eb-93d1-45c89b51c98d" containerID="e07751231ecd2c1bac7ab9dc1c3119b408b4a7492ce747cab103a356af9d0fe5" exitCode=137 Oct 31 08:13:21.987171 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:21.987053 4935 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-03ede73268/switch-graph-490af-5ccfdb64c7-mzjwg" Oct 31 08:13:21.987171 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:21.987096 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-03ede73268/switch-graph-490af-5ccfdb64c7-mzjwg" event={"ID":"8f678a8c-1a16-41eb-93d1-45c89b51c98d","Type":"ContainerDied","Data":"e07751231ecd2c1bac7ab9dc1c3119b408b4a7492ce747cab103a356af9d0fe5"} Oct 31 08:13:21.987171 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:21.987145 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-03ede73268/switch-graph-490af-5ccfdb64c7-mzjwg" event={"ID":"8f678a8c-1a16-41eb-93d1-45c89b51c98d","Type":"ContainerDied","Data":"8c342a67dfcced990c23afe315b1d68ea919f6a6f9f6fe459e52656f4a136801"} Oct 31 08:13:21.987171 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:21.987171 4935 scope.go:122] "RemoveContainer" containerID="e07751231ecd2c1bac7ab9dc1c3119b408b4a7492ce747cab103a356af9d0fe5" Oct 31 08:13:21.988444 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:21.988424 4935 generic.go:356] "Generic (PLEG): container finished" podID="62d26bfc-7aed-4be2-a24c-33afc28acf05" containerID="5f595ac212c238a5e9d78671d9f0ab0f8a732768b77ffe8d3ff79bcd77e89a89" exitCode=0 Oct 31 08:13:21.988556 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:21.988475 4935 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-03ede73268/splitter-graph-d859c-75c5cd656-ffflh" Oct 31 08:13:21.988625 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:21.988477 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-03ede73268/splitter-graph-d859c-75c5cd656-ffflh" event={"ID":"62d26bfc-7aed-4be2-a24c-33afc28acf05","Type":"ContainerDied","Data":"5f595ac212c238a5e9d78671d9f0ab0f8a732768b77ffe8d3ff79bcd77e89a89"} Oct 31 08:13:21.988625 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:21.988602 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-03ede73268/splitter-graph-d859c-75c5cd656-ffflh" event={"ID":"62d26bfc-7aed-4be2-a24c-33afc28acf05","Type":"ContainerDied","Data":"6955dd3a6de482edbffaf5f466c48584db912df37a7bd1f46adca8f43a193f7f"} Oct 31 08:13:21.989797 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:21.989778 4935 generic.go:356] "Generic (PLEG): container finished" podID="6ada0c88-70ac-48e5-9e91-f6fd589b40cb" containerID="83ab399634f48fa661c5a3d7c2a75ec489fe3201223e5beb7bdb3a881ec08e68" exitCode=0 Oct 31 08:13:21.989900 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:21.989811 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-03ede73268/switch-graph-43360-f8cd54dcb-4959p" event={"ID":"6ada0c88-70ac-48e5-9e91-f6fd589b40cb","Type":"ContainerDied","Data":"83ab399634f48fa661c5a3d7c2a75ec489fe3201223e5beb7bdb3a881ec08e68"} Oct 31 08:13:21.989900 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:21.989831 4935 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-03ede73268/switch-graph-43360-f8cd54dcb-4959p" Oct 31 08:13:21.989900 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:21.989839 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-03ede73268/switch-graph-43360-f8cd54dcb-4959p" event={"ID":"6ada0c88-70ac-48e5-9e91-f6fd589b40cb","Type":"ContainerDied","Data":"91de279a18c859bca6dad8129dd2cf5522bcec4e55606312f8d3d9e7365f2117"} Oct 31 08:13:22.000408 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:22.000384 4935 scope.go:122] "RemoveContainer" containerID="e07751231ecd2c1bac7ab9dc1c3119b408b4a7492ce747cab103a356af9d0fe5" Oct 31 08:13:22.000881 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:13:22.000759 4935 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e07751231ecd2c1bac7ab9dc1c3119b408b4a7492ce747cab103a356af9d0fe5\": container with ID starting with e07751231ecd2c1bac7ab9dc1c3119b408b4a7492ce747cab103a356af9d0fe5 not found: ID does not exist" containerID="e07751231ecd2c1bac7ab9dc1c3119b408b4a7492ce747cab103a356af9d0fe5" Oct 31 08:13:22.000881 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:22.000811 4935 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e07751231ecd2c1bac7ab9dc1c3119b408b4a7492ce747cab103a356af9d0fe5"} err="failed to get container status \"e07751231ecd2c1bac7ab9dc1c3119b408b4a7492ce747cab103a356af9d0fe5\": rpc error: code = NotFound desc = could not find container \"e07751231ecd2c1bac7ab9dc1c3119b408b4a7492ce747cab103a356af9d0fe5\": container with ID starting with e07751231ecd2c1bac7ab9dc1c3119b408b4a7492ce747cab103a356af9d0fe5 not found: ID does not exist" Oct 31 08:13:22.000881 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:22.000831 4935 scope.go:122] "RemoveContainer" containerID="5f595ac212c238a5e9d78671d9f0ab0f8a732768b77ffe8d3ff79bcd77e89a89" Oct 31 08:13:22.006885 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:22.006865 4935 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ada0c88-70ac-48e5-9e91-f6fd589b40cb-openshift-service-ca-bundle\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 08:13:22.006885 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:22.006886 4935 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/62d26bfc-7aed-4be2-a24c-33afc28acf05-proxy-tls\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 08:13:22.007033 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:22.006896 4935 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6ada0c88-70ac-48e5-9e91-f6fd589b40cb-proxy-tls\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 08:13:22.007033 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:22.006904 4935 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62d26bfc-7aed-4be2-a24c-33afc28acf05-openshift-service-ca-bundle\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 08:13:22.007033 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:22.006913 4935 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8f678a8c-1a16-41eb-93d1-45c89b51c98d-openshift-service-ca-bundle\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 08:13:22.007033 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:22.006922 4935 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8f678a8c-1a16-41eb-93d1-45c89b51c98d-proxy-tls\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 08:13:22.009928 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:22.009914 4935 scope.go:122] "RemoveContainer" containerID="5f595ac212c238a5e9d78671d9f0ab0f8a732768b77ffe8d3ff79bcd77e89a89" Oct 31 08:13:22.010164 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:13:22.010148 4935 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f595ac212c238a5e9d78671d9f0ab0f8a732768b77ffe8d3ff79bcd77e89a89\": container with ID starting with 5f595ac212c238a5e9d78671d9f0ab0f8a732768b77ffe8d3ff79bcd77e89a89 not found: ID does not exist" containerID="5f595ac212c238a5e9d78671d9f0ab0f8a732768b77ffe8d3ff79bcd77e89a89" Oct 31 08:13:22.010215 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:22.010170 4935 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f595ac212c238a5e9d78671d9f0ab0f8a732768b77ffe8d3ff79bcd77e89a89"} err="failed to get container status \"5f595ac212c238a5e9d78671d9f0ab0f8a732768b77ffe8d3ff79bcd77e89a89\": rpc error: code = NotFound desc = could not find container \"5f595ac212c238a5e9d78671d9f0ab0f8a732768b77ffe8d3ff79bcd77e89a89\": container with ID starting with 5f595ac212c238a5e9d78671d9f0ab0f8a732768b77ffe8d3ff79bcd77e89a89 not found: ID does not exist" Oct 31 08:13:22.010215 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:22.010183 4935 scope.go:122] "RemoveContainer" containerID="83ab399634f48fa661c5a3d7c2a75ec489fe3201223e5beb7bdb3a881ec08e68" Oct 31 08:13:22.019588 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:22.019568 4935 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-03ede73268/switch-graph-490af-5ccfdb64c7-mzjwg"] Oct 31 08:13:22.022249 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:22.022229 4935 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-03ede73268/switch-graph-490af-5ccfdb64c7-mzjwg"] Oct 31 08:13:22.025017 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:22.024997 4935 scope.go:122] "RemoveContainer" containerID="83ab399634f48fa661c5a3d7c2a75ec489fe3201223e5beb7bdb3a881ec08e68" Oct 31 08:13:22.025242 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:13:22.025227 4935 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83ab399634f48fa661c5a3d7c2a75ec489fe3201223e5beb7bdb3a881ec08e68\": container with ID starting with 83ab399634f48fa661c5a3d7c2a75ec489fe3201223e5beb7bdb3a881ec08e68 not found: ID does not exist" containerID="83ab399634f48fa661c5a3d7c2a75ec489fe3201223e5beb7bdb3a881ec08e68" Oct 31 08:13:22.025307 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:22.025247 4935 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83ab399634f48fa661c5a3d7c2a75ec489fe3201223e5beb7bdb3a881ec08e68"} err="failed to get container status \"83ab399634f48fa661c5a3d7c2a75ec489fe3201223e5beb7bdb3a881ec08e68\": rpc error: code = NotFound desc = could not find container \"83ab399634f48fa661c5a3d7c2a75ec489fe3201223e5beb7bdb3a881ec08e68\": container with ID starting with 83ab399634f48fa661c5a3d7c2a75ec489fe3201223e5beb7bdb3a881ec08e68 not found: ID does not exist" Oct 31 08:13:22.032428 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:22.032408 4935 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-03ede73268/splitter-graph-d859c-75c5cd656-ffflh"] Oct 31 08:13:22.036173 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:22.036154 4935 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-03ede73268/splitter-graph-d859c-75c5cd656-ffflh"] Oct 31 08:13:22.046477 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:22.046458 4935 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-03ede73268/switch-graph-43360-f8cd54dcb-4959p"] Oct 31 08:13:22.050332 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:22.050315 4935 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-03ede73268/switch-graph-43360-f8cd54dcb-4959p"] Oct 31 08:13:22.241810 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:22.241778 4935 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="62d26bfc-7aed-4be2-a24c-33afc28acf05" path="/var/lib/kubelet/pods/62d26bfc-7aed-4be2-a24c-33afc28acf05/volumes" Oct 31 08:13:22.242106 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:22.242094 4935 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6ada0c88-70ac-48e5-9e91-f6fd589b40cb" path="/var/lib/kubelet/pods/6ada0c88-70ac-48e5-9e91-f6fd589b40cb/volumes" Oct 31 08:13:22.242403 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:22.242392 4935 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8f678a8c-1a16-41eb-93d1-45c89b51c98d" path="/var/lib/kubelet/pods/8f678a8c-1a16-41eb-93d1-45c89b51c98d/volumes" Oct 31 08:13:30.365030 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:30.364997 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:13:30.567828 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:30.567793 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:13:30.830222 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:30.830195 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:13:31.454568 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:13:31.454532 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 08:13:36.222620 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:36.222583 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:13:36.423456 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:13:36.423422 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:14:01.454249 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:14:01.454212 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 08:14:18.618546 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:14:18.618507 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:14:31.454439 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:14:31.454400 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 08:14:51.116806 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:14:51.116758 4935 controller.go:251] "Failed to update lease" err="Put \"https://a51a8a8dfd08c47608cb84249ed1c067-21202dd1f9d41abd.elb.us-east-2.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-130-48.us-east-2.compute.internal?timeout=10s\": context deadline exceeded" Oct 31 08:15:01.117862 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:15:01.117821 4935 controller.go:251] "Failed to update lease" err="Put \"https://a51a8a8dfd08c47608cb84249ed1c067-21202dd1f9d41abd.elb.us-east-2.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-130-48.us-east-2.compute.internal?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Oct 31 08:15:01.455130 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:15:01.455092 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 08:15:11.118648 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:15:11.118612 4935 controller.go:251] "Failed to update lease" err="Put \"https://a51a8a8dfd08c47608cb84249ed1c067-21202dd1f9d41abd.elb.us-east-2.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-130-48.us-east-2.compute.internal?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Oct 31 08:15:14.850213 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:14.850182 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:15:15.054131 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.054100 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:15:15.941909 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.941852 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.941909 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.941886 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5t598\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.941909 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.941868 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-bcvgm\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.941909 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.941877 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.941879 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-keda\"/\"kedaorg-certs\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.941859 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"kserve\"/\"llmisvc-webhook-server-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.941955 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.941973 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.941980 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.941978 4935 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942013 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942031 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942035 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-sm468\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942068 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942098 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942105 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942101 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942115 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942131 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942145 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-2schs\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942155 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942163 4935 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942169 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942187 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"e2e-gw0-5c1f590758\"/\"model-chainer-serving-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942189 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942188 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942210 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ssdfd\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942222 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942247 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942253 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942251 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-kqb4j\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942260 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942270 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942276 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"e2e-gw0-5c1f590758\"/\"default-dockercfg-kdf9f\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942290 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"kserve\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942305 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942312 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xb7gt\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942319 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942327 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942339 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942368 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"kube-system\"/\"konnectivity-agent\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942373 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942388 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942392 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"kube-system\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942399 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942404 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942411 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942425 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942431 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942436 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942465 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-dns\"/\"dns-default\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942468 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942478 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942471 4935 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942499 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-jjzj5\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942504 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942507 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942522 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942540 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942541 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942555 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-prf6j\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942559 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942552 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942569 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942572 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-tghnm\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942585 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942601 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942608 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942614 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9wbww\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942630 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-x8m6s\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942623 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-jshc6\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942606 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942646 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"kserve\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942648 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942665 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942648 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942668 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942667 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942681 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942695 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"kube-system\"/\"original-pull-secret\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942698 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"e2e-gw0-5c1f590758\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942707 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-78wjc\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942706 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942717 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942726 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942725 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942745 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942749 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942754 4935 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942754 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942755 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942774 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942784 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942786 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942794 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942805 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.942760 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942811 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.945878 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942815 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"e2e-gw0-5c1f590758\"/\"switch-graph-8ae00-serving-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.945878 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942823 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.945878 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942834 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.945878 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942841 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.945878 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942835 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"e2e-gw0-5c1f590758\"/\"error-404-isvc-8ae00-predictor-serving-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.945878 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942851 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.945878 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942846 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-p6nqs\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.945878 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942855 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"e2e-gw0-5c1f590758\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.945878 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942873 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"kserve\"/\"llmisvc-controller-manager-dockercfg-f2rph\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.945878 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942873 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.945878 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942875 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-hztdx\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.945878 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942874 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.945878 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942900 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.945878 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942910 4935 reflector.go:578] "Warning: watch ended with error" reflector="pkg/kubelet/config/apiserver.go:66" type="*v1.Pod" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.945878 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942912 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-lhm78\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.945878 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942873 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-x6jvt\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.945878 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942904 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-sl77n\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.945878 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942884 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.945878 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942911 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.945878 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942920 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.945878 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942928 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-65sf9\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.945878 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942935 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.945878 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942933 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.945878 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942953 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.945878 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:15.942958 4935 reflector.go:578] "Warning: watch ended with error" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: http2: client connection lost\") has prevented the request from succeeding" Oct 31 08:15:15.945878 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:15:15.942961 4935 controller.go:251] "Failed to update lease" err="Put \"https://a51a8a8dfd08c47608cb84249ed1c067-21202dd1f9d41abd.elb.us-east-2.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-130-48.us-east-2.compute.internal?timeout=10s\": http2: client connection lost" Oct 31 08:15:16.754969 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:16.754941 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Oct 31 08:15:16.758182 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:16.758161 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\"" Oct 31 08:15:16.763473 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:16.763455 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-x6jvt\"" Oct 31 08:15:16.769632 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:16.769614 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Oct 31 08:15:16.770771 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:16.770746 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Oct 31 08:15:16.770869 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:16.770809 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\"" Oct 31 08:15:16.774118 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:16.774102 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-bcvgm\"" Oct 31 08:15:16.776297 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:16.776280 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Oct 31 08:15:16.803493 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:16.803477 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Oct 31 08:15:16.811821 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:16.811804 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-webhook-server-cert\"" Oct 31 08:15:16.831211 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:16.831192 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Oct 31 08:15:16.835673 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:16.835659 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-prf6j\"" Oct 31 08:15:16.846008 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:16.845992 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Oct 31 08:15:16.850462 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:16.850447 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Oct 31 08:15:16.850556 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:16.850497 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-lhm78\"" Oct 31 08:15:16.852797 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:16.852783 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Oct 31 08:15:16.861246 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:16.861229 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Oct 31 08:15:16.864884 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:16.864870 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Oct 31 08:15:16.871450 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:16.871433 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Oct 31 08:15:16.873335 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:16.873319 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Oct 31 08:15:16.873569 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:16.873554 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Oct 31 08:15:16.880921 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:16.880909 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Oct 31 08:15:16.893563 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:16.893544 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-kqb4j\"" Oct 31 08:15:16.895419 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:16.895406 4935 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Oct 31 08:15:16.901911 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:16.901884 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\"" Oct 31 08:15:16.906290 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:16.906277 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5t598\"" Oct 31 08:15:16.912637 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:16.912621 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Oct 31 08:15:16.917024 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:16.917009 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\"" Oct 31 08:15:16.917984 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:16.917972 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Oct 31 08:15:16.931419 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:16.931401 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Oct 31 08:15:16.934682 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:16.934668 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Oct 31 08:15:16.939992 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:16.939979 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-controller-manager-dockercfg-f2rph\"" Oct 31 08:15:16.941057 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:16.941042 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-78wjc\"" Oct 31 08:15:16.941141 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:16.941126 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Oct 31 08:15:16.948525 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:16.948508 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-5c1f590758\"/\"model-chainer-serving-cert\"" Oct 31 08:15:16.954904 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:16.954889 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Oct 31 08:15:16.961544 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:16.961529 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-5c1f590758\"/\"error-404-isvc-8ae00-predictor-serving-cert\"" Oct 31 08:15:16.962519 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:16.962506 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Oct 31 08:15:16.967933 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:16.967922 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-65sf9\"" Oct 31 08:15:16.973296 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:16.973283 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Oct 31 08:15:16.974141 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:16.974128 4935 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Oct 31 08:15:16.976459 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:16.976445 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Oct 31 08:15:16.985902 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:16.985888 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Oct 31 08:15:16.985962 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:16.985950 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Oct 31 08:15:16.993257 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:16.993244 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Oct 31 08:15:16.994298 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:16.994284 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Oct 31 08:15:16.996584 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:16.996574 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\"" Oct 31 08:15:17.009082 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.009039 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-5c1f590758\"/\"switch-graph-8ae00-serving-cert\"" Oct 31 08:15:17.013411 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.013396 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-jshc6\"" Oct 31 08:15:17.015641 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.015628 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Oct 31 08:15:17.016588 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.016576 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-p6nqs\"" Oct 31 08:15:17.034186 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.034167 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Oct 31 08:15:17.037583 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.037571 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-x8m6s\"" Oct 31 08:15:17.048059 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.048042 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Oct 31 08:15:17.083880 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.083863 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Oct 31 08:15:17.112873 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.112857 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Oct 31 08:15:17.112873 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.112861 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\"" Oct 31 08:15:17.113153 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.113138 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Oct 31 08:15:17.133375 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.133339 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-5c1f590758\"/\"kube-root-ca.crt\"" Oct 31 08:15:17.138821 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.138808 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Oct 31 08:15:17.139685 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.139671 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Oct 31 08:15:17.147517 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.147493 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Oct 31 08:15:17.147602 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.147537 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\"" Oct 31 08:15:17.147602 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.147558 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Oct 31 08:15:17.147928 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.147912 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Oct 31 08:15:17.150406 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.150393 4935 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161" Oct 31 08:15:17.156174 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.156153 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-5c1f590758\"/\"default-dockercfg-kdf9f\"" Oct 31 08:15:17.162640 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.162617 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Oct 31 08:15:17.167199 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.167179 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Oct 31 08:15:17.181572 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.181557 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Oct 31 08:15:17.181642 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.181618 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Oct 31 08:15:17.200299 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.200278 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\"" Oct 31 08:15:17.202643 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.202629 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ssdfd\"" Oct 31 08:15:17.212150 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.212135 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-tghnm\"" Oct 31 08:15:17.220540 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.220523 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Oct 31 08:15:17.224005 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.223989 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\"" Oct 31 08:15:17.229287 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.229274 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Oct 31 08:15:17.232666 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.232652 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-hztdx\"" Oct 31 08:15:17.233553 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.233540 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Oct 31 08:15:17.236853 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.236842 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-sm468\"" Oct 31 08:15:17.246065 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.246051 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Oct 31 08:15:17.247972 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.247958 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Oct 31 08:15:17.248083 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.248072 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Oct 31 08:15:17.266890 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.266850 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Oct 31 08:15:17.268918 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.268905 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Oct 31 08:15:17.271120 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.271106 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Oct 31 08:15:17.300629 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.298048 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Oct 31 08:15:17.318358 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.318328 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xb7gt\"" Oct 31 08:15:17.337201 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.337184 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Oct 31 08:15:17.340491 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.340472 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Oct 31 08:15:17.351989 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.351974 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Oct 31 08:15:17.354337 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.354317 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Oct 31 08:15:17.357626 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.357600 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Oct 31 08:15:17.357707 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.357689 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Oct 31 08:15:17.363132 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.363117 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Oct 31 08:15:17.363195 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.363136 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Oct 31 08:15:17.372943 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.372929 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-jjzj5\"" Oct 31 08:15:17.387856 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.387839 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Oct 31 08:15:17.388454 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.388441 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\"" Oct 31 08:15:17.391668 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.391655 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Oct 31 08:15:17.392829 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.392813 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-2schs\"" Oct 31 08:15:17.392882 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.392855 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Oct 31 08:15:17.402551 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.402535 4935 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Oct 31 08:15:17.413051 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.413031 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9wbww\"" Oct 31 08:15:17.417622 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.417606 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Oct 31 08:15:17.419809 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.419796 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Oct 31 08:15:17.423126 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.423113 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Oct 31 08:15:17.436722 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.436706 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-5c1f590758\"/\"openshift-service-ca.crt\"" Oct 31 08:15:17.453844 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.453796 4935 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Oct 31 08:15:17.454008 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.453994 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Oct 31 08:15:17.457034 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.457018 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Oct 31 08:15:17.463737 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.463721 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Oct 31 08:15:17.481660 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.481641 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Oct 31 08:15:17.503625 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.503609 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Oct 31 08:15:17.505454 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.505441 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Oct 31 08:15:17.506926 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.506913 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Oct 31 08:15:17.508875 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.508862 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Oct 31 08:15:17.509127 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.509115 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Oct 31 08:15:17.523703 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.523653 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Oct 31 08:15:17.529256 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.529240 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Oct 31 08:15:17.530204 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.530187 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-sl77n\"" Oct 31 08:15:17.538811 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:17.538795 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\"" Oct 31 08:15:18.657302 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:18.657266 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:15:22.627302 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:22.627270 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:15:22.828945 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:22.828914 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:15:23.090765 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:23.090729 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:15:23.288765 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:23.288717 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:15:26.457787 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:26.457744 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jztcz_396e1c5e-3700-4fe6-b367-54ad2a244225/console-operator/2.log" Oct 31 08:15:26.463797 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:15:26.463776 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jztcz_396e1c5e-3700-4fe6-b367-54ad2a244225/console-operator/2.log" Oct 31 08:15:31.454534 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:15:31.454497 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 08:16:01.455186 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:16:01.455148 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 08:16:18.724634 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:16:18.724551 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:16:31.453649 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:16:31.453605 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 08:17:01.454914 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:17:01.454877 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 08:17:18.775676 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:17:18.775637 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:17:31.454274 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:17:31.454238 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 08:18:01.454047 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:18:01.454011 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 08:18:18.874535 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:18:18.874490 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:18:31.454953 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:18:31.454918 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 08:19:01.455075 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:19:01.455036 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 08:19:18.791377 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:19:18.791280 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:19:31.454641 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:19:31.454601 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 08:19:59.337204 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:19:59.337175 4935 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-5c1f590758/error-404-isvc-8ae00-predictor-7f849b78d-rjwg7"] Oct 31 08:19:59.337749 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:19:59.337417 4935 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-5c1f590758/error-404-isvc-8ae00-predictor-7f849b78d-rjwg7" podUID="364dfb0e-d451-4015-b577-785b0ae174b4" containerName="kserve-container" containerID="cri-o://a5e7f2c10a96a42de3eaec7705f1f9fae5ff95456a708c1a270820bc74e1d757" gracePeriod=30 Oct 31 08:19:59.337749 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:19:59.337473 4935 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-5c1f590758/error-404-isvc-8ae00-predictor-7f849b78d-rjwg7" podUID="364dfb0e-d451-4015-b577-785b0ae174b4" containerName="kube-rbac-proxy" containerID="cri-o://58243a78d18ef50a032cf3149324285c64bde508a152a6f4722661d5e8a2a66e" gracePeriod=30 Oct 31 08:19:59.494646 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:19:59.494613 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:19:59.580809 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:19:59.580778 4935 generic.go:356] "Generic (PLEG): container finished" podID="364dfb0e-d451-4015-b577-785b0ae174b4" containerID="58243a78d18ef50a032cf3149324285c64bde508a152a6f4722661d5e8a2a66e" exitCode=0 Oct 31 08:19:59.580986 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:19:59.580851 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-5c1f590758/error-404-isvc-8ae00-predictor-7f849b78d-rjwg7" event={"ID":"364dfb0e-d451-4015-b577-785b0ae174b4","Type":"ContainerDied","Data":"58243a78d18ef50a032cf3149324285c64bde508a152a6f4722661d5e8a2a66e"} Oct 31 08:19:59.705056 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:19:59.705022 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:20:01.259634 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:01.259594 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-5c1f590758/error-404-isvc-8ae00-predictor-7f849b78d-rjwg7" podUID="364dfb0e-d451-4015-b577-785b0ae174b4" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.40:8643/healthz\": dial tcp 10.132.0.40:8643: connect: connection refused" Oct 31 08:20:01.453727 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:20:01.453688 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 08:20:02.388447 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:02.388417 4935 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-5c1f590758/error-404-isvc-8ae00-predictor-7f849b78d-rjwg7" Oct 31 08:20:02.505645 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:02.505557 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/364dfb0e-d451-4015-b577-785b0ae174b4-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/364dfb0e-d451-4015-b577-785b0ae174b4-kube-rbac-proxy-sar-config\") pod \"364dfb0e-d451-4015-b577-785b0ae174b4\" (UID: \"364dfb0e-d451-4015-b577-785b0ae174b4\") " Oct 31 08:20:02.505645 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:02.505600 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/364dfb0e-d451-4015-b577-785b0ae174b4-kube-api-access-rmprj\" (UniqueName: \"kubernetes.io/projected/364dfb0e-d451-4015-b577-785b0ae174b4-kube-api-access-rmprj\") pod \"364dfb0e-d451-4015-b577-785b0ae174b4\" (UID: \"364dfb0e-d451-4015-b577-785b0ae174b4\") " Oct 31 08:20:02.505645 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:02.505627 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/364dfb0e-d451-4015-b577-785b0ae174b4-proxy-tls\" (UniqueName: \"kubernetes.io/secret/364dfb0e-d451-4015-b577-785b0ae174b4-proxy-tls\") pod \"364dfb0e-d451-4015-b577-785b0ae174b4\" (UID: \"364dfb0e-d451-4015-b577-785b0ae174b4\") " Oct 31 08:20:02.506015 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:02.505984 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/364dfb0e-d451-4015-b577-785b0ae174b4-kube-rbac-proxy-sar-config" pod "364dfb0e-d451-4015-b577-785b0ae174b4" (UID: "364dfb0e-d451-4015-b577-785b0ae174b4"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 31 08:20:02.507889 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:02.507869 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/364dfb0e-d451-4015-b577-785b0ae174b4-proxy-tls" pod "364dfb0e-d451-4015-b577-785b0ae174b4" (UID: "364dfb0e-d451-4015-b577-785b0ae174b4"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 31 08:20:02.508004 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:02.507985 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/364dfb0e-d451-4015-b577-785b0ae174b4-kube-api-access-rmprj" pod "364dfb0e-d451-4015-b577-785b0ae174b4" (UID: "364dfb0e-d451-4015-b577-785b0ae174b4"). InnerVolumeSpecName "kube-api-access-rmprj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 31 08:20:02.594341 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:02.594310 4935 generic.go:356] "Generic (PLEG): container finished" podID="364dfb0e-d451-4015-b577-785b0ae174b4" containerID="a5e7f2c10a96a42de3eaec7705f1f9fae5ff95456a708c1a270820bc74e1d757" exitCode=0 Oct 31 08:20:02.594550 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:02.594375 4935 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-5c1f590758/error-404-isvc-8ae00-predictor-7f849b78d-rjwg7" Oct 31 08:20:02.594550 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:02.594383 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-5c1f590758/error-404-isvc-8ae00-predictor-7f849b78d-rjwg7" event={"ID":"364dfb0e-d451-4015-b577-785b0ae174b4","Type":"ContainerDied","Data":"a5e7f2c10a96a42de3eaec7705f1f9fae5ff95456a708c1a270820bc74e1d757"} Oct 31 08:20:02.594550 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:02.594430 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-5c1f590758/error-404-isvc-8ae00-predictor-7f849b78d-rjwg7" event={"ID":"364dfb0e-d451-4015-b577-785b0ae174b4","Type":"ContainerDied","Data":"32e30d452d7f26ed9ecf9737b1d7619484c1e3024ed8e07bbf36acd5fc60b715"} Oct 31 08:20:02.594550 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:02.594452 4935 scope.go:122] "RemoveContainer" containerID="58243a78d18ef50a032cf3149324285c64bde508a152a6f4722661d5e8a2a66e" Oct 31 08:20:02.603485 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:02.603469 4935 scope.go:122] "RemoveContainer" containerID="a5e7f2c10a96a42de3eaec7705f1f9fae5ff95456a708c1a270820bc74e1d757" Oct 31 08:20:02.606123 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:02.606105 4935 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/364dfb0e-d451-4015-b577-785b0ae174b4-kube-rbac-proxy-sar-config\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 08:20:02.606123 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:02.606124 4935 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rmprj\" (UniqueName: \"kubernetes.io/projected/364dfb0e-d451-4015-b577-785b0ae174b4-kube-api-access-rmprj\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 08:20:02.606242 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:02.606135 4935 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/364dfb0e-d451-4015-b577-785b0ae174b4-proxy-tls\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 08:20:02.610775 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:02.610757 4935 scope.go:122] "RemoveContainer" containerID="58243a78d18ef50a032cf3149324285c64bde508a152a6f4722661d5e8a2a66e" Oct 31 08:20:02.611037 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:20:02.611020 4935 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58243a78d18ef50a032cf3149324285c64bde508a152a6f4722661d5e8a2a66e\": container with ID starting with 58243a78d18ef50a032cf3149324285c64bde508a152a6f4722661d5e8a2a66e not found: ID does not exist" containerID="58243a78d18ef50a032cf3149324285c64bde508a152a6f4722661d5e8a2a66e" Oct 31 08:20:02.611079 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:02.611045 4935 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58243a78d18ef50a032cf3149324285c64bde508a152a6f4722661d5e8a2a66e"} err="failed to get container status \"58243a78d18ef50a032cf3149324285c64bde508a152a6f4722661d5e8a2a66e\": rpc error: code = NotFound desc = could not find container \"58243a78d18ef50a032cf3149324285c64bde508a152a6f4722661d5e8a2a66e\": container with ID starting with 58243a78d18ef50a032cf3149324285c64bde508a152a6f4722661d5e8a2a66e not found: ID does not exist" Oct 31 08:20:02.611079 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:02.611063 4935 scope.go:122] "RemoveContainer" containerID="a5e7f2c10a96a42de3eaec7705f1f9fae5ff95456a708c1a270820bc74e1d757" Oct 31 08:20:02.611317 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:20:02.611292 4935 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5e7f2c10a96a42de3eaec7705f1f9fae5ff95456a708c1a270820bc74e1d757\": container with ID starting with a5e7f2c10a96a42de3eaec7705f1f9fae5ff95456a708c1a270820bc74e1d757 not found: ID does not exist" containerID="a5e7f2c10a96a42de3eaec7705f1f9fae5ff95456a708c1a270820bc74e1d757" Oct 31 08:20:02.611384 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:02.611324 4935 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5e7f2c10a96a42de3eaec7705f1f9fae5ff95456a708c1a270820bc74e1d757"} err="failed to get container status \"a5e7f2c10a96a42de3eaec7705f1f9fae5ff95456a708c1a270820bc74e1d757\": rpc error: code = NotFound desc = could not find container \"a5e7f2c10a96a42de3eaec7705f1f9fae5ff95456a708c1a270820bc74e1d757\": container with ID starting with a5e7f2c10a96a42de3eaec7705f1f9fae5ff95456a708c1a270820bc74e1d757 not found: ID does not exist" Oct 31 08:20:02.616216 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:02.616197 4935 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-5c1f590758/error-404-isvc-8ae00-predictor-7f849b78d-rjwg7"] Oct 31 08:20:02.620416 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:02.620398 4935 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-5c1f590758/error-404-isvc-8ae00-predictor-7f849b78d-rjwg7"] Oct 31 08:20:04.242485 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:04.242456 4935 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="364dfb0e-d451-4015-b577-785b0ae174b4" path="/var/lib/kubelet/pods/364dfb0e-d451-4015-b577-785b0ae174b4/volumes" Oct 31 08:20:04.594827 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:04.594734 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:20:06.222722 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:06.222679 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:20:06.426850 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:06.426821 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:20:09.459203 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:09.459170 4935 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-5c1f590758/model-chainer-6b979c876d-j6srw"] Oct 31 08:20:09.459673 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:09.459375 4935 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-5c1f590758/model-chainer-6b979c876d-j6srw" podUID="5bbf5852-c16d-41a1-b2c1-b9a6ecddd898" containerName="model-chainer" containerID="cri-o://9c3a1f01995027b8648284d8dea326546e8e862c2baa8a4e4e5031a19c30be96" gracePeriod=30 Oct 31 08:20:09.481919 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:09.481888 4935 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-5c1f590758/switch-graph-8ae00-86769987cd-rhthk"] Oct 31 08:20:09.482101 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:09.482077 4935 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-5c1f590758/switch-graph-8ae00-86769987cd-rhthk" podUID="9c26d8ba-fd9e-424a-acbf-1836ba32033a" containerName="switch-graph-8ae00" containerID="cri-o://d8ad9cab2c34e0db0bd76eb00814734f5331f6f8cf012a10d341b3ee4ee22118" gracePeriod=30 Oct 31 08:20:09.660464 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:09.660365 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:20:10.569667 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:10.569631 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-5c1f590758/switch-graph-8ae00-86769987cd-rhthk" podUID="9c26d8ba-fd9e-424a-acbf-1836ba32033a" containerName="switch-graph-8ae00" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 31 08:20:11.013696 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:11.013665 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-5c1f590758/model-chainer-6b979c876d-j6srw" podUID="5bbf5852-c16d-41a1-b2c1-b9a6ecddd898" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 31 08:20:15.570758 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:15.570715 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-5c1f590758/switch-graph-8ae00-86769987cd-rhthk" podUID="9c26d8ba-fd9e-424a-acbf-1836ba32033a" containerName="switch-graph-8ae00" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 31 08:20:16.015939 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:16.015893 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-5c1f590758/model-chainer-6b979c876d-j6srw" podUID="5bbf5852-c16d-41a1-b2c1-b9a6ecddd898" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 31 08:20:18.919503 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:18.919469 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:20:20.570088 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:20.570050 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-5c1f590758/switch-graph-8ae00-86769987cd-rhthk" podUID="9c26d8ba-fd9e-424a-acbf-1836ba32033a" containerName="switch-graph-8ae00" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 31 08:20:20.570499 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:20.570158 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-5c1f590758/switch-graph-8ae00-86769987cd-rhthk" Oct 31 08:20:21.013377 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:21.013319 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-5c1f590758/model-chainer-6b979c876d-j6srw" podUID="5bbf5852-c16d-41a1-b2c1-b9a6ecddd898" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 31 08:20:21.013567 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:21.013434 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-5c1f590758/model-chainer-6b979c876d-j6srw" Oct 31 08:20:23.091133 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:23.091108 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-5c1f590758_model-chainer-6b979c876d-j6srw_5bbf5852-c16d-41a1-b2c1-b9a6ecddd898/model-chainer/0.log" Oct 31 08:20:23.226095 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:23.226069 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-5c1f590758_switch-graph-8ae00-86769987cd-rhthk_9c26d8ba-fd9e-424a-acbf-1836ba32033a/switch-graph-8ae00/0.log" Oct 31 08:20:23.917107 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:23.917080 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-5c1f590758_model-chainer-6b979c876d-j6srw_5bbf5852-c16d-41a1-b2c1-b9a6ecddd898/model-chainer/0.log" Oct 31 08:20:23.976752 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:23.976724 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-5c1f590758_switch-graph-8ae00-86769987cd-rhthk_9c26d8ba-fd9e-424a-acbf-1836ba32033a/switch-graph-8ae00/0.log" Oct 31 08:20:24.659400 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:24.659366 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-5c1f590758_model-chainer-6b979c876d-j6srw_5bbf5852-c16d-41a1-b2c1-b9a6ecddd898/model-chainer/0.log" Oct 31 08:20:24.719059 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:24.719033 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-5c1f590758_switch-graph-8ae00-86769987cd-rhthk_9c26d8ba-fd9e-424a-acbf-1836ba32033a/switch-graph-8ae00/0.log" Oct 31 08:20:25.387823 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:25.387800 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-5c1f590758_model-chainer-6b979c876d-j6srw_5bbf5852-c16d-41a1-b2c1-b9a6ecddd898/model-chainer/0.log" Oct 31 08:20:25.449221 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:25.449194 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-5c1f590758_switch-graph-8ae00-86769987cd-rhthk_9c26d8ba-fd9e-424a-acbf-1836ba32033a/switch-graph-8ae00/0.log" Oct 31 08:20:25.569701 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:25.569671 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-5c1f590758/switch-graph-8ae00-86769987cd-rhthk" podUID="9c26d8ba-fd9e-424a-acbf-1836ba32033a" containerName="switch-graph-8ae00" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 31 08:20:26.013630 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:26.013591 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-5c1f590758/model-chainer-6b979c876d-j6srw" podUID="5bbf5852-c16d-41a1-b2c1-b9a6ecddd898" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 31 08:20:26.128144 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:26.128110 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-5c1f590758_model-chainer-6b979c876d-j6srw_5bbf5852-c16d-41a1-b2c1-b9a6ecddd898/model-chainer/0.log" Oct 31 08:20:26.188709 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:26.188663 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-5c1f590758_switch-graph-8ae00-86769987cd-rhthk_9c26d8ba-fd9e-424a-acbf-1836ba32033a/switch-graph-8ae00/0.log" Oct 31 08:20:26.482805 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:26.482782 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jztcz_396e1c5e-3700-4fe6-b367-54ad2a244225/console-operator/2.log" Oct 31 08:20:26.487798 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:26.487777 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jztcz_396e1c5e-3700-4fe6-b367-54ad2a244225/console-operator/2.log" Oct 31 08:20:26.860494 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:26.860415 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-5c1f590758_model-chainer-6b979c876d-j6srw_5bbf5852-c16d-41a1-b2c1-b9a6ecddd898/model-chainer/0.log" Oct 31 08:20:26.921824 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:26.921792 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-5c1f590758_switch-graph-8ae00-86769987cd-rhthk_9c26d8ba-fd9e-424a-acbf-1836ba32033a/switch-graph-8ae00/0.log" Oct 31 08:20:27.597274 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:27.597235 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-5c1f590758_model-chainer-6b979c876d-j6srw_5bbf5852-c16d-41a1-b2c1-b9a6ecddd898/model-chainer/0.log" Oct 31 08:20:27.657735 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:27.657707 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-5c1f590758_switch-graph-8ae00-86769987cd-rhthk_9c26d8ba-fd9e-424a-acbf-1836ba32033a/switch-graph-8ae00/0.log" Oct 31 08:20:28.320329 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:28.320302 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-5c1f590758_model-chainer-6b979c876d-j6srw_5bbf5852-c16d-41a1-b2c1-b9a6ecddd898/model-chainer/0.log" Oct 31 08:20:28.380407 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:28.380380 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-5c1f590758_switch-graph-8ae00-86769987cd-rhthk_9c26d8ba-fd9e-424a-acbf-1836ba32033a/switch-graph-8ae00/0.log" Oct 31 08:20:29.056716 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:29.056688 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-5c1f590758_model-chainer-6b979c876d-j6srw_5bbf5852-c16d-41a1-b2c1-b9a6ecddd898/model-chainer/0.log" Oct 31 08:20:29.114041 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:29.114016 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-5c1f590758_switch-graph-8ae00-86769987cd-rhthk_9c26d8ba-fd9e-424a-acbf-1836ba32033a/switch-graph-8ae00/0.log" Oct 31 08:20:29.780570 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:29.780540 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-5c1f590758_model-chainer-6b979c876d-j6srw_5bbf5852-c16d-41a1-b2c1-b9a6ecddd898/model-chainer/0.log" Oct 31 08:20:29.841158 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:29.841126 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-5c1f590758_switch-graph-8ae00-86769987cd-rhthk_9c26d8ba-fd9e-424a-acbf-1836ba32033a/switch-graph-8ae00/0.log" Oct 31 08:20:30.509839 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:30.509813 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-5c1f590758_model-chainer-6b979c876d-j6srw_5bbf5852-c16d-41a1-b2c1-b9a6ecddd898/model-chainer/0.log" Oct 31 08:20:30.567548 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:30.567526 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-5c1f590758_switch-graph-8ae00-86769987cd-rhthk_9c26d8ba-fd9e-424a-acbf-1836ba32033a/switch-graph-8ae00/0.log" Oct 31 08:20:30.570079 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:30.570053 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-5c1f590758/switch-graph-8ae00-86769987cd-rhthk" podUID="9c26d8ba-fd9e-424a-acbf-1836ba32033a" containerName="switch-graph-8ae00" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 31 08:20:31.013698 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:31.013666 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-5c1f590758/model-chainer-6b979c876d-j6srw" podUID="5bbf5852-c16d-41a1-b2c1-b9a6ecddd898" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 31 08:20:31.235021 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:31.234990 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-5c1f590758_model-chainer-6b979c876d-j6srw_5bbf5852-c16d-41a1-b2c1-b9a6ecddd898/model-chainer/0.log" Oct 31 08:20:31.291814 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:31.291740 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-5c1f590758_switch-graph-8ae00-86769987cd-rhthk_9c26d8ba-fd9e-424a-acbf-1836ba32033a/switch-graph-8ae00/0.log" Oct 31 08:20:31.453666 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:20:31.453636 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 08:20:31.958028 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:31.957993 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-5c1f590758_model-chainer-6b979c876d-j6srw_5bbf5852-c16d-41a1-b2c1-b9a6ecddd898/model-chainer/0.log" Oct 31 08:20:32.084317 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:32.084293 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-5c1f590758_switch-graph-8ae00-86769987cd-rhthk_9c26d8ba-fd9e-424a-acbf-1836ba32033a/switch-graph-8ae00/0.log" Oct 31 08:20:32.800642 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:32.800613 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-5c1f590758_model-chainer-6b979c876d-j6srw_5bbf5852-c16d-41a1-b2c1-b9a6ecddd898/model-chainer/0.log" Oct 31 08:20:32.866730 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:32.866710 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-5c1f590758_switch-graph-8ae00-86769987cd-rhthk_9c26d8ba-fd9e-424a-acbf-1836ba32033a/switch-graph-8ae00/0.log" Oct 31 08:20:33.570233 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:33.570206 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-5c1f590758_model-chainer-6b979c876d-j6srw_5bbf5852-c16d-41a1-b2c1-b9a6ecddd898/model-chainer/0.log" Oct 31 08:20:33.634596 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:33.634569 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-5c1f590758_switch-graph-8ae00-86769987cd-rhthk_9c26d8ba-fd9e-424a-acbf-1836ba32033a/switch-graph-8ae00/0.log" Oct 31 08:20:34.359832 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:34.359804 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-5c1f590758_model-chainer-6b979c876d-j6srw_5bbf5852-c16d-41a1-b2c1-b9a6ecddd898/model-chainer/0.log" Oct 31 08:20:34.424145 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:34.424117 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-5c1f590758_switch-graph-8ae00-86769987cd-rhthk_9c26d8ba-fd9e-424a-acbf-1836ba32033a/switch-graph-8ae00/0.log" Oct 31 08:20:35.157788 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:35.157764 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-5c1f590758_model-chainer-6b979c876d-j6srw_5bbf5852-c16d-41a1-b2c1-b9a6ecddd898/model-chainer/0.log" Oct 31 08:20:35.223801 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:35.223773 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-5c1f590758_switch-graph-8ae00-86769987cd-rhthk_9c26d8ba-fd9e-424a-acbf-1836ba32033a/switch-graph-8ae00/0.log" Oct 31 08:20:35.570047 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:35.570003 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-5c1f590758/switch-graph-8ae00-86769987cd-rhthk" podUID="9c26d8ba-fd9e-424a-acbf-1836ba32033a" containerName="switch-graph-8ae00" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 31 08:20:35.990229 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:35.990201 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-5c1f590758_model-chainer-6b979c876d-j6srw_5bbf5852-c16d-41a1-b2c1-b9a6ecddd898/model-chainer/0.log" Oct 31 08:20:36.013517 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:36.013492 4935 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-5c1f590758/model-chainer-6b979c876d-j6srw" podUID="5bbf5852-c16d-41a1-b2c1-b9a6ecddd898" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 31 08:20:36.051930 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:36.051896 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-5c1f590758_switch-graph-8ae00-86769987cd-rhthk_9c26d8ba-fd9e-424a-acbf-1836ba32033a/switch-graph-8ae00/0.log" Oct 31 08:20:36.738004 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:36.737978 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-5c1f590758_model-chainer-6b979c876d-j6srw_5bbf5852-c16d-41a1-b2c1-b9a6ecddd898/model-chainer/0.log" Oct 31 08:20:36.798520 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:36.798496 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-5c1f590758_switch-graph-8ae00-86769987cd-rhthk_9c26d8ba-fd9e-424a-acbf-1836ba32033a/switch-graph-8ae00/0.log" Oct 31 08:20:37.465553 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:37.465525 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-5c1f590758_model-chainer-6b979c876d-j6srw_5bbf5852-c16d-41a1-b2c1-b9a6ecddd898/model-chainer/0.log" Oct 31 08:20:37.526122 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:37.526096 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-5c1f590758_switch-graph-8ae00-86769987cd-rhthk_9c26d8ba-fd9e-424a-acbf-1836ba32033a/switch-graph-8ae00/0.log" Oct 31 08:20:38.208599 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:38.208573 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-5c1f590758_model-chainer-6b979c876d-j6srw_5bbf5852-c16d-41a1-b2c1-b9a6ecddd898/model-chainer/0.log" Oct 31 08:20:38.270651 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:38.270617 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-5c1f590758_switch-graph-8ae00-86769987cd-rhthk_9c26d8ba-fd9e-424a-acbf-1836ba32033a/switch-graph-8ae00/0.log" Oct 31 08:20:39.659366 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:39.659334 4935 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-5c1f590758/switch-graph-8ae00-86769987cd-rhthk" Oct 31 08:20:39.747551 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:39.747478 4935 generic.go:356] "Generic (PLEG): container finished" podID="5bbf5852-c16d-41a1-b2c1-b9a6ecddd898" containerID="9c3a1f01995027b8648284d8dea326546e8e862c2baa8a4e4e5031a19c30be96" exitCode=0 Oct 31 08:20:39.747697 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:39.747548 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-5c1f590758/model-chainer-6b979c876d-j6srw" event={"ID":"5bbf5852-c16d-41a1-b2c1-b9a6ecddd898","Type":"ContainerDied","Data":"9c3a1f01995027b8648284d8dea326546e8e862c2baa8a4e4e5031a19c30be96"} Oct 31 08:20:39.748840 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:39.748819 4935 generic.go:356] "Generic (PLEG): container finished" podID="9c26d8ba-fd9e-424a-acbf-1836ba32033a" containerID="d8ad9cab2c34e0db0bd76eb00814734f5331f6f8cf012a10d341b3ee4ee22118" exitCode=137 Oct 31 08:20:39.748976 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:39.748855 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-5c1f590758/switch-graph-8ae00-86769987cd-rhthk" event={"ID":"9c26d8ba-fd9e-424a-acbf-1836ba32033a","Type":"ContainerDied","Data":"d8ad9cab2c34e0db0bd76eb00814734f5331f6f8cf012a10d341b3ee4ee22118"} Oct 31 08:20:39.748976 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:39.748878 4935 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-5c1f590758/switch-graph-8ae00-86769987cd-rhthk" Oct 31 08:20:39.748976 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:39.748891 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-5c1f590758/switch-graph-8ae00-86769987cd-rhthk" event={"ID":"9c26d8ba-fd9e-424a-acbf-1836ba32033a","Type":"ContainerDied","Data":"7d1cd625406ecc057743f7b2a2f03265289dbd3e44c2ac8861844b0b8340fd07"} Oct 31 08:20:39.748976 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:39.748914 4935 scope.go:122] "RemoveContainer" containerID="d8ad9cab2c34e0db0bd76eb00814734f5331f6f8cf012a10d341b3ee4ee22118" Oct 31 08:20:39.757020 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:39.757000 4935 scope.go:122] "RemoveContainer" containerID="d8ad9cab2c34e0db0bd76eb00814734f5331f6f8cf012a10d341b3ee4ee22118" Oct 31 08:20:39.757297 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:20:39.757278 4935 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8ad9cab2c34e0db0bd76eb00814734f5331f6f8cf012a10d341b3ee4ee22118\": container with ID starting with d8ad9cab2c34e0db0bd76eb00814734f5331f6f8cf012a10d341b3ee4ee22118 not found: ID does not exist" containerID="d8ad9cab2c34e0db0bd76eb00814734f5331f6f8cf012a10d341b3ee4ee22118" Oct 31 08:20:39.757384 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:39.757306 4935 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8ad9cab2c34e0db0bd76eb00814734f5331f6f8cf012a10d341b3ee4ee22118"} err="failed to get container status \"d8ad9cab2c34e0db0bd76eb00814734f5331f6f8cf012a10d341b3ee4ee22118\": rpc error: code = NotFound desc = could not find container \"d8ad9cab2c34e0db0bd76eb00814734f5331f6f8cf012a10d341b3ee4ee22118\": container with ID starting with d8ad9cab2c34e0db0bd76eb00814734f5331f6f8cf012a10d341b3ee4ee22118 not found: ID does not exist" Oct 31 08:20:39.782555 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:39.782536 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9c26d8ba-fd9e-424a-acbf-1836ba32033a-openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c26d8ba-fd9e-424a-acbf-1836ba32033a-openshift-service-ca-bundle\") pod \"9c26d8ba-fd9e-424a-acbf-1836ba32033a\" (UID: \"9c26d8ba-fd9e-424a-acbf-1836ba32033a\") " Oct 31 08:20:39.782662 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:39.782604 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9c26d8ba-fd9e-424a-acbf-1836ba32033a-proxy-tls\" (UniqueName: \"kubernetes.io/secret/9c26d8ba-fd9e-424a-acbf-1836ba32033a-proxy-tls\") pod \"9c26d8ba-fd9e-424a-acbf-1836ba32033a\" (UID: \"9c26d8ba-fd9e-424a-acbf-1836ba32033a\") " Oct 31 08:20:39.782870 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:39.782849 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c26d8ba-fd9e-424a-acbf-1836ba32033a-openshift-service-ca-bundle" pod "9c26d8ba-fd9e-424a-acbf-1836ba32033a" (UID: "9c26d8ba-fd9e-424a-acbf-1836ba32033a"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 31 08:20:39.784656 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:39.784632 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c26d8ba-fd9e-424a-acbf-1836ba32033a-proxy-tls" pod "9c26d8ba-fd9e-424a-acbf-1836ba32033a" (UID: "9c26d8ba-fd9e-424a-acbf-1836ba32033a"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 31 08:20:39.883094 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:39.883070 4935 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9c26d8ba-fd9e-424a-acbf-1836ba32033a-proxy-tls\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 08:20:39.883094 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:39.883091 4935 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c26d8ba-fd9e-424a-acbf-1836ba32033a-openshift-service-ca-bundle\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 08:20:40.071184 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:40.071157 4935 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-5c1f590758/switch-graph-8ae00-86769987cd-rhthk"] Oct 31 08:20:40.075271 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:40.075244 4935 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-5c1f590758/switch-graph-8ae00-86769987cd-rhthk"] Oct 31 08:20:40.087722 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:40.087701 4935 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-5c1f590758/model-chainer-6b979c876d-j6srw" Oct 31 08:20:40.184677 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:40.184644 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5bbf5852-c16d-41a1-b2c1-b9a6ecddd898-proxy-tls\" (UniqueName: \"kubernetes.io/secret/5bbf5852-c16d-41a1-b2c1-b9a6ecddd898-proxy-tls\") pod \"5bbf5852-c16d-41a1-b2c1-b9a6ecddd898\" (UID: \"5bbf5852-c16d-41a1-b2c1-b9a6ecddd898\") " Oct 31 08:20:40.184862 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:40.184715 4935 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5bbf5852-c16d-41a1-b2c1-b9a6ecddd898-openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bbf5852-c16d-41a1-b2c1-b9a6ecddd898-openshift-service-ca-bundle\") pod \"5bbf5852-c16d-41a1-b2c1-b9a6ecddd898\" (UID: \"5bbf5852-c16d-41a1-b2c1-b9a6ecddd898\") " Oct 31 08:20:40.185071 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:40.185040 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5bbf5852-c16d-41a1-b2c1-b9a6ecddd898-openshift-service-ca-bundle" pod "5bbf5852-c16d-41a1-b2c1-b9a6ecddd898" (UID: "5bbf5852-c16d-41a1-b2c1-b9a6ecddd898"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 31 08:20:40.186867 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:40.186849 4935 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5bbf5852-c16d-41a1-b2c1-b9a6ecddd898-proxy-tls" pod "5bbf5852-c16d-41a1-b2c1-b9a6ecddd898" (UID: "5bbf5852-c16d-41a1-b2c1-b9a6ecddd898"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 31 08:20:40.242076 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:40.242048 4935 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9c26d8ba-fd9e-424a-acbf-1836ba32033a" path="/var/lib/kubelet/pods/9c26d8ba-fd9e-424a-acbf-1836ba32033a/volumes" Oct 31 08:20:40.286039 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:40.285989 4935 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5bbf5852-c16d-41a1-b2c1-b9a6ecddd898-proxy-tls\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 08:20:40.286039 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:40.286010 4935 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5bbf5852-c16d-41a1-b2c1-b9a6ecddd898-openshift-service-ca-bundle\") on node \"ip-10-0-130-48.us-east-2.compute.internal\" DevicePath \"\"" Oct 31 08:20:40.753523 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:40.753495 4935 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-5c1f590758/model-chainer-6b979c876d-j6srw" Oct 31 08:20:40.753964 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:40.753492 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-5c1f590758/model-chainer-6b979c876d-j6srw" event={"ID":"5bbf5852-c16d-41a1-b2c1-b9a6ecddd898","Type":"ContainerDied","Data":"35c97a945244c779e203422ed96e7af76fbd26966515c1d9c3f878d2e1e9b828"} Oct 31 08:20:40.753964 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:40.753628 4935 scope.go:122] "RemoveContainer" containerID="9c3a1f01995027b8648284d8dea326546e8e862c2baa8a4e4e5031a19c30be96" Oct 31 08:20:40.769789 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:40.769766 4935 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-5c1f590758/model-chainer-6b979c876d-j6srw"] Oct 31 08:20:40.774129 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:40.774111 4935 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-5c1f590758/model-chainer-6b979c876d-j6srw"] Oct 31 08:20:42.242492 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:42.242460 4935 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5bbf5852-c16d-41a1-b2c1-b9a6ecddd898" path="/var/lib/kubelet/pods/5bbf5852-c16d-41a1-b2c1-b9a6ecddd898/volumes" Oct 31 08:20:47.888268 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:47.888231 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:20:48.085494 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:48.085462 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:20:48.285336 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:48.285307 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:20:48.486728 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:48.486691 4935 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-xnprb/must-gather-wbvk6"] Oct 31 08:20:48.487009 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:48.486996 4935 cpu_manager.go:414] "containerMap: removing container" podUID="9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71" containerName="kserve-container" Oct 31 08:20:48.487052 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:48.487011 4935 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71" containerName="kserve-container" Oct 31 08:20:48.487052 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:48.487034 4935 cpu_manager.go:414] "containerMap: removing container" podUID="8f678a8c-1a16-41eb-93d1-45c89b51c98d" containerName="switch-graph-490af" Oct 31 08:20:48.487052 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:48.487040 4935 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8f678a8c-1a16-41eb-93d1-45c89b51c98d" containerName="switch-graph-490af" Oct 31 08:20:48.487052 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:48.487052 4935 cpu_manager.go:414] "containerMap: removing container" podUID="364dfb0e-d451-4015-b577-785b0ae174b4" containerName="kube-rbac-proxy" Oct 31 08:20:48.487170 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:48.487057 4935 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="364dfb0e-d451-4015-b577-785b0ae174b4" containerName="kube-rbac-proxy" Oct 31 08:20:48.487170 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:48.487065 4935 cpu_manager.go:414] "containerMap: removing container" podUID="9c26d8ba-fd9e-424a-acbf-1836ba32033a" containerName="switch-graph-8ae00" Oct 31 08:20:48.487170 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:48.487069 4935 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9c26d8ba-fd9e-424a-acbf-1836ba32033a" containerName="switch-graph-8ae00" Oct 31 08:20:48.487170 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:48.487079 4935 cpu_manager.go:414] "containerMap: removing container" podUID="5bbf5852-c16d-41a1-b2c1-b9a6ecddd898" containerName="model-chainer" Oct 31 08:20:48.487170 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:48.487083 4935 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5bbf5852-c16d-41a1-b2c1-b9a6ecddd898" containerName="model-chainer" Oct 31 08:20:48.487170 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:48.487094 4935 cpu_manager.go:414] "containerMap: removing container" podUID="6ada0c88-70ac-48e5-9e91-f6fd589b40cb" containerName="switch-graph-43360" Oct 31 08:20:48.487170 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:48.487103 4935 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6ada0c88-70ac-48e5-9e91-f6fd589b40cb" containerName="switch-graph-43360" Oct 31 08:20:48.487170 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:48.487113 4935 cpu_manager.go:414] "containerMap: removing container" podUID="9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71" containerName="kube-rbac-proxy" Oct 31 08:20:48.487170 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:48.487118 4935 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71" containerName="kube-rbac-proxy" Oct 31 08:20:48.487170 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:48.487123 4935 cpu_manager.go:414] "containerMap: removing container" podUID="364dfb0e-d451-4015-b577-785b0ae174b4" containerName="kserve-container" Oct 31 08:20:48.487170 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:48.487127 4935 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="364dfb0e-d451-4015-b577-785b0ae174b4" containerName="kserve-container" Oct 31 08:20:48.487170 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:48.487132 4935 cpu_manager.go:414] "containerMap: removing container" podUID="62d26bfc-7aed-4be2-a24c-33afc28acf05" containerName="splitter-graph-d859c" Oct 31 08:20:48.487170 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:48.487137 4935 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="62d26bfc-7aed-4be2-a24c-33afc28acf05" containerName="splitter-graph-d859c" Oct 31 08:20:48.487582 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:48.487192 4935 memory_manager.go:365] "RemoveStaleState removing state" podUID="8f678a8c-1a16-41eb-93d1-45c89b51c98d" containerName="switch-graph-490af" Oct 31 08:20:48.487582 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:48.487201 4935 memory_manager.go:365] "RemoveStaleState removing state" podUID="364dfb0e-d451-4015-b577-785b0ae174b4" containerName="kube-rbac-proxy" Oct 31 08:20:48.487582 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:48.487206 4935 memory_manager.go:365] "RemoveStaleState removing state" podUID="9c26d8ba-fd9e-424a-acbf-1836ba32033a" containerName="switch-graph-8ae00" Oct 31 08:20:48.487582 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:48.487211 4935 memory_manager.go:365] "RemoveStaleState removing state" podUID="5bbf5852-c16d-41a1-b2c1-b9a6ecddd898" containerName="model-chainer" Oct 31 08:20:48.487582 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:48.487218 4935 memory_manager.go:365] "RemoveStaleState removing state" podUID="6ada0c88-70ac-48e5-9e91-f6fd589b40cb" containerName="switch-graph-43360" Oct 31 08:20:48.487582 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:48.487222 4935 memory_manager.go:365] "RemoveStaleState removing state" podUID="9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71" containerName="kube-rbac-proxy" Oct 31 08:20:48.487582 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:48.487229 4935 memory_manager.go:365] "RemoveStaleState removing state" podUID="364dfb0e-d451-4015-b577-785b0ae174b4" containerName="kserve-container" Oct 31 08:20:48.487582 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:48.487233 4935 memory_manager.go:365] "RemoveStaleState removing state" podUID="62d26bfc-7aed-4be2-a24c-33afc28acf05" containerName="splitter-graph-d859c" Oct 31 08:20:48.487582 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:48.487238 4935 memory_manager.go:365] "RemoveStaleState removing state" podUID="9e47b4c7-1cd9-4662-9fc2-500a5a9c1a71" containerName="kserve-container" Oct 31 08:20:48.491596 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:48.491581 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xnprb/must-gather-wbvk6" Oct 31 08:20:48.495546 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:48.495523 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xnprb\"/\"openshift-service-ca.crt\"" Oct 31 08:20:48.496619 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:48.496598 4935 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-xnprb\"/\"default-dockercfg-wqp2z\"" Oct 31 08:20:48.496890 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:48.496870 4935 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xnprb\"/\"kube-root-ca.crt\"" Oct 31 08:20:48.498150 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:48.498126 4935 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xnprb/must-gather-wbvk6"] Oct 31 08:20:48.585933 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:48.585867 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:20:48.649590 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:48.649567 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x28lm\" (UniqueName: \"kubernetes.io/projected/ae83d2b1-73f0-4af7-9872-fa8befaf2175-kube-api-access-x28lm\") pod \"must-gather-wbvk6\" (UID: \"ae83d2b1-73f0-4af7-9872-fa8befaf2175\") " pod="openshift-must-gather-xnprb/must-gather-wbvk6" Oct 31 08:20:48.649682 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:48.649601 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ae83d2b1-73f0-4af7-9872-fa8befaf2175-must-gather-output\") pod \"must-gather-wbvk6\" (UID: \"ae83d2b1-73f0-4af7-9872-fa8befaf2175\") " pod="openshift-must-gather-xnprb/must-gather-wbvk6" Oct 31 08:20:48.750179 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:48.750155 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x28lm\" (UniqueName: \"kubernetes.io/projected/ae83d2b1-73f0-4af7-9872-fa8befaf2175-kube-api-access-x28lm\") pod \"must-gather-wbvk6\" (UID: \"ae83d2b1-73f0-4af7-9872-fa8befaf2175\") " pod="openshift-must-gather-xnprb/must-gather-wbvk6" Oct 31 08:20:48.750276 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:48.750198 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ae83d2b1-73f0-4af7-9872-fa8befaf2175-must-gather-output\") pod \"must-gather-wbvk6\" (UID: \"ae83d2b1-73f0-4af7-9872-fa8befaf2175\") " pod="openshift-must-gather-xnprb/must-gather-wbvk6" Oct 31 08:20:48.750538 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:48.750521 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ae83d2b1-73f0-4af7-9872-fa8befaf2175-must-gather-output\") pod \"must-gather-wbvk6\" (UID: \"ae83d2b1-73f0-4af7-9872-fa8befaf2175\") " pod="openshift-must-gather-xnprb/must-gather-wbvk6" Oct 31 08:20:48.762736 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:48.762712 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x28lm\" (UniqueName: \"kubernetes.io/projected/ae83d2b1-73f0-4af7-9872-fa8befaf2175-kube-api-access-x28lm\") pod \"must-gather-wbvk6\" (UID: \"ae83d2b1-73f0-4af7-9872-fa8befaf2175\") " pod="openshift-must-gather-xnprb/must-gather-wbvk6" Oct 31 08:20:48.785633 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:48.785613 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:20:48.811491 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:48.811474 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xnprb/must-gather-wbvk6" Oct 31 08:20:49.093439 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:49.093417 4935 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xnprb/must-gather-wbvk6"] Oct 31 08:20:49.095539 ip-10-0-130-48 kubenswrapper[4935]: W1031 08:20:49.095510 4935 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podae83d2b1_73f0_4af7_9872_fa8befaf2175.slice/crio-07db1745af8a4368182c513a5645a18da7dbe76b72dbf2e0f0e5b6c337dce12a WatchSource:0}: Error finding container 07db1745af8a4368182c513a5645a18da7dbe76b72dbf2e0f0e5b6c337dce12a: Status 404 returned error can't find the container with id 07db1745af8a4368182c513a5645a18da7dbe76b72dbf2e0f0e5b6c337dce12a Oct 31 08:20:49.097884 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:49.097868 4935 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 31 08:20:49.793444 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:49.793410 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xnprb/must-gather-wbvk6" event={"ID":"ae83d2b1-73f0-4af7-9872-fa8befaf2175","Type":"ContainerStarted","Data":"07db1745af8a4368182c513a5645a18da7dbe76b72dbf2e0f0e5b6c337dce12a"} Oct 31 08:20:50.801229 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:50.801196 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xnprb/must-gather-wbvk6" event={"ID":"ae83d2b1-73f0-4af7-9872-fa8befaf2175","Type":"ContainerStarted","Data":"fec7d6b63395a80f739e47053d39a8254f37a12b6a3b76b1632cbbbc2176eefc"} Oct 31 08:20:50.801668 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:50.801238 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xnprb/must-gather-wbvk6" event={"ID":"ae83d2b1-73f0-4af7-9872-fa8befaf2175","Type":"ContainerStarted","Data":"4f56a837fabdc073b076da1951e80660e369d6ee1eba3b2acc6104a650691c3b"} Oct 31 08:20:50.819508 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:50.819462 4935 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-xnprb/must-gather-wbvk6" podStartSLOduration=1.906701075 podStartE2EDuration="2.819443256s" totalImagesPullingTime="912.742181ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 08:20:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 08:20:50.818003542 +0000 UTC m=+3024.688535079" watchObservedRunningTime="2026-10-31 08:20:50.819443256 +0000 UTC m=+3024.689974795" Oct 31 08:20:51.563744 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:51.563713 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-8hpcb_297b35d7-aa65-49c7-8460-ac3f9860248c/global-pull-secret-syncer/0.log" Oct 31 08:20:51.589940 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:51.589914 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-cpkfr_e84bef13-a137-49f7-931d-9f94952dacfc/connection-checker/0.log" Oct 31 08:20:51.614524 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:51.614499 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-vhcq9_3cbf4695-0536-45d5-bc48-be56861b87b8/connection-checker/0.log" Oct 31 08:20:51.638142 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:51.638116 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-xkmbn_6fae9634-7abe-40c6-a892-9eb58166e9ad/connection-checker/0.log" Oct 31 08:20:51.680819 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:51.680777 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" Oct 31 08:20:51.782321 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:51.782292 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-48.us-east-2.compute.internal_8927b882957b5b07fb0714896c6f1a7f/haproxy/0.log" Oct 31 08:20:54.058372 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:54.055111 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:20:54.253218 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:54.253177 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:20:54.454709 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:54.454640 4935 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-pl5cl_d25f2854-fd38-44e7-ad7e-da744d57e671/konnectivity-agent/0.log" limit=1024 Oct 31 08:20:57.142080 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:57.142027 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-884576c57-282kr_5e743234-2b5f-4c91-9b06-189f546cbce4/cluster-monitoring-operator/0.log" Oct 31 08:20:57.307395 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:57.306522 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8c6xb_0c09aa6f-d0d8-4276-b1f2-f61446aadd99/node-exporter/0.log" Oct 31 08:20:57.333790 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:57.333759 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8c6xb_0c09aa6f-d0d8-4276-b1f2-f61446aadd99/kube-rbac-proxy/0.log" Oct 31 08:20:57.362995 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:57.362966 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8c6xb_0c09aa6f-d0d8-4276-b1f2-f61446aadd99/init-textfile/0.log" Oct 31 08:20:59.184878 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:59.184847 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-86c478bb88-s75pm_563d3b8b-0040-439e-b0b2-e858eed5abf6/networking-console-plugin/0.log" Oct 31 08:20:59.382064 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:59.381050 4935 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-xnprb/perf-node-gather-daemonset-7wd7l"] Oct 31 08:20:59.386902 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:59.386871 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xnprb/perf-node-gather-daemonset-7wd7l" Oct 31 08:20:59.398910 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:59.398134 4935 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xnprb/perf-node-gather-daemonset-7wd7l"] Oct 31 08:20:59.407920 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:59.407891 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f61199cc-80b7-4230-9cdc-cfe3c53de024-podres\") pod \"perf-node-gather-daemonset-7wd7l\" (UID: \"f61199cc-80b7-4230-9cdc-cfe3c53de024\") " pod="openshift-must-gather-xnprb/perf-node-gather-daemonset-7wd7l" Oct 31 08:20:59.408072 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:59.407997 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f61199cc-80b7-4230-9cdc-cfe3c53de024-sys\") pod \"perf-node-gather-daemonset-7wd7l\" (UID: \"f61199cc-80b7-4230-9cdc-cfe3c53de024\") " pod="openshift-must-gather-xnprb/perf-node-gather-daemonset-7wd7l" Oct 31 08:20:59.408072 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:59.408053 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f61199cc-80b7-4230-9cdc-cfe3c53de024-proc\") pod \"perf-node-gather-daemonset-7wd7l\" (UID: \"f61199cc-80b7-4230-9cdc-cfe3c53de024\") " pod="openshift-must-gather-xnprb/perf-node-gather-daemonset-7wd7l" Oct 31 08:20:59.408173 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:59.408099 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f61199cc-80b7-4230-9cdc-cfe3c53de024-lib-modules\") pod \"perf-node-gather-daemonset-7wd7l\" (UID: \"f61199cc-80b7-4230-9cdc-cfe3c53de024\") " pod="openshift-must-gather-xnprb/perf-node-gather-daemonset-7wd7l" Oct 31 08:20:59.408238 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:59.408166 4935 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfgg4\" (UniqueName: \"kubernetes.io/projected/f61199cc-80b7-4230-9cdc-cfe3c53de024-kube-api-access-dfgg4\") pod \"perf-node-gather-daemonset-7wd7l\" (UID: \"f61199cc-80b7-4230-9cdc-cfe3c53de024\") " pod="openshift-must-gather-xnprb/perf-node-gather-daemonset-7wd7l" Oct 31 08:20:59.508576 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:59.508546 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f61199cc-80b7-4230-9cdc-cfe3c53de024-proc\") pod \"perf-node-gather-daemonset-7wd7l\" (UID: \"f61199cc-80b7-4230-9cdc-cfe3c53de024\") " pod="openshift-must-gather-xnprb/perf-node-gather-daemonset-7wd7l" Oct 31 08:20:59.508576 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:59.508578 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f61199cc-80b7-4230-9cdc-cfe3c53de024-lib-modules\") pod \"perf-node-gather-daemonset-7wd7l\" (UID: \"f61199cc-80b7-4230-9cdc-cfe3c53de024\") " pod="openshift-must-gather-xnprb/perf-node-gather-daemonset-7wd7l" Oct 31 08:20:59.508774 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:59.508596 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfgg4\" (UniqueName: \"kubernetes.io/projected/f61199cc-80b7-4230-9cdc-cfe3c53de024-kube-api-access-dfgg4\") pod \"perf-node-gather-daemonset-7wd7l\" (UID: \"f61199cc-80b7-4230-9cdc-cfe3c53de024\") " pod="openshift-must-gather-xnprb/perf-node-gather-daemonset-7wd7l" Oct 31 08:20:59.508774 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:59.508613 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f61199cc-80b7-4230-9cdc-cfe3c53de024-podres\") pod \"perf-node-gather-daemonset-7wd7l\" (UID: \"f61199cc-80b7-4230-9cdc-cfe3c53de024\") " pod="openshift-must-gather-xnprb/perf-node-gather-daemonset-7wd7l" Oct 31 08:20:59.508774 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:59.508651 4935 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f61199cc-80b7-4230-9cdc-cfe3c53de024-sys\") pod \"perf-node-gather-daemonset-7wd7l\" (UID: \"f61199cc-80b7-4230-9cdc-cfe3c53de024\") " pod="openshift-must-gather-xnprb/perf-node-gather-daemonset-7wd7l" Oct 31 08:20:59.508774 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:59.508718 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f61199cc-80b7-4230-9cdc-cfe3c53de024-sys\") pod \"perf-node-gather-daemonset-7wd7l\" (UID: \"f61199cc-80b7-4230-9cdc-cfe3c53de024\") " pod="openshift-must-gather-xnprb/perf-node-gather-daemonset-7wd7l" Oct 31 08:20:59.508774 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:59.508758 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f61199cc-80b7-4230-9cdc-cfe3c53de024-proc\") pod \"perf-node-gather-daemonset-7wd7l\" (UID: \"f61199cc-80b7-4230-9cdc-cfe3c53de024\") " pod="openshift-must-gather-xnprb/perf-node-gather-daemonset-7wd7l" Oct 31 08:20:59.508948 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:59.508797 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f61199cc-80b7-4230-9cdc-cfe3c53de024-lib-modules\") pod \"perf-node-gather-daemonset-7wd7l\" (UID: \"f61199cc-80b7-4230-9cdc-cfe3c53de024\") " pod="openshift-must-gather-xnprb/perf-node-gather-daemonset-7wd7l" Oct 31 08:20:59.509062 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:59.509047 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f61199cc-80b7-4230-9cdc-cfe3c53de024-podres\") pod \"perf-node-gather-daemonset-7wd7l\" (UID: \"f61199cc-80b7-4230-9cdc-cfe3c53de024\") " pod="openshift-must-gather-xnprb/perf-node-gather-daemonset-7wd7l" Oct 31 08:20:59.520987 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:59.520959 4935 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfgg4\" (UniqueName: \"kubernetes.io/projected/f61199cc-80b7-4230-9cdc-cfe3c53de024-kube-api-access-dfgg4\") pod \"perf-node-gather-daemonset-7wd7l\" (UID: \"f61199cc-80b7-4230-9cdc-cfe3c53de024\") " pod="openshift-must-gather-xnprb/perf-node-gather-daemonset-7wd7l" Oct 31 08:20:59.667572 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:59.667538 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jztcz_396e1c5e-3700-4fe6-b367-54ad2a244225/console-operator/2.log" Oct 31 08:20:59.673210 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:59.673188 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-jztcz_396e1c5e-3700-4fe6-b367-54ad2a244225/console-operator/3.log" Oct 31 08:20:59.708717 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:20:59.708500 4935 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xnprb/perf-node-gather-daemonset-7wd7l" Oct 31 08:21:00.008002 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:21:00.007965 4935 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xnprb/perf-node-gather-daemonset-7wd7l"] Oct 31 08:21:00.690055 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:21:00.690026 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-58c9f5f46d-m6sns_54296c9d-1c43-44cd-bfb6-d81fe892484c/volume-data-source-validator/0.log" Oct 31 08:21:00.912644 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:21:00.912607 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xnprb/perf-node-gather-daemonset-7wd7l" event={"ID":"f61199cc-80b7-4230-9cdc-cfe3c53de024","Type":"ContainerStarted","Data":"35fa4878efaaea90bcccd034afbaee0e8adedffee0fa1dde150772f7e1ed4339"} Oct 31 08:21:00.912830 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:21:00.912657 4935 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xnprb/perf-node-gather-daemonset-7wd7l" event={"ID":"f61199cc-80b7-4230-9cdc-cfe3c53de024","Type":"ContainerStarted","Data":"89536c59f3be3cc1fcbed86ae62309f983db8bb5574e3c188d73bce8c3725336"} Oct 31 08:21:00.912830 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:21:00.912740 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-xnprb/perf-node-gather-daemonset-7wd7l" Oct 31 08:21:00.931583 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:21:00.931534 4935 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-xnprb/perf-node-gather-daemonset-7wd7l" podStartSLOduration=1.931519215 podStartE2EDuration="1.931519215s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 08:20:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 08:21:00.930707269 +0000 UTC m=+3034.801238807" watchObservedRunningTime="2026-10-31 08:21:00.931519215 +0000 UTC m=+3034.802050752" Oct 31 08:21:01.476880 ip-10-0-130-48 kubenswrapper[4935]: E1031 08:21:01.476829 4935 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mvmr5" containerName="extractor" Oct 31 08:21:01.733055 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:21:01.732957 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-z5sfp_58bc3ea1-f58c-4891-a049-1289a99be055/dns/0.log" Oct 31 08:21:01.757792 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:21:01.757769 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-z5sfp_58bc3ea1-f58c-4891-a049-1289a99be055/kube-rbac-proxy/0.log" Oct 31 08:21:01.839839 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:21:01.839815 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-scrms_53c36e19-9ef6-47a2-9bc2-387115539850/dns-node-resolver/0.log" Oct 31 08:21:02.360952 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:21:02.360925 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rhg7w_236238a7-255d-40f7-a99b-d65903b3bcb7/node-ca/0.log" Oct 31 08:21:03.129975 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:21:03.129946 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5549647ff-cxcdt_28be803a-a932-4009-ad61-b356366821ef/router/0.log" Oct 31 08:21:03.471009 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:21:03.470984 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-22cmd_00a0d978-8020-46df-932d-b0dd22f946f9/serve-healthcheck-canary/0.log" Oct 31 08:21:03.867742 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:21:03.867660 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-7c4986d449-hz58r_b4dd8884-1d6e-4abe-ad6c-3c947623bb60/insights-operator/0.log" Oct 31 08:21:03.868939 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:21:03.868912 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-7c4986d449-hz58r_b4dd8884-1d6e-4abe-ad6c-3c947623bb60/insights-operator/1.log" Oct 31 08:21:04.034274 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:21:04.034247 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mvmr5_f13bc62a-1894-472c-95de-262afa399b38/kube-rbac-proxy/0.log" Oct 31 08:21:04.056408 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:21:04.056369 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mvmr5_f13bc62a-1894-472c-95de-262afa399b38/exporter/0.log" Oct 31 08:21:04.078679 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:21:04.078660 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mvmr5_f13bc62a-1894-472c-95de-262afa399b38/extractor/0.log" Oct 31 08:21:04.112796 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:21:04.112774 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_periodic-gathering-rmx96-5ppn9_ff188433-6826-4666-9e78-f61f337069d4/insights-gathering/0.log" Oct 31 08:21:06.587102 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:21:06.587072 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_llmisvc-controller-manager-86c86ffb6b-gfp9q_53ccf16f-450f-46cf-8c1e-5bc99e15789a/manager/0.log" Oct 31 08:21:06.928052 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:21:06.927975 4935 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-xnprb/perf-node-gather-daemonset-7wd7l" Oct 31 08:21:11.132571 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:21:11.132535 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-66df5d496b-dwlls_acda6b08-2faa-4836-888e-d4ecf324cb1d/migrator/0.log" Oct 31 08:21:11.450405 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:21:11.450375 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7d4bc45dd6-46qvv_f3bf19fd-bc58-4dff-a1a8-4f61381e1fde/kube-storage-version-migrator-operator/1.log" Oct 31 08:21:11.451280 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:21:11.451259 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7d4bc45dd6-46qvv_f3bf19fd-bc58-4dff-a1a8-4f61381e1fde/kube-storage-version-migrator-operator/0.log" Oct 31 08:21:12.425889 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:21:12.425863 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ktzg4_ed522b38-611a-452a-b201-3ca416428d42/kube-multus-additional-cni-plugins/0.log" Oct 31 08:21:12.447906 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:21:12.447882 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ktzg4_ed522b38-611a-452a-b201-3ca416428d42/egress-router-binary-copy/0.log" Oct 31 08:21:12.469571 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:21:12.469555 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ktzg4_ed522b38-611a-452a-b201-3ca416428d42/cni-plugins/0.log" Oct 31 08:21:12.491774 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:21:12.491752 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ktzg4_ed522b38-611a-452a-b201-3ca416428d42/bond-cni-plugin/0.log" Oct 31 08:21:12.514112 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:21:12.514090 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ktzg4_ed522b38-611a-452a-b201-3ca416428d42/routeoverride-cni/0.log" Oct 31 08:21:12.535996 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:21:12.535975 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ktzg4_ed522b38-611a-452a-b201-3ca416428d42/whereabouts-cni-bincopy/0.log" Oct 31 08:21:12.557477 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:21:12.557455 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ktzg4_ed522b38-611a-452a-b201-3ca416428d42/whereabouts-cni/0.log" Oct 31 08:21:12.818232 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:21:12.818205 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-w6vnh_6ae10413-8f49-4dfe-8aad-00dcb9d390c8/kube-multus/0.log" Oct 31 08:21:12.885315 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:21:12.885290 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-7thxr_28ec9621-7ec5-421c-a88b-51bb20d3a7bf/network-metrics-daemon/0.log" Oct 31 08:21:12.906103 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:21:12.906084 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-7thxr_28ec9621-7ec5-421c-a88b-51bb20d3a7bf/kube-rbac-proxy/0.log" Oct 31 08:21:13.710782 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:21:13.710752 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-98whn_e00eb9e3-0d3c-4541-b238-4130fe1de2f1/ovn-controller/0.log" Oct 31 08:21:13.730012 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:21:13.729991 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-98whn_e00eb9e3-0d3c-4541-b238-4130fe1de2f1/ovn-acl-logging/0.log" Oct 31 08:21:13.755166 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:21:13.755142 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-98whn_e00eb9e3-0d3c-4541-b238-4130fe1de2f1/kube-rbac-proxy-node/0.log" Oct 31 08:21:13.779071 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:21:13.779043 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-98whn_e00eb9e3-0d3c-4541-b238-4130fe1de2f1/kube-rbac-proxy-ovn-metrics/0.log" Oct 31 08:21:13.801827 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:21:13.801804 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-98whn_e00eb9e3-0d3c-4541-b238-4130fe1de2f1/northd/0.log" Oct 31 08:21:13.824886 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:21:13.824866 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-98whn_e00eb9e3-0d3c-4541-b238-4130fe1de2f1/nbdb/0.log" Oct 31 08:21:13.849875 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:21:13.849851 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-98whn_e00eb9e3-0d3c-4541-b238-4130fe1de2f1/sbdb/0.log" Oct 31 08:21:13.903820 ip-10-0-130-48 kubenswrapper[4935]: I1031 08:21:13.903795 4935 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-98whn_e00eb9e3-0d3c-4541-b238-4130fe1de2f1/ovnkube-controller/0.log"