Jul 10 16:54:15.800331 ip-10-0-136-7 systemd[1]: Starting Kubernetes Kubelet... Jul 10 16:54:16.201549 ip-10-0-136-7 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 16:54:16.201549 ip-10-0-136-7 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 10 16:54:16.201549 ip-10-0-136-7 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 16:54:16.201549 ip-10-0-136-7 kubenswrapper[2578]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 10 16:54:16.201549 ip-10-0-136-7 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 16:54:16.202545 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.202457 2578 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 10 16:54:16.206808 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206789 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 16:54:16.206808 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206807 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 16:54:16.206808 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206811 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 16:54:16.206919 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206814 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 10 16:54:16.206919 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206817 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 16:54:16.206919 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206820 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 16:54:16.206919 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206823 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 16:54:16.206919 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206826 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 16:54:16.206919 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206828 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 16:54:16.206919 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206831 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 16:54:16.206919 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206833 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 16:54:16.206919 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206836 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 16:54:16.206919 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206846 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 16:54:16.206919 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206850 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 16:54:16.206919 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206853 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 16:54:16.206919 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206856 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 16:54:16.206919 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206858 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 16:54:16.206919 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206861 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 16:54:16.206919 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206864 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 16:54:16.206919 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206867 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 16:54:16.206919 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206869 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 16:54:16.206919 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206872 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 16:54:16.207394 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206874 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 16:54:16.207394 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206877 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 16:54:16.207394 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206880 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 16:54:16.207394 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206884 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 16:54:16.207394 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206888 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 16:54:16.207394 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206891 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 16:54:16.207394 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206895 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 16:54:16.207394 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206897 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 16:54:16.207394 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206900 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 16:54:16.207394 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206903 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 16:54:16.207394 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206906 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 16:54:16.207394 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206908 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 16:54:16.207394 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206911 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 16:54:16.207394 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206914 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 16:54:16.207394 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206917 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 16:54:16.207394 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206920 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 16:54:16.207394 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206923 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 16:54:16.207394 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206926 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 16:54:16.207394 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206929 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 16:54:16.207394 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206931 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 16:54:16.207870 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206934 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 16:54:16.207870 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206937 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 16:54:16.207870 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206939 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 16:54:16.207870 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206942 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 16:54:16.207870 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206944 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 16:54:16.207870 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206947 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 16:54:16.207870 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206950 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 16:54:16.207870 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206952 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 16:54:16.207870 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206955 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 16:54:16.207870 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206957 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 16:54:16.207870 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206960 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 16:54:16.207870 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206963 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 16:54:16.207870 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206965 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 16:54:16.207870 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206968 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 16:54:16.207870 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206971 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 16:54:16.207870 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206973 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 16:54:16.207870 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206976 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 16:54:16.207870 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206978 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 16:54:16.207870 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206981 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 16:54:16.207870 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206983 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 16:54:16.208369 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206986 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 16:54:16.208369 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206988 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 16:54:16.208369 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206991 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 16:54:16.208369 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206994 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 16:54:16.208369 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206996 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 16:54:16.208369 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.206999 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 16:54:16.208369 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207003 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 16:54:16.208369 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207005 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 16:54:16.208369 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207008 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 16:54:16.208369 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207017 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 16:54:16.208369 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207020 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 16:54:16.208369 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207023 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 16:54:16.208369 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207026 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 16:54:16.208369 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207028 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 16:54:16.208369 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207031 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 16:54:16.208369 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207033 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 16:54:16.208369 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207036 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 16:54:16.208369 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207038 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 16:54:16.208369 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207041 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 16:54:16.208369 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207057 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 16:54:16.208842 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207060 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 16:54:16.208842 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207063 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 16:54:16.208842 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207066 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 16:54:16.208842 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207068 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 16:54:16.208842 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207072 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 16:54:16.208842 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207076 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 16:54:16.208842 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207078 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 16:54:16.208842 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207081 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 16:54:16.208842 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207084 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 16:54:16.208842 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207087 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 16:54:16.208842 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207089 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 16:54:16.208842 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207092 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 16:54:16.208842 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207094 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 16:54:16.208842 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207097 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 16:54:16.208842 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207100 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 16:54:16.208842 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207104 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 16:54:16.208842 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207107 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 16:54:16.208842 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207822 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 16:54:16.208842 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207829 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 16:54:16.209318 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207832 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 16:54:16.209318 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207835 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 16:54:16.209318 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207837 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 16:54:16.209318 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207841 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 16:54:16.209318 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207843 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 16:54:16.209318 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207846 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 16:54:16.209318 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207849 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 16:54:16.209318 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207851 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 16:54:16.209318 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207856 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 16:54:16.209318 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207859 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 16:54:16.209318 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207862 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 16:54:16.209318 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207865 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 16:54:16.209318 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207868 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 16:54:16.209318 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207871 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 16:54:16.209318 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207874 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 16:54:16.209318 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207877 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 16:54:16.209318 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207880 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 16:54:16.209318 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207882 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 16:54:16.209318 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207885 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 16:54:16.209792 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207889 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 16:54:16.209792 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207893 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 16:54:16.209792 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207895 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 16:54:16.209792 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207898 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 16:54:16.209792 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207901 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 16:54:16.209792 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207903 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 16:54:16.209792 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207906 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 16:54:16.209792 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207909 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 16:54:16.209792 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207912 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 16:54:16.209792 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207915 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 16:54:16.209792 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207917 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 16:54:16.209792 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207921 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 16:54:16.209792 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207925 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 16:54:16.209792 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207928 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 16:54:16.209792 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207931 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 16:54:16.209792 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207934 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 16:54:16.209792 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207936 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 16:54:16.209792 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207939 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 16:54:16.209792 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207942 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 16:54:16.209792 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207945 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 16:54:16.210316 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207947 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 16:54:16.210316 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207950 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 16:54:16.210316 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207952 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 16:54:16.210316 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207955 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 16:54:16.210316 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207957 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 16:54:16.210316 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207960 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 16:54:16.210316 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207962 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 16:54:16.210316 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207965 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 16:54:16.210316 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207968 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 16:54:16.210316 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207970 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 16:54:16.210316 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207973 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 16:54:16.210316 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207975 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 16:54:16.210316 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207978 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 16:54:16.210316 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207980 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 16:54:16.210316 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207982 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 16:54:16.210316 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207985 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 16:54:16.210316 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207987 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 16:54:16.210316 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207990 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 16:54:16.210316 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207992 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 16:54:16.210316 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207995 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 16:54:16.210807 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.207997 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 16:54:16.210807 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208000 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 16:54:16.210807 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208004 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 16:54:16.210807 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208008 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 16:54:16.210807 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208011 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 16:54:16.210807 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208013 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 16:54:16.210807 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208016 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 16:54:16.210807 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208019 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 16:54:16.210807 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208021 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 16:54:16.210807 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208024 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 16:54:16.210807 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208027 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 16:54:16.210807 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208030 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 16:54:16.210807 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208033 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 16:54:16.210807 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208036 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 16:54:16.210807 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208038 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 10 16:54:16.210807 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208041 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 16:54:16.210807 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208059 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 16:54:16.210807 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208062 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 16:54:16.210807 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208065 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 16:54:16.211292 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208067 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 16:54:16.211292 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208070 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 16:54:16.211292 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208072 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 16:54:16.211292 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208075 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 16:54:16.211292 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208077 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 16:54:16.211292 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208080 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 16:54:16.211292 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208083 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 16:54:16.211292 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208086 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 16:54:16.211292 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208088 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 16:54:16.211292 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208091 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 16:54:16.211292 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208094 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 16:54:16.211292 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208096 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 16:54:16.211292 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208099 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 16:54:16.211292 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208101 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 16:54:16.211292 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208104 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 16:54:16.211292 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208107 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 16:54:16.211292 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208110 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 16:54:16.211292 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208112 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 16:54:16.211292 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208115 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 16:54:16.211292 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208485 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 16:54:16.211770 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208489 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 10 16:54:16.211770 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208492 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 16:54:16.211770 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208495 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 16:54:16.211770 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208498 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 16:54:16.211770 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208501 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 16:54:16.211770 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208503 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 16:54:16.211770 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208507 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 16:54:16.211770 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208509 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 16:54:16.211770 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208512 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 16:54:16.211770 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208514 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 16:54:16.211770 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208517 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 16:54:16.211770 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208520 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 16:54:16.211770 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208523 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 16:54:16.211770 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208525 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 16:54:16.211770 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208527 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 16:54:16.211770 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208530 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 16:54:16.211770 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208533 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 16:54:16.211770 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208535 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 16:54:16.211770 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208538 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 16:54:16.211770 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208540 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 16:54:16.212282 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208543 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 16:54:16.212282 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208545 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 16:54:16.212282 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208548 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 16:54:16.212282 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208551 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 16:54:16.212282 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208553 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 16:54:16.212282 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208555 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 16:54:16.212282 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208558 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 16:54:16.212282 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208561 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 16:54:16.212282 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208564 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 16:54:16.212282 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208567 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 16:54:16.212282 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208571 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 16:54:16.212282 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208573 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 16:54:16.212282 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208578 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 16:54:16.212282 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208581 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 16:54:16.212282 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208585 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 16:54:16.212282 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208589 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 16:54:16.212282 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208592 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 16:54:16.212282 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208595 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 16:54:16.212282 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208598 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 16:54:16.212742 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208601 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 16:54:16.212742 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208603 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 16:54:16.212742 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208606 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 16:54:16.212742 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208609 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 16:54:16.212742 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208611 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 16:54:16.212742 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208614 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 16:54:16.212742 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208616 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 16:54:16.212742 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208619 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 16:54:16.212742 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208621 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 16:54:16.212742 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208624 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 16:54:16.212742 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208627 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 16:54:16.212742 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208629 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 16:54:16.212742 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208633 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 16:54:16.212742 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208635 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 16:54:16.212742 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208638 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 16:54:16.212742 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208641 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 16:54:16.212742 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208643 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 16:54:16.212742 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208647 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 16:54:16.212742 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208650 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 16:54:16.213229 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208653 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 16:54:16.213229 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208656 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 16:54:16.213229 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208661 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 16:54:16.213229 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208664 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 16:54:16.213229 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208667 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 16:54:16.213229 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208669 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 16:54:16.213229 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208672 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 16:54:16.213229 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208675 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 16:54:16.213229 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208677 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 16:54:16.213229 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208680 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 16:54:16.213229 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208683 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 16:54:16.213229 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208686 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 16:54:16.213229 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208688 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 16:54:16.213229 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208691 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 16:54:16.213229 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208693 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 16:54:16.213229 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208696 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 16:54:16.213229 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208698 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 16:54:16.213229 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208701 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 16:54:16.213229 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208703 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 16:54:16.213229 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208706 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 16:54:16.213745 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208708 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 16:54:16.213745 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208711 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 16:54:16.213745 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208713 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 16:54:16.213745 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208716 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 16:54:16.213745 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208719 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 16:54:16.213745 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208721 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 16:54:16.213745 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208724 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 16:54:16.213745 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208726 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 16:54:16.213745 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208729 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 16:54:16.213745 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208731 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 16:54:16.213745 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208734 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 16:54:16.213745 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208736 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 16:54:16.213745 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208739 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 16:54:16.213745 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208741 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 16:54:16.213745 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208745 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 16:54:16.213745 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208748 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 16:54:16.213745 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208750 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 16:54:16.213745 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208753 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 16:54:16.213745 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208755 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 16:54:16.213745 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.208758 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 16:54:16.214269 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.208837 2578 flags.go:64] FLAG: --address="0.0.0.0" Jul 10 16:54:16.214269 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.208848 2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 10 16:54:16.214269 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.208858 2578 flags.go:64] FLAG: --anonymous-auth="true" Jul 10 16:54:16.214269 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.208865 2578 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 10 16:54:16.214269 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.208871 2578 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 10 16:54:16.214269 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.208877 2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 10 16:54:16.214269 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.208884 2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 10 16:54:16.214269 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.208889 2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 10 16:54:16.214269 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.208892 2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 10 16:54:16.214269 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.208895 2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 10 16:54:16.214269 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.208899 2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 10 16:54:16.214269 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.208902 2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 10 16:54:16.214269 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.208905 2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 10 16:54:16.214269 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.208908 2578 flags.go:64] FLAG: --cgroup-root="" Jul 10 16:54:16.214269 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.208911 2578 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 10 16:54:16.214269 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.208914 2578 flags.go:64] FLAG: --client-ca-file="" Jul 10 16:54:16.214269 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.208917 2578 flags.go:64] FLAG: --cloud-provider="external" Jul 10 16:54:16.214269 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.208920 2578 flags.go:64] FLAG: --cluster-dns="[]" Jul 10 16:54:16.214269 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.208926 2578 flags.go:64] FLAG: --cluster-domain="" Jul 10 16:54:16.214269 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.208929 2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 10 16:54:16.214269 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.208932 2578 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 10 16:54:16.214269 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.208935 2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 10 16:54:16.214269 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.208938 2578 flags.go:64] FLAG: --container-log-max-files="5" Jul 10 16:54:16.214269 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.208942 2578 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 10 16:54:16.214842 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.208945 2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 10 16:54:16.214842 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.208948 2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 10 16:54:16.214842 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.208952 2578 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 10 16:54:16.214842 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.208955 2578 flags.go:64] FLAG: --contention-profiling="false" Jul 10 16:54:16.214842 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.208958 2578 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 10 16:54:16.214842 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.208962 2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 10 16:54:16.214842 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.208965 2578 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 10 16:54:16.214842 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.208968 2578 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 10 16:54:16.214842 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.208972 2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 10 16:54:16.214842 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.208975 2578 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 10 16:54:16.214842 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.208978 2578 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 10 16:54:16.214842 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.208981 2578 flags.go:64] FLAG: --enable-load-reader="false" Jul 10 16:54:16.214842 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.208984 2578 flags.go:64] FLAG: --enable-server="true" Jul 10 16:54:16.214842 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.208988 2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 10 16:54:16.214842 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.208993 2578 flags.go:64] FLAG: --event-burst="100" Jul 10 16:54:16.214842 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.208996 2578 flags.go:64] FLAG: --event-qps="50" Jul 10 16:54:16.214842 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.208999 2578 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 10 16:54:16.214842 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209002 2578 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 10 16:54:16.214842 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209005 2578 flags.go:64] FLAG: --eviction-hard="" Jul 10 16:54:16.214842 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209008 2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 10 16:54:16.214842 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209011 2578 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 10 16:54:16.214842 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209014 2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 10 16:54:16.214842 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209017 2578 flags.go:64] FLAG: --eviction-soft="" Jul 10 16:54:16.214842 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209020 2578 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 10 16:54:16.214842 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209024 2578 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 10 16:54:16.215554 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209027 2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 10 16:54:16.215554 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209030 2578 flags.go:64] FLAG: --experimental-mounter-path="" Jul 10 16:54:16.215554 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209033 2578 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 10 16:54:16.215554 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209036 2578 flags.go:64] FLAG: --fail-swap-on="true" Jul 10 16:54:16.215554 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209038 2578 flags.go:64] FLAG: --feature-gates="" Jul 10 16:54:16.215554 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209042 2578 flags.go:64] FLAG: --file-check-frequency="20s" Jul 10 16:54:16.215554 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209059 2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 10 16:54:16.215554 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209063 2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 10 16:54:16.215554 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209066 2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 10 16:54:16.215554 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209069 2578 flags.go:64] FLAG: --healthz-port="10248" Jul 10 16:54:16.215554 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209073 2578 flags.go:64] FLAG: --help="false" Jul 10 16:54:16.215554 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209076 2578 flags.go:64] FLAG: --hostname-override="ip-10-0-136-7.ec2.internal" Jul 10 16:54:16.215554 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209079 2578 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 10 16:54:16.215554 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209082 2578 flags.go:64] FLAG: --http-check-frequency="20s" Jul 10 16:54:16.215554 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209085 2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 10 16:54:16.215554 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209088 2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 10 16:54:16.215554 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209092 2578 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 10 16:54:16.215554 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209095 2578 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 10 16:54:16.215554 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209097 2578 flags.go:64] FLAG: --image-service-endpoint="" Jul 10 16:54:16.215554 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209101 2578 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 10 16:54:16.215554 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209104 2578 flags.go:64] FLAG: --kube-api-burst="100" Jul 10 16:54:16.215554 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209108 2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 10 16:54:16.215554 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209111 2578 flags.go:64] FLAG: --kube-api-qps="50" Jul 10 16:54:16.215554 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209114 2578 flags.go:64] FLAG: --kube-reserved="" Jul 10 16:54:16.216274 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209116 2578 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 10 16:54:16.216274 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209119 2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 10 16:54:16.216274 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209122 2578 flags.go:64] FLAG: --kubelet-cgroups="" Jul 10 16:54:16.216274 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209125 2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 10 16:54:16.216274 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209128 2578 flags.go:64] FLAG: --lock-file="" Jul 10 16:54:16.216274 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209130 2578 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 10 16:54:16.216274 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209133 2578 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 10 16:54:16.216274 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209137 2578 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 10 16:54:16.216274 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209142 2578 flags.go:64] FLAG: --log-json-split-stream="false" Jul 10 16:54:16.216274 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209145 2578 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 10 16:54:16.216274 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209148 2578 flags.go:64] FLAG: --log-text-split-stream="false" Jul 10 16:54:16.216274 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209150 2578 flags.go:64] FLAG: --logging-format="text" Jul 10 16:54:16.216274 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209153 2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 10 16:54:16.216274 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209157 2578 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 10 16:54:16.216274 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209159 2578 flags.go:64] FLAG: --manifest-url="" Jul 10 16:54:16.216274 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209163 2578 flags.go:64] FLAG: --manifest-url-header="" Jul 10 16:54:16.216274 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209167 2578 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 10 16:54:16.216274 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209170 2578 flags.go:64] FLAG: --max-open-files="1000000" Jul 10 16:54:16.216274 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209174 2578 flags.go:64] FLAG: --max-pods="110" Jul 10 16:54:16.216274 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209177 2578 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 10 16:54:16.216274 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209180 2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 10 16:54:16.216274 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209182 2578 flags.go:64] FLAG: --memory-manager-policy="None" Jul 10 16:54:16.216274 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209185 2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 10 16:54:16.216274 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209188 2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 10 16:54:16.216274 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209192 2578 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 10 16:54:16.216894 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209195 2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 10 16:54:16.216894 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209202 2578 flags.go:64] FLAG: --node-status-max-images="50" Jul 10 16:54:16.216894 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209205 2578 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 10 16:54:16.216894 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209208 2578 flags.go:64] FLAG: --oom-score-adj="-999" Jul 10 16:54:16.216894 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209211 2578 flags.go:64] FLAG: --pod-cidr="" Jul 10 16:54:16.216894 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209214 2578 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 10 16:54:16.216894 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209220 2578 flags.go:64] FLAG: --pod-manifest-path="" Jul 10 16:54:16.216894 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209223 2578 flags.go:64] FLAG: --pod-max-pids="-1" Jul 10 16:54:16.216894 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209226 2578 flags.go:64] FLAG: --pods-per-core="0" Jul 10 16:54:16.216894 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209229 2578 flags.go:64] FLAG: --port="10250" Jul 10 16:54:16.216894 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209232 2578 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 10 16:54:16.216894 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209235 2578 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-09520a49df856afa0" Jul 10 16:54:16.216894 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209238 2578 flags.go:64] FLAG: --qos-reserved="" Jul 10 16:54:16.216894 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209243 2578 flags.go:64] FLAG: --read-only-port="10255" Jul 10 16:54:16.216894 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209246 2578 flags.go:64] FLAG: --register-node="true" Jul 10 16:54:16.216894 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209249 2578 flags.go:64] FLAG: --register-with-taints="" Jul 10 16:54:16.216894 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209253 2578 flags.go:64] FLAG: --registry-burst="10" Jul 10 16:54:16.216894 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209256 2578 flags.go:64] FLAG: --registry-qps="5" Jul 10 16:54:16.216894 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209258 2578 flags.go:64] FLAG: --reserved-cpus="" Jul 10 16:54:16.216894 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209261 2578 flags.go:64] FLAG: --reserved-memory="" Jul 10 16:54:16.216894 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209265 2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 10 16:54:16.216894 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209268 2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 10 16:54:16.216894 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209271 2578 flags.go:64] FLAG: --rotate-certificates="false" Jul 10 16:54:16.216894 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209274 2578 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 10 16:54:16.216894 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209277 2578 flags.go:64] FLAG: --runonce="false" Jul 10 16:54:16.217514 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209279 2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 10 16:54:16.217514 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209282 2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 10 16:54:16.217514 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209285 2578 flags.go:64] FLAG: --seccomp-default="false" Jul 10 16:54:16.217514 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209288 2578 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 10 16:54:16.217514 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209291 2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 10 16:54:16.217514 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209294 2578 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 10 16:54:16.217514 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209297 2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 10 16:54:16.217514 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209300 2578 flags.go:64] FLAG: --storage-driver-password="root" Jul 10 16:54:16.217514 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209303 2578 flags.go:64] FLAG: --storage-driver-secure="false" Jul 10 16:54:16.217514 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209306 2578 flags.go:64] FLAG: --storage-driver-table="stats" Jul 10 16:54:16.217514 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209309 2578 flags.go:64] FLAG: --storage-driver-user="root" Jul 10 16:54:16.217514 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209312 2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 10 16:54:16.217514 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209315 2578 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 10 16:54:16.217514 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209318 2578 flags.go:64] FLAG: --system-cgroups="" Jul 10 16:54:16.217514 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209321 2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 10 16:54:16.217514 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209327 2578 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 10 16:54:16.217514 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209330 2578 flags.go:64] FLAG: --tls-cert-file="" Jul 10 16:54:16.217514 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209333 2578 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 10 16:54:16.217514 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209341 2578 flags.go:64] FLAG: --tls-min-version="" Jul 10 16:54:16.217514 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209344 2578 flags.go:64] FLAG: --tls-private-key-file="" Jul 10 16:54:16.217514 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209348 2578 flags.go:64] FLAG: --topology-manager-policy="none" Jul 10 16:54:16.217514 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209351 2578 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 10 16:54:16.217514 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209353 2578 flags.go:64] FLAG: --topology-manager-scope="container" Jul 10 16:54:16.217514 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209358 2578 flags.go:64] FLAG: --v="2" Jul 10 16:54:16.217514 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209363 2578 flags.go:64] FLAG: --version="false" Jul 10 16:54:16.218161 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209367 2578 flags.go:64] FLAG: --vmodule="" Jul 10 16:54:16.218161 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209371 2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 10 16:54:16.218161 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.209374 2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 10 16:54:16.218161 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209491 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 16:54:16.218161 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209495 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 16:54:16.218161 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209499 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 16:54:16.218161 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209502 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 16:54:16.218161 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209505 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 16:54:16.218161 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209508 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 16:54:16.218161 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209510 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 16:54:16.218161 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209513 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 16:54:16.218161 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209516 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 16:54:16.218161 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209519 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 16:54:16.218161 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209521 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 16:54:16.218161 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209524 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 16:54:16.218161 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209526 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 16:54:16.218161 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209529 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 16:54:16.218161 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209532 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 16:54:16.218161 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209534 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 16:54:16.218161 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209537 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 16:54:16.218688 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209540 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 16:54:16.218688 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209542 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 16:54:16.218688 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209545 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 16:54:16.218688 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209548 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 16:54:16.218688 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209551 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 16:54:16.218688 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209553 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 16:54:16.218688 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209556 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 16:54:16.218688 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209560 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 16:54:16.218688 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209563 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 10 16:54:16.218688 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209565 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 16:54:16.218688 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209569 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 16:54:16.218688 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209572 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 16:54:16.218688 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209575 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 16:54:16.218688 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209577 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 16:54:16.218688 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209580 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 16:54:16.218688 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209583 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 16:54:16.218688 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209585 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 16:54:16.218688 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209588 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 16:54:16.218688 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209590 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 16:54:16.218688 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209593 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 16:54:16.219290 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209595 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 16:54:16.219290 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209598 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 16:54:16.219290 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209601 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 16:54:16.219290 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209603 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 16:54:16.219290 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209606 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 16:54:16.219290 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209608 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 16:54:16.219290 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209611 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 16:54:16.219290 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209613 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 16:54:16.219290 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209616 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 16:54:16.219290 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209618 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 16:54:16.219290 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209621 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 16:54:16.219290 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209623 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 16:54:16.219290 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209626 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 16:54:16.219290 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209629 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 16:54:16.219290 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209631 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 16:54:16.219290 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209635 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 16:54:16.219290 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209639 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 16:54:16.219290 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209642 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 16:54:16.219290 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209645 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 16:54:16.219290 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209649 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 16:54:16.219761 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209652 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 16:54:16.219761 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209654 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 16:54:16.219761 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209659 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 16:54:16.219761 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209663 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 16:54:16.219761 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209666 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 16:54:16.219761 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209668 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 16:54:16.219761 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209671 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 16:54:16.219761 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209674 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 16:54:16.219761 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209676 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 16:54:16.219761 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209679 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 16:54:16.219761 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209681 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 16:54:16.219761 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209684 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 16:54:16.219761 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209686 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 16:54:16.219761 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209690 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 16:54:16.219761 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209694 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 16:54:16.219761 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209697 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 16:54:16.219761 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209699 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 16:54:16.219761 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209702 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 16:54:16.219761 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209705 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 16:54:16.220234 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209707 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 16:54:16.220234 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209710 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 16:54:16.220234 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209712 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 16:54:16.220234 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209715 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 16:54:16.220234 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209717 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 16:54:16.220234 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209720 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 16:54:16.220234 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209722 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 16:54:16.220234 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209725 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 16:54:16.220234 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209728 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 16:54:16.220234 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209731 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 16:54:16.220234 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209733 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 16:54:16.220234 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209736 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 16:54:16.220234 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209740 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 16:54:16.220234 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209742 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 16:54:16.220234 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209745 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 16:54:16.220234 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209748 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 16:54:16.220234 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209751 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 16:54:16.220234 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209753 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 16:54:16.220234 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209756 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 16:54:16.220234 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209759 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 16:54:16.220720 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209761 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 16:54:16.220720 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209764 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 16:54:16.220720 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.209766 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 16:54:16.220720 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.210248 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 16:54:16.220720 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.217024 2578 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 10 16:54:16.220720 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.217038 2578 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 10 16:54:16.220720 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.217729 2578 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 10 16:54:16.220720 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.217737 2578 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 10 16:54:16.220720 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217795 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 16:54:16.220720 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217802 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 16:54:16.220720 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217807 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 16:54:16.220720 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217811 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 16:54:16.220720 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217814 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 16:54:16.220720 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217818 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 16:54:16.221079 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217821 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 16:54:16.221079 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217824 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 16:54:16.221079 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217827 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 16:54:16.221079 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217831 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 16:54:16.221079 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217834 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 16:54:16.221079 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217838 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 16:54:16.221079 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217841 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 16:54:16.221079 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217843 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 16:54:16.221079 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217846 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 16:54:16.221079 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217849 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 16:54:16.221079 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217851 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 16:54:16.221079 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217855 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 16:54:16.221079 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217857 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 16:54:16.221079 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217860 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 16:54:16.221079 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217863 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 16:54:16.221079 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217866 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 16:54:16.221079 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217869 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 16:54:16.221079 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217871 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 16:54:16.221079 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217874 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 16:54:16.221079 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217876 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 16:54:16.221622 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217879 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 16:54:16.221622 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217881 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 16:54:16.221622 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217884 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 16:54:16.221622 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217888 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 16:54:16.221622 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217890 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 16:54:16.221622 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217893 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 16:54:16.221622 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217896 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 16:54:16.221622 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217898 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 16:54:16.221622 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217901 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 16:54:16.221622 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217904 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 16:54:16.221622 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217907 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 16:54:16.221622 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217910 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 16:54:16.221622 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217912 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 16:54:16.221622 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217915 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 16:54:16.221622 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217917 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 16:54:16.221622 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217920 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 16:54:16.221622 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217922 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 16:54:16.221622 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217925 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 16:54:16.221622 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217927 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 16:54:16.221622 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217930 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 16:54:16.222153 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217932 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 16:54:16.222153 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217935 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 16:54:16.222153 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217937 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 16:54:16.222153 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217940 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 16:54:16.222153 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217943 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 16:54:16.222153 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217945 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 16:54:16.222153 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217948 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 16:54:16.222153 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217951 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 16:54:16.222153 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217953 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 16:54:16.222153 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217956 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 16:54:16.222153 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217967 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 16:54:16.222153 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217970 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 16:54:16.222153 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217972 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 16:54:16.222153 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217975 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 16:54:16.222153 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217978 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 16:54:16.222153 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217981 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 16:54:16.222153 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217984 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 16:54:16.222153 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217987 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 16:54:16.222153 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217990 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 16:54:16.222153 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217992 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 16:54:16.222645 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217995 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 16:54:16.222645 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.217997 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 16:54:16.222645 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218000 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 16:54:16.222645 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218002 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 16:54:16.222645 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218005 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 16:54:16.222645 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218007 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 16:54:16.222645 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218010 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 16:54:16.222645 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218012 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 16:54:16.222645 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218015 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 16:54:16.222645 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218017 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 16:54:16.222645 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218020 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 16:54:16.222645 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218022 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 16:54:16.222645 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218024 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 16:54:16.222645 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218027 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 16:54:16.222645 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218030 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 16:54:16.222645 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218032 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 16:54:16.222645 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218035 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 16:54:16.222645 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218038 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 16:54:16.222645 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218040 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 16:54:16.222645 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218060 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 16:54:16.223345 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218063 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 16:54:16.223345 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218066 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 16:54:16.223345 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218069 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 16:54:16.223345 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218072 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 16:54:16.223345 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218074 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 16:54:16.223345 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218077 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 16:54:16.223345 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218079 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 16:54:16.223345 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218082 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 16:54:16.223345 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218084 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 16:54:16.223345 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218088 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 16:54:16.223345 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218091 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 16:54:16.223345 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218093 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 10 16:54:16.223345 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218096 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 16:54:16.223345 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.218101 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 16:54:16.223345 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218215 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 16:54:16.223716 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218221 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 16:54:16.223716 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218225 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 16:54:16.223716 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218228 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 16:54:16.223716 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218230 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 16:54:16.223716 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218233 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 16:54:16.223716 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218236 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 16:54:16.223716 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218238 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 16:54:16.223716 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218241 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 16:54:16.223716 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218244 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 16:54:16.223716 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218247 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 16:54:16.223716 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218249 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 16:54:16.223716 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218252 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 16:54:16.223716 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218255 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 16:54:16.223716 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218258 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 16:54:16.223716 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218261 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 16:54:16.223716 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218263 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 16:54:16.223716 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218266 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 16:54:16.223716 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218268 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 16:54:16.223716 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218271 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 16:54:16.223716 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218273 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 16:54:16.224220 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218276 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 16:54:16.224220 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218278 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 16:54:16.224220 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218280 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 16:54:16.224220 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218283 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 16:54:16.224220 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218285 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 16:54:16.224220 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218288 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 16:54:16.224220 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218290 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 16:54:16.224220 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218293 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 16:54:16.224220 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218296 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 16:54:16.224220 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218298 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 16:54:16.224220 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218301 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 16:54:16.224220 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218303 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 16:54:16.224220 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218306 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 16:54:16.224220 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218309 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 16:54:16.224220 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218311 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 16:54:16.224220 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218314 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 16:54:16.224220 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218316 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 16:54:16.224220 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218320 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 16:54:16.224220 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218324 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 16:54:16.224775 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218327 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 16:54:16.224775 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218330 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 16:54:16.224775 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218333 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 16:54:16.224775 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218335 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 16:54:16.224775 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218338 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 16:54:16.224775 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218341 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 16:54:16.224775 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218344 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 16:54:16.224775 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218348 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 16:54:16.224775 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218351 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 16:54:16.224775 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218354 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 16:54:16.224775 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218357 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 16:54:16.224775 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218361 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 16:54:16.224775 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218364 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 16:54:16.224775 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218366 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 16:54:16.224775 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218369 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 16:54:16.224775 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218372 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 16:54:16.224775 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218374 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 16:54:16.224775 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218377 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 10 16:54:16.224775 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218380 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 16:54:16.225250 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218382 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 16:54:16.225250 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218385 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 16:54:16.225250 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218388 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 16:54:16.225250 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218391 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 16:54:16.225250 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218394 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 16:54:16.225250 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218396 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 16:54:16.225250 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218399 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 16:54:16.225250 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218402 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 16:54:16.225250 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218405 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 16:54:16.225250 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218407 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 16:54:16.225250 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218410 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 16:54:16.225250 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218413 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 16:54:16.225250 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218416 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 16:54:16.225250 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218418 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 16:54:16.225250 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218421 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 16:54:16.225250 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218423 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 16:54:16.225250 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218426 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 16:54:16.225250 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218428 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 16:54:16.225250 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218431 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 16:54:16.225250 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218436 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 16:54:16.225250 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218438 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 16:54:16.226042 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218441 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 16:54:16.226042 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218443 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 16:54:16.226042 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218446 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 16:54:16.226042 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218448 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 16:54:16.226042 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218451 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 16:54:16.226042 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218454 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 16:54:16.226042 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218456 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 16:54:16.226042 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218458 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 16:54:16.226042 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218461 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 16:54:16.226042 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218463 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 16:54:16.226042 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218466 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 16:54:16.226042 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218468 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 16:54:16.226042 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218471 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 16:54:16.226042 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218473 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 16:54:16.226042 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218476 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 16:54:16.226042 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218479 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 16:54:16.226042 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218481 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 16:54:16.226042 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218484 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 16:54:16.226042 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:16.218486 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 16:54:16.226833 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.218492 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 16:54:16.226833 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.218604 2578 server.go:962] "Client rotation is on, will bootstrap in background" Jul 10 16:54:16.226833 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.221343 2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 10 16:54:16.226833 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.222754 2578 server.go:1019] "Starting client certificate rotation" Jul 10 16:54:16.226833 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.222875 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 10 16:54:16.226833 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.222936 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 10 16:54:16.226833 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.225763 2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 10 16:54:16.228487 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.228456 2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 10 16:54:16.240896 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.240875 2578 log.go:25] "Validated CRI v1 runtime API" Jul 10 16:54:16.243077 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.243041 2578 log.go:25] "Validated CRI v1 image API" Jul 10 16:54:16.244617 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.244603 2578 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 10 16:54:16.247895 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.247873 2578 fs.go:135] Filesystem UUIDs: map[67022127-721f-4161-b753-b91ef299155b:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 c9cf69ee-459d-4e51-92b0-42333175e59e:/dev/nvme0n1p3] Jul 10 16:54:16.247968 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.247893 2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 10 16:54:16.252281 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.252173 2578 manager.go:216] Machine: {Timestamp:2026-07-10 16:54:16.251656549 +0000 UTC m=+0.340985590 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3115512 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec211351a2aa3b9156ff0dfc7c0ab124 SystemUUID:ec211351-a2aa-3b91-56ff-0dfc7c0ab124 BootID:ad8b4b0e-28b5-415b-9970-306a319d596a Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ce:f0:6a:b4:b3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ce:f0:6a:b4:b3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:e2:65:7a:7d:6e:32 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 10 16:54:16.252281 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.252274 2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 10 16:54:16.252416 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.252355 2578 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 10 16:54:16.255151 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.255117 2578 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 10 16:54:16.255294 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.255155 2578 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-7.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 10 16:54:16.255644 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.255627 2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 10 16:54:16.255973 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.255962 2578 topology_manager.go:138] "Creating topology manager with none policy" Jul 10 16:54:16.256007 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.255976 2578 container_manager_linux.go:309] "Creating device plugin manager" Jul 10 16:54:16.256007 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.255984 2578 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 10 16:54:16.256707 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.256697 2578 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 10 16:54:16.257498 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.257488 2578 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 10 16:54:16.260260 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.260249 2578 state_mem.go:36] "Initialized new in-memory state store" Jul 10 16:54:16.260575 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.260565 2578 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 10 16:54:16.260608 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.260602 2578 kubelet.go:486] "Attempting to sync node with API server" Jul 10 16:54:16.260634 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.260615 2578 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 10 16:54:16.260634 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.260630 2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 10 16:54:16.260712 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.260639 2578 kubelet.go:398] "Adding apiserver pod source" Jul 10 16:54:16.260712 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.260648 2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 10 16:54:16.261006 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.260995 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 10 16:54:16.261067 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.261012 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 10 16:54:16.264059 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.264027 2578 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 10 16:54:16.265431 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.265416 2578 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 10 16:54:16.265481 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.265449 2578 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 10 16:54:16.266062 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.266035 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 10 16:54:16.266095 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.266071 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 10 16:54:16.266095 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.266081 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 10 16:54:16.266095 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.266089 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 10 16:54:16.266185 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.266101 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 10 16:54:16.266185 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.266109 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 10 16:54:16.266185 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.266149 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 10 16:54:16.266185 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.266156 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 10 16:54:16.266185 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.266163 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 10 16:54:16.266185 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.266169 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 10 16:54:16.266185 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.266178 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 10 16:54:16.266185 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.266186 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 10 16:54:16.266906 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.266894 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 10 16:54:16.266906 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.266907 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 10 16:54:16.270937 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.270923 2578 server.go:1268] "Started kubelet" Jul 10 16:54:16.271130 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.271078 2578 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 10 16:54:16.271296 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.271260 2578 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 10 16:54:16.271296 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.271092 2578 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 10 16:54:16.271814 ip-10-0-136-7 systemd[1]: Started Kubernetes Kubelet. Jul 10 16:54:16.272567 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.272551 2578 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 10 16:54:16.273337 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.273324 2578 server.go:310] "Adding debug handlers to kubelet server" Jul 10 16:54:16.273427 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.273411 2578 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-7.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 10 16:54:16.273504 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:16.273486 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-7.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 10 16:54:16.273606 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:16.273581 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 10 16:54:16.275198 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:16.275178 2578 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 10 16:54:16.276316 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.276301 2578 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 10 16:54:16.276386 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.276319 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 10 16:54:16.276386 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:16.276369 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-7.ec2.internal\" not found" Jul 10 16:54:16.277022 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.277007 2578 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 10 16:54:16.277022 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.277023 2578 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 10 16:54:16.277162 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.277020 2578 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 10 16:54:16.277224 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.277185 2578 reconstruct.go:97] "Volume reconstruction finished" Jul 10 16:54:16.277224 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.277193 2578 reconciler.go:29] "Reconciler: start to sync state" Jul 10 16:54:16.277341 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.277327 2578 factory.go:55] Registering systemd factory Jul 10 16:54:16.277384 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.277348 2578 factory.go:223] Registration of the systemd container factory successfully Jul 10 16:54:16.277634 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.277617 2578 factory.go:153] Registering CRI-O factory Jul 10 16:54:16.277634 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.277636 2578 factory.go:223] Registration of the crio container factory successfully Jul 10 16:54:16.277777 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.277703 2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 10 16:54:16.277777 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.277730 2578 factory.go:103] Registering Raw factory Jul 10 16:54:16.277777 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.277747 2578 manager.go:1196] Started watching for new ooms in manager Jul 10 16:54:16.278150 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.278136 2578 manager.go:349] Starting recovery of all containers Jul 10 16:54:16.279477 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:16.279433 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 10 16:54:16.279723 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:16.279699 2578 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-7.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 10 16:54:16.281141 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:16.279509 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-7.ec2.internal.18c0fbeefb048d53 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-7.ec2.internal,UID:ip-10-0-136-7.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-7.ec2.internal,},FirstTimestamp:2026-07-10 16:54:16.270884179 +0000 UTC m=+0.360213220,LastTimestamp:2026-07-10 16:54:16.270884179 +0000 UTC m=+0.360213220,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-7.ec2.internal,}" Jul 10 16:54:16.288135 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.287947 2578 manager.go:354] Recovery completed Jul 10 16:54:16.294332 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.294319 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 16:54:16.296645 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.296626 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-7.ec2.internal" event="NodeHasSufficientMemory" Jul 10 16:54:16.296707 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.296664 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-7.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 16:54:16.296707 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.296675 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-7.ec2.internal" event="NodeHasSufficientPID" Jul 10 16:54:16.297271 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.297256 2578 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 10 16:54:16.297271 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.297270 2578 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 10 16:54:16.297407 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.297280 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 16:54:16.297407 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.297287 2578 state_mem.go:36] "Initialized new in-memory state store" Jul 10 16:54:16.298072 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.298033 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-7.ec2.internal" event="NodeHasSufficientMemory" Jul 10 16:54:16.298129 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.298079 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-7.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 16:54:16.298129 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.298090 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-7.ec2.internal" event="NodeHasSufficientPID" Jul 10 16:54:16.298630 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.298619 2578 policy_none.go:49] "None policy: Start" Jul 10 16:54:16.298662 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.298636 2578 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 10 16:54:16.298662 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.298646 2578 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 10 16:54:16.299349 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:16.299271 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-7.ec2.internal.18c0fbeefc8dafac default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-7.ec2.internal,UID:ip-10-0-136-7.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-7.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-7.ec2.internal,},FirstTimestamp:2026-07-10 16:54:16.29664862 +0000 UTC m=+0.385977664,LastTimestamp:2026-07-10 16:54:16.29664862 +0000 UTC m=+0.385977664,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-7.ec2.internal,}" Jul 10 16:54:16.299622 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.299609 2578 policy_none.go:47] "Start" Jul 10 16:54:16.310425 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:16.310359 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-7.ec2.internal.18c0fbeefc8e0016 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-7.ec2.internal,UID:ip-10-0-136-7.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-136-7.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-136-7.ec2.internal,},FirstTimestamp:2026-07-10 16:54:16.296669206 +0000 UTC m=+0.385998247,LastTimestamp:2026-07-10 16:54:16.296669206 +0000 UTC m=+0.385998247,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-7.ec2.internal,}" Jul 10 16:54:16.318939 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.318918 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rjj6x" Jul 10 16:54:16.320832 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:16.320752 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-7.ec2.internal.18c0fbeefc8e27c2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-7.ec2.internal,UID:ip-10-0-136-7.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-136-7.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-136-7.ec2.internal,},FirstTimestamp:2026-07-10 16:54:16.296679362 +0000 UTC m=+0.386008402,LastTimestamp:2026-07-10 16:54:16.296679362 +0000 UTC m=+0.386008402,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-7.ec2.internal,}" Jul 10 16:54:16.328932 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.328914 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rjj6x" Jul 10 16:54:16.332191 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:16.332122 2578 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-136-7.ec2.internal.18c0fbeefc8dafac\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-7.ec2.internal.18c0fbeefc8dafac default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-7.ec2.internal,UID:ip-10-0-136-7.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-7.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-7.ec2.internal,},FirstTimestamp:2026-07-10 16:54:16.29664862 +0000 UTC m=+0.385977664,LastTimestamp:2026-07-10 16:54:16.298063445 +0000 UTC m=+0.387392492,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-7.ec2.internal,}" Jul 10 16:54:16.346712 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.346695 2578 manager.go:337] "Starting Device Plugin manager" Jul 10 16:54:16.346788 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:16.346732 2578 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 10 16:54:16.346788 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.346742 2578 server.go:87] "Starting device plugin registration server" Jul 10 16:54:16.347000 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.346977 2578 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 10 16:54:16.347074 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.346990 2578 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 10 16:54:16.347134 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.347072 2578 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 10 16:54:16.347177 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.347162 2578 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 10 16:54:16.347177 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.347174 2578 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 10 16:54:16.347702 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:16.347682 2578 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 10 16:54:16.347764 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:16.347727 2578 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-7.ec2.internal\" not found" Jul 10 16:54:16.417846 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.417807 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 10 16:54:16.419251 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.419231 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 10 16:54:16.419366 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.419265 2578 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 10 16:54:16.419366 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.419293 2578 kubelet.go:2448] "Starting kubelet main sync loop" Jul 10 16:54:16.419366 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:16.419331 2578 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 10 16:54:16.422203 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.422186 2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 16:54:16.447783 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.447761 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 16:54:16.449841 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.449824 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-7.ec2.internal" event="NodeHasSufficientMemory" Jul 10 16:54:16.449941 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.449857 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-7.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 16:54:16.449941 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.449870 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-7.ec2.internal" event="NodeHasSufficientPID" Jul 10 16:54:16.449941 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.449893 2578 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-7.ec2.internal" Jul 10 16:54:16.460108 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.460061 2578 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-7.ec2.internal" Jul 10 16:54:16.460108 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:16.460086 2578 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-7.ec2.internal\": node \"ip-10-0-136-7.ec2.internal\" not found" Jul 10 16:54:16.508126 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:16.508094 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-7.ec2.internal\" not found" Jul 10 16:54:16.520159 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.520131 2578 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-7.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-7.ec2.internal"] Jul 10 16:54:16.520233 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.520219 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 16:54:16.521790 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.521774 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-7.ec2.internal" event="NodeHasSufficientMemory" Jul 10 16:54:16.521887 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.521804 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-7.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 16:54:16.521887 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.521813 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-7.ec2.internal" event="NodeHasSufficientPID" Jul 10 16:54:16.523132 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.523115 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 16:54:16.523269 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.523251 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-7.ec2.internal" Jul 10 16:54:16.523377 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.523290 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 16:54:16.523913 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.523895 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-7.ec2.internal" event="NodeHasSufficientMemory" Jul 10 16:54:16.523980 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.523924 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-7.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 16:54:16.523980 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.523934 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-7.ec2.internal" event="NodeHasSufficientPID" Jul 10 16:54:16.523980 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.523938 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-7.ec2.internal" event="NodeHasSufficientMemory" Jul 10 16:54:16.523980 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.523958 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-7.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 16:54:16.523980 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.523968 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-7.ec2.internal" event="NodeHasSufficientPID" Jul 10 16:54:16.525148 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.525132 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-7.ec2.internal" Jul 10 16:54:16.525247 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.525163 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 16:54:16.525928 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.525914 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-7.ec2.internal" event="NodeHasSufficientMemory" Jul 10 16:54:16.526010 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.525939 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-7.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 16:54:16.526010 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.525950 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-7.ec2.internal" event="NodeHasSufficientPID" Jul 10 16:54:16.545802 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:16.545779 2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-7.ec2.internal\" not found" node="ip-10-0-136-7.ec2.internal" Jul 10 16:54:16.550562 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:16.550543 2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-7.ec2.internal\" not found" node="ip-10-0-136-7.ec2.internal" Jul 10 16:54:16.578529 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.578494 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ac3f323c6b38ca654d9468732cf2193c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-7.ec2.internal\" (UID: \"ac3f323c6b38ca654d9468732cf2193c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-7.ec2.internal" Jul 10 16:54:16.578529 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.578529 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ac3f323c6b38ca654d9468732cf2193c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-7.ec2.internal\" (UID: \"ac3f323c6b38ca654d9468732cf2193c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-7.ec2.internal" Jul 10 16:54:16.578714 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.578548 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/374e81600297ab5c614c91e79771b865-config\") pod \"kube-apiserver-proxy-ip-10-0-136-7.ec2.internal\" (UID: \"374e81600297ab5c614c91e79771b865\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-7.ec2.internal" Jul 10 16:54:16.608756 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:16.608718 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-7.ec2.internal\" not found" Jul 10 16:54:16.679468 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.679442 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ac3f323c6b38ca654d9468732cf2193c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-7.ec2.internal\" (UID: \"ac3f323c6b38ca654d9468732cf2193c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-7.ec2.internal" Jul 10 16:54:16.679552 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.679474 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ac3f323c6b38ca654d9468732cf2193c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-7.ec2.internal\" (UID: \"ac3f323c6b38ca654d9468732cf2193c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-7.ec2.internal" Jul 10 16:54:16.679552 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.679510 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/374e81600297ab5c614c91e79771b865-config\") pod \"kube-apiserver-proxy-ip-10-0-136-7.ec2.internal\" (UID: \"374e81600297ab5c614c91e79771b865\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-7.ec2.internal" Jul 10 16:54:16.679552 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.679544 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ac3f323c6b38ca654d9468732cf2193c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-7.ec2.internal\" (UID: \"ac3f323c6b38ca654d9468732cf2193c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-7.ec2.internal" Jul 10 16:54:16.679646 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.679551 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/374e81600297ab5c614c91e79771b865-config\") pod \"kube-apiserver-proxy-ip-10-0-136-7.ec2.internal\" (UID: \"374e81600297ab5c614c91e79771b865\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-7.ec2.internal" Jul 10 16:54:16.679646 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.679557 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ac3f323c6b38ca654d9468732cf2193c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-7.ec2.internal\" (UID: \"ac3f323c6b38ca654d9468732cf2193c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-7.ec2.internal" Jul 10 16:54:16.708867 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:16.708810 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-7.ec2.internal\" not found" Jul 10 16:54:16.809707 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:16.809667 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-7.ec2.internal\" not found" Jul 10 16:54:16.848888 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.848850 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-7.ec2.internal" Jul 10 16:54:16.853103 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:16.853088 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-7.ec2.internal" Jul 10 16:54:16.909940 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:16.909910 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-7.ec2.internal\" not found" Jul 10 16:54:17.010518 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:17.010466 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-7.ec2.internal\" not found" Jul 10 16:54:17.111008 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:17.110926 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-7.ec2.internal\" not found" Jul 10 16:54:17.185983 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.185950 2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 16:54:17.211264 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:17.211229 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-7.ec2.internal\" not found" Jul 10 16:54:17.222190 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.222154 2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 10 16:54:17.222353 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.222334 2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 10 16:54:17.222394 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.222366 2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 10 16:54:17.250470 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.250440 2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 16:54:17.260894 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.260860 2578 apiserver.go:52] "Watching apiserver" Jul 10 16:54:17.270835 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.270805 2578 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 10 16:54:17.271189 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.271160 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-gqgkg","openshift-cluster-node-tuning-operator/tuned-djhll","openshift-dns/node-resolver-6kk9g","openshift-image-registry/node-ca-w9kb4","openshift-network-operator/iptables-alerter-sdvff","kube-system/global-pull-secret-syncer-dgk2w","kube-system/konnectivity-agent-t6rhv","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8tnf","openshift-multus/multus-9qtfh","openshift-multus/multus-additional-cni-plugins-5d4nh","openshift-multus/network-metrics-daemon-jr8jk","openshift-network-diagnostics/network-check-target-jvkw6"] Jul 10 16:54:17.272752 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.272737 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.273913 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.273891 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6kk9g" Jul 10 16:54:17.275120 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.275097 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-sdvff" Jul 10 16:54:17.276212 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.276194 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-t6rhv" Jul 10 16:54:17.276403 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.276388 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 10 16:54:17.276752 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.276739 2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-7.ec2.internal" Jul 10 16:54:17.277141 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.277110 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 10 16:54:17.277188 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.277112 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 10 16:54:17.277433 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.277417 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8tnf" Jul 10 16:54:17.278751 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.278734 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9qtfh" Jul 10 16:54:17.280004 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.279984 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5d4nh" Jul 10 16:54:17.281203 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.281185 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jr8jk" Jul 10 16:54:17.281299 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:17.281248 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jr8jk" podUID="d43ce12f-a8fd-4a94-bb94-09e6562e1250" Jul 10 16:54:17.281643 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.281629 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 10 16:54:17.282006 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.281991 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b9a53126-f278-4a9d-9735-5f73dd0c1964-host-kubelet\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.282059 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.282017 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b9a53126-f278-4a9d-9735-5f73dd0c1964-var-lib-openvswitch\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.282059 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.282039 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b9a53126-f278-4a9d-9735-5f73dd0c1964-run-ovn\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.282152 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.282082 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gn762\" (UniqueName: \"kubernetes.io/projected/b9a53126-f278-4a9d-9735-5f73dd0c1964-kube-api-access-gn762\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.282152 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.282118 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/dc38a8bd-42b1-4809-88ce-66aa03182c8c-registration-dir\") pod \"aws-ebs-csi-driver-node-l8tnf\" (UID: \"dc38a8bd-42b1-4809-88ce-66aa03182c8c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8tnf" Jul 10 16:54:17.282259 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.282167 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqkhg\" (UniqueName: \"kubernetes.io/projected/dc38a8bd-42b1-4809-88ce-66aa03182c8c-kube-api-access-tqkhg\") pod \"aws-ebs-csi-driver-node-l8tnf\" (UID: \"dc38a8bd-42b1-4809-88ce-66aa03182c8c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8tnf" Jul 10 16:54:17.282259 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.282196 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b9a53126-f278-4a9d-9735-5f73dd0c1964-node-log\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.282259 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.282223 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b9a53126-f278-4a9d-9735-5f73dd0c1964-host-cni-bin\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.282259 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.282248 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b9a53126-f278-4a9d-9735-5f73dd0c1964-env-overrides\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.282416 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.282283 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-6jdbp\"" Jul 10 16:54:17.282416 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.282286 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b9a53126-f278-4a9d-9735-5f73dd0c1964-ovnkube-script-lib\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.282416 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.282333 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d85dbfa-1b16-4e26-b207-6bb7064345ee-tmp-dir\") pod \"node-resolver-6kk9g\" (UID: \"1d85dbfa-1b16-4e26-b207-6bb7064345ee\") " pod="openshift-dns/node-resolver-6kk9g" Jul 10 16:54:17.282416 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.282357 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/08351a06-c5eb-4c51-923c-283f5b28a7d2-konnectivity-ca\") pod \"konnectivity-agent-t6rhv\" (UID: \"08351a06-c5eb-4c51-923c-283f5b28a7d2\") " pod="kube-system/konnectivity-agent-t6rhv" Jul 10 16:54:17.282416 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.282373 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b9a53126-f278-4a9d-9735-5f73dd0c1964-log-socket\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.282416 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.282394 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b9a53126-f278-4a9d-9735-5f73dd0c1964-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.282642 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.282442 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b9a53126-f278-4a9d-9735-5f73dd0c1964-ovn-node-metrics-cert\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.282642 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.282461 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1d85dbfa-1b16-4e26-b207-6bb7064345ee-hosts-file\") pod \"node-resolver-6kk9g\" (UID: \"1d85dbfa-1b16-4e26-b207-6bb7064345ee\") " pod="openshift-dns/node-resolver-6kk9g" Jul 10 16:54:17.282642 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.282476 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cea5b1b2-0b38-41e0-ac8d-7e573be5ddb8-iptables-alerter-script\") pod \"iptables-alerter-sdvff\" (UID: \"cea5b1b2-0b38-41e0-ac8d-7e573be5ddb8\") " pod="openshift-network-operator/iptables-alerter-sdvff" Jul 10 16:54:17.282642 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.282483 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 10 16:54:17.282642 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.282550 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jvkw6" Jul 10 16:54:17.282642 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.282550 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/dc38a8bd-42b1-4809-88ce-66aa03182c8c-device-dir\") pod \"aws-ebs-csi-driver-node-l8tnf\" (UID: \"dc38a8bd-42b1-4809-88ce-66aa03182c8c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8tnf" Jul 10 16:54:17.282642 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.282589 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/dc38a8bd-42b1-4809-88ce-66aa03182c8c-etc-selinux\") pod \"aws-ebs-csi-driver-node-l8tnf\" (UID: \"dc38a8bd-42b1-4809-88ce-66aa03182c8c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8tnf" Jul 10 16:54:17.282642 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:17.282617 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jvkw6" podUID="16033c9b-cf7b-4afc-abe6-df0ded50ff80" Jul 10 16:54:17.282642 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.282640 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b9a53126-f278-4a9d-9735-5f73dd0c1964-host-run-netns\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.282984 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.282664 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-94qtp\"" Jul 10 16:54:17.282984 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.282666 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b9a53126-f278-4a9d-9735-5f73dd0c1964-ovnkube-config\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.282984 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.282698 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 10 16:54:17.282984 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.282710 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/dc38a8bd-42b1-4809-88ce-66aa03182c8c-sys-fs\") pod \"aws-ebs-csi-driver-node-l8tnf\" (UID: \"dc38a8bd-42b1-4809-88ce-66aa03182c8c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8tnf" Jul 10 16:54:17.282984 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.282737 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b9a53126-f278-4a9d-9735-5f73dd0c1964-systemd-units\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.282984 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.282764 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b9a53126-f278-4a9d-9735-5f73dd0c1964-etc-openvswitch\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.282984 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.282800 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b9a53126-f278-4a9d-9735-5f73dd0c1964-host-cni-netd\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.282984 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.282832 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b9a53126-f278-4a9d-9735-5f73dd0c1964-host-slash\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.282984 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.282870 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b9a53126-f278-4a9d-9735-5f73dd0c1964-run-systemd\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.282984 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.282906 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b9a53126-f278-4a9d-9735-5f73dd0c1964-run-openvswitch\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.282984 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.282940 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-jdbq9\"" Jul 10 16:54:17.282984 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.282958 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b9a53126-f278-4a9d-9735-5f73dd0c1964-host-run-ovn-kubernetes\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.282984 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.282978 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tt6z5\" (UniqueName: \"kubernetes.io/projected/1d85dbfa-1b16-4e26-b207-6bb7064345ee-kube-api-access-tt6z5\") pod \"node-resolver-6kk9g\" (UID: \"1d85dbfa-1b16-4e26-b207-6bb7064345ee\") " pod="openshift-dns/node-resolver-6kk9g" Jul 10 16:54:17.283466 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.282993 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cea5b1b2-0b38-41e0-ac8d-7e573be5ddb8-host-slash\") pod \"iptables-alerter-sdvff\" (UID: \"cea5b1b2-0b38-41e0-ac8d-7e573be5ddb8\") " pod="openshift-network-operator/iptables-alerter-sdvff" Jul 10 16:54:17.283466 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.283010 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2h2db\" (UniqueName: \"kubernetes.io/projected/cea5b1b2-0b38-41e0-ac8d-7e573be5ddb8-kube-api-access-2h2db\") pod \"iptables-alerter-sdvff\" (UID: \"cea5b1b2-0b38-41e0-ac8d-7e573be5ddb8\") " pod="openshift-network-operator/iptables-alerter-sdvff" Jul 10 16:54:17.283466 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.283035 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/dc38a8bd-42b1-4809-88ce-66aa03182c8c-socket-dir\") pod \"aws-ebs-csi-driver-node-l8tnf\" (UID: \"dc38a8bd-42b1-4809-88ce-66aa03182c8c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8tnf" Jul 10 16:54:17.283466 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.283071 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/08351a06-c5eb-4c51-923c-283f5b28a7d2-agent-certs\") pod \"konnectivity-agent-t6rhv\" (UID: \"08351a06-c5eb-4c51-923c-283f5b28a7d2\") " pod="kube-system/konnectivity-agent-t6rhv" Jul 10 16:54:17.283466 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.283094 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/dc38a8bd-42b1-4809-88ce-66aa03182c8c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-l8tnf\" (UID: \"dc38a8bd-42b1-4809-88ce-66aa03182c8c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8tnf" Jul 10 16:54:17.283466 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.283316 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 10 16:54:17.283724 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.283711 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 10 16:54:17.283932 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.283917 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-djhll" Jul 10 16:54:17.284991 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.284976 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-w9kb4" Jul 10 16:54:17.286211 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.286191 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgk2w" Jul 10 16:54:17.286284 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:17.286259 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dgk2w" podUID="a0b23658-26fb-454a-a42e-88432bf12b0a" Jul 10 16:54:17.289394 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.289365 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 10 16:54:17.293065 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.293047 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 10 16:54:17.293167 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.293083 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 10 16:54:17.293167 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.293094 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 10 16:54:17.293167 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.293054 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 10 16:54:17.293323 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.293087 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 10 16:54:17.293323 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.293186 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 10 16:54:17.293451 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.293434 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 10 16:54:17.293497 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.293437 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 10 16:54:17.293542 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.293525 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 10 16:54:17.293606 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.293591 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 10 16:54:17.294669 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.294110 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 10 16:54:17.294669 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.294477 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-stnzf\"" Jul 10 16:54:17.294669 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.294523 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 10 16:54:17.294875 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.294807 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 10 16:54:17.296828 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.295257 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-vrzvb\"" Jul 10 16:54:17.296828 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.295477 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-f8qx5\"" Jul 10 16:54:17.296828 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.295683 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 10 16:54:17.296828 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.295844 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-zzq62\"" Jul 10 16:54:17.296828 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.295909 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 10 16:54:17.296828 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.296074 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 10 16:54:17.296828 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.296326 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-chnvg\"" Jul 10 16:54:17.296828 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.296505 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 10 16:54:17.303251 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.303233 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 10 16:54:17.308396 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.308375 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 10 16:54:17.308670 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.308647 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-t4fz8\"" Jul 10 16:54:17.312673 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.312657 2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 10 16:54:17.322182 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.322166 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 10 16:54:17.322263 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.322233 2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-7.ec2.internal" Jul 10 16:54:17.322388 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.322367 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-7.ec2.internal"] Jul 10 16:54:17.333153 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.333096 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-09 16:49:16 +0000 UTC" deadline="2028-03-31 11:34:10.66759914 +0000 UTC" Jul 10 16:54:17.333153 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.333149 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15114h39m53.334452666s" Jul 10 16:54:17.342692 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.342671 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-136-7.ec2.internal"] Jul 10 16:54:17.342985 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.342965 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 10 16:54:17.377322 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.377294 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-6hmpj" Jul 10 16:54:17.378022 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.378007 2578 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 10 16:54:17.383506 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.383481 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b9a53126-f278-4a9d-9735-5f73dd0c1964-ovn-node-metrics-cert\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.383506 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.383508 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/dc38a8bd-42b1-4809-88ce-66aa03182c8c-device-dir\") pod \"aws-ebs-csi-driver-node-l8tnf\" (UID: \"dc38a8bd-42b1-4809-88ce-66aa03182c8c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8tnf" Jul 10 16:54:17.383719 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.383527 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/dc38a8bd-42b1-4809-88ce-66aa03182c8c-etc-selinux\") pod \"aws-ebs-csi-driver-node-l8tnf\" (UID: \"dc38a8bd-42b1-4809-88ce-66aa03182c8c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8tnf" Jul 10 16:54:17.383719 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.383549 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/53987bf8-a205-4326-8757-ea8b3a6c4514-host\") pod \"node-ca-w9kb4\" (UID: \"53987bf8-a205-4326-8757-ea8b3a6c4514\") " pod="openshift-image-registry/node-ca-w9kb4" Jul 10 16:54:17.383719 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.383567 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b9a53126-f278-4a9d-9735-5f73dd0c1964-ovnkube-config\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.383719 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.383619 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/dc38a8bd-42b1-4809-88ce-66aa03182c8c-device-dir\") pod \"aws-ebs-csi-driver-node-l8tnf\" (UID: \"dc38a8bd-42b1-4809-88ce-66aa03182c8c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8tnf" Jul 10 16:54:17.383719 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.383626 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a0b23658-26fb-454a-a42e-88432bf12b0a-kubelet-config\") pod \"global-pull-secret-syncer-dgk2w\" (UID: \"a0b23658-26fb-454a-a42e-88432bf12b0a\") " pod="kube-system/global-pull-secret-syncer-dgk2w" Jul 10 16:54:17.383719 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.383707 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/dc38a8bd-42b1-4809-88ce-66aa03182c8c-etc-selinux\") pod \"aws-ebs-csi-driver-node-l8tnf\" (UID: \"dc38a8bd-42b1-4809-88ce-66aa03182c8c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8tnf" Jul 10 16:54:17.384015 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.383774 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1-cni-binary-copy\") pod \"multus-9qtfh\" (UID: \"a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1\") " pod="openshift-multus/multus-9qtfh" Jul 10 16:54:17.384015 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.383810 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1-host-run-k8s-cni-cncf-io\") pod \"multus-9qtfh\" (UID: \"a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1\") " pod="openshift-multus/multus-9qtfh" Jul 10 16:54:17.384015 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.383841 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ed7de755-820a-4395-8b5f-ea6abfb86699-var-lib-kubelet\") pod \"tuned-djhll\" (UID: \"ed7de755-820a-4395-8b5f-ea6abfb86699\") " pod="openshift-cluster-node-tuning-operator/tuned-djhll" Jul 10 16:54:17.384015 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.383869 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flfl2\" (UniqueName: \"kubernetes.io/projected/ed7de755-820a-4395-8b5f-ea6abfb86699-kube-api-access-flfl2\") pod \"tuned-djhll\" (UID: \"ed7de755-820a-4395-8b5f-ea6abfb86699\") " pod="openshift-cluster-node-tuning-operator/tuned-djhll" Jul 10 16:54:17.384015 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.383899 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6e1806e2-a453-4929-8632-267349af86c7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5d4nh\" (UID: \"6e1806e2-a453-4929-8632-267349af86c7\") " pod="openshift-multus/multus-additional-cni-plugins-5d4nh" Jul 10 16:54:17.384015 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.383906 2578 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 10 16:54:17.384015 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.383923 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ed7de755-820a-4395-8b5f-ea6abfb86699-etc-modprobe-d\") pod \"tuned-djhll\" (UID: \"ed7de755-820a-4395-8b5f-ea6abfb86699\") " pod="openshift-cluster-node-tuning-operator/tuned-djhll" Jul 10 16:54:17.384015 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.383949 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1-etc-kubernetes\") pod \"multus-9qtfh\" (UID: \"a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1\") " pod="openshift-multus/multus-9qtfh" Jul 10 16:54:17.384015 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.384004 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6e1806e2-a453-4929-8632-267349af86c7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5d4nh\" (UID: \"6e1806e2-a453-4929-8632-267349af86c7\") " pod="openshift-multus/multus-additional-cni-plugins-5d4nh" Jul 10 16:54:17.384490 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.384032 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6e1806e2-a453-4929-8632-267349af86c7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5d4nh\" (UID: \"6e1806e2-a453-4929-8632-267349af86c7\") " pod="openshift-multus/multus-additional-cni-plugins-5d4nh" Jul 10 16:54:17.384490 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.384088 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b9a53126-f278-4a9d-9735-5f73dd0c1964-host-slash\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.384490 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.384115 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b9a53126-f278-4a9d-9735-5f73dd0c1964-run-openvswitch\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.384490 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.384143 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tt6z5\" (UniqueName: \"kubernetes.io/projected/1d85dbfa-1b16-4e26-b207-6bb7064345ee-kube-api-access-tt6z5\") pod \"node-resolver-6kk9g\" (UID: \"1d85dbfa-1b16-4e26-b207-6bb7064345ee\") " pod="openshift-dns/node-resolver-6kk9g" Jul 10 16:54:17.384490 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.384167 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cea5b1b2-0b38-41e0-ac8d-7e573be5ddb8-host-slash\") pod \"iptables-alerter-sdvff\" (UID: \"cea5b1b2-0b38-41e0-ac8d-7e573be5ddb8\") " pod="openshift-network-operator/iptables-alerter-sdvff" Jul 10 16:54:17.384490 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.384191 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/53987bf8-a205-4326-8757-ea8b3a6c4514-serviceca\") pod \"node-ca-w9kb4\" (UID: \"53987bf8-a205-4326-8757-ea8b3a6c4514\") " pod="openshift-image-registry/node-ca-w9kb4" Jul 10 16:54:17.384490 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.384197 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b9a53126-f278-4a9d-9735-5f73dd0c1964-host-slash\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.384490 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.384214 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1-cnibin\") pod \"multus-9qtfh\" (UID: \"a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1\") " pod="openshift-multus/multus-9qtfh" Jul 10 16:54:17.384490 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.384286 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cea5b1b2-0b38-41e0-ac8d-7e573be5ddb8-host-slash\") pod \"iptables-alerter-sdvff\" (UID: \"cea5b1b2-0b38-41e0-ac8d-7e573be5ddb8\") " pod="openshift-network-operator/iptables-alerter-sdvff" Jul 10 16:54:17.384490 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.384377 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b9a53126-f278-4a9d-9735-5f73dd0c1964-run-openvswitch\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.384490 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.384406 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1-os-release\") pod \"multus-9qtfh\" (UID: \"a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1\") " pod="openshift-multus/multus-9qtfh" Jul 10 16:54:17.384490 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.384432 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6e1806e2-a453-4929-8632-267349af86c7-cni-binary-copy\") pod \"multus-additional-cni-plugins-5d4nh\" (UID: \"6e1806e2-a453-4929-8632-267349af86c7\") " pod="openshift-multus/multus-additional-cni-plugins-5d4nh" Jul 10 16:54:17.384490 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.384459 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6e1806e2-a453-4929-8632-267349af86c7-cnibin\") pod \"multus-additional-cni-plugins-5d4nh\" (UID: \"6e1806e2-a453-4929-8632-267349af86c7\") " pod="openshift-multus/multus-additional-cni-plugins-5d4nh" Jul 10 16:54:17.384490 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.384483 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ed7de755-820a-4395-8b5f-ea6abfb86699-etc-systemd\") pod \"tuned-djhll\" (UID: \"ed7de755-820a-4395-8b5f-ea6abfb86699\") " pod="openshift-cluster-node-tuning-operator/tuned-djhll" Jul 10 16:54:17.385156 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.384512 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b9a53126-f278-4a9d-9735-5f73dd0c1964-host-kubelet\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.385156 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.384541 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/dc38a8bd-42b1-4809-88ce-66aa03182c8c-registration-dir\") pod \"aws-ebs-csi-driver-node-l8tnf\" (UID: \"dc38a8bd-42b1-4809-88ce-66aa03182c8c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8tnf" Jul 10 16:54:17.385156 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.384559 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b9a53126-f278-4a9d-9735-5f73dd0c1964-host-kubelet\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.385156 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.384566 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ed7de755-820a-4395-8b5f-ea6abfb86699-etc-kubernetes\") pod \"tuned-djhll\" (UID: \"ed7de755-820a-4395-8b5f-ea6abfb86699\") " pod="openshift-cluster-node-tuning-operator/tuned-djhll" Jul 10 16:54:17.385156 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.384603 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ed7de755-820a-4395-8b5f-ea6abfb86699-run\") pod \"tuned-djhll\" (UID: \"ed7de755-820a-4395-8b5f-ea6abfb86699\") " pod="openshift-cluster-node-tuning-operator/tuned-djhll" Jul 10 16:54:17.385156 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.384625 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ed7de755-820a-4395-8b5f-ea6abfb86699-sys\") pod \"tuned-djhll\" (UID: \"ed7de755-820a-4395-8b5f-ea6abfb86699\") " pod="openshift-cluster-node-tuning-operator/tuned-djhll" Jul 10 16:54:17.385156 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.384631 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/dc38a8bd-42b1-4809-88ce-66aa03182c8c-registration-dir\") pod \"aws-ebs-csi-driver-node-l8tnf\" (UID: \"dc38a8bd-42b1-4809-88ce-66aa03182c8c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8tnf" Jul 10 16:54:17.385156 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.384637 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b9a53126-f278-4a9d-9735-5f73dd0c1964-ovnkube-config\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.385156 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.384649 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b9a53126-f278-4a9d-9735-5f73dd0c1964-host-cni-bin\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.385156 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.384684 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b9a53126-f278-4a9d-9735-5f73dd0c1964-env-overrides\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.385156 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.384704 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b9a53126-f278-4a9d-9735-5f73dd0c1964-host-cni-bin\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.385156 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.384710 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/08351a06-c5eb-4c51-923c-283f5b28a7d2-konnectivity-ca\") pod \"konnectivity-agent-t6rhv\" (UID: \"08351a06-c5eb-4c51-923c-283f5b28a7d2\") " pod="kube-system/konnectivity-agent-t6rhv" Jul 10 16:54:17.385156 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.384738 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1-host-var-lib-cni-multus\") pod \"multus-9qtfh\" (UID: \"a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1\") " pod="openshift-multus/multus-9qtfh" Jul 10 16:54:17.385156 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.384857 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b9a53126-f278-4a9d-9735-5f73dd0c1964-log-socket\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.385156 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.384967 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b9a53126-f278-4a9d-9735-5f73dd0c1964-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.385156 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.384993 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1d85dbfa-1b16-4e26-b207-6bb7064345ee-hosts-file\") pod \"node-resolver-6kk9g\" (UID: \"1d85dbfa-1b16-4e26-b207-6bb7064345ee\") " pod="openshift-dns/node-resolver-6kk9g" Jul 10 16:54:17.385156 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.385017 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cea5b1b2-0b38-41e0-ac8d-7e573be5ddb8-iptables-alerter-script\") pod \"iptables-alerter-sdvff\" (UID: \"cea5b1b2-0b38-41e0-ac8d-7e573be5ddb8\") " pod="openshift-network-operator/iptables-alerter-sdvff" Jul 10 16:54:17.386207 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.385037 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdq6w\" (UniqueName: \"kubernetes.io/projected/53987bf8-a205-4326-8757-ea8b3a6c4514-kube-api-access-bdq6w\") pod \"node-ca-w9kb4\" (UID: \"53987bf8-a205-4326-8757-ea8b3a6c4514\") " pod="openshift-image-registry/node-ca-w9kb4" Jul 10 16:54:17.386207 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.385059 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a0b23658-26fb-454a-a42e-88432bf12b0a-original-pull-secret\") pod \"global-pull-secret-syncer-dgk2w\" (UID: \"a0b23658-26fb-454a-a42e-88432bf12b0a\") " pod="kube-system/global-pull-secret-syncer-dgk2w" Jul 10 16:54:17.386207 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.385080 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1-hostroot\") pod \"multus-9qtfh\" (UID: \"a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1\") " pod="openshift-multus/multus-9qtfh" Jul 10 16:54:17.386207 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.385105 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b9a53126-f278-4a9d-9735-5f73dd0c1964-host-run-netns\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.386207 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.385156 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/dc38a8bd-42b1-4809-88ce-66aa03182c8c-sys-fs\") pod \"aws-ebs-csi-driver-node-l8tnf\" (UID: \"dc38a8bd-42b1-4809-88ce-66aa03182c8c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8tnf" Jul 10 16:54:17.386207 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.385180 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1-host-var-lib-cni-bin\") pod \"multus-9qtfh\" (UID: \"a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1\") " pod="openshift-multus/multus-9qtfh" Jul 10 16:54:17.386207 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.385185 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/08351a06-c5eb-4c51-923c-283f5b28a7d2-konnectivity-ca\") pod \"konnectivity-agent-t6rhv\" (UID: \"08351a06-c5eb-4c51-923c-283f5b28a7d2\") " pod="kube-system/konnectivity-agent-t6rhv" Jul 10 16:54:17.386207 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.385202 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1-multus-conf-dir\") pod \"multus-9qtfh\" (UID: \"a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1\") " pod="openshift-multus/multus-9qtfh" Jul 10 16:54:17.386207 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.385226 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5ggj\" (UniqueName: \"kubernetes.io/projected/a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1-kube-api-access-n5ggj\") pod \"multus-9qtfh\" (UID: \"a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1\") " pod="openshift-multus/multus-9qtfh" Jul 10 16:54:17.386207 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.385251 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b9a53126-f278-4a9d-9735-5f73dd0c1964-systemd-units\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.386207 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.385269 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b9a53126-f278-4a9d-9735-5f73dd0c1964-log-socket\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.386207 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.385274 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1-host-run-netns\") pod \"multus-9qtfh\" (UID: \"a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1\") " pod="openshift-multus/multus-9qtfh" Jul 10 16:54:17.386207 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.385313 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1-host-run-multus-certs\") pod \"multus-9qtfh\" (UID: \"a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1\") " pod="openshift-multus/multus-9qtfh" Jul 10 16:54:17.386207 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.385337 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b9a53126-f278-4a9d-9735-5f73dd0c1964-etc-openvswitch\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.386207 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.385350 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b9a53126-f278-4a9d-9735-5f73dd0c1964-host-run-netns\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.386207 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.385359 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b9a53126-f278-4a9d-9735-5f73dd0c1964-host-cni-netd\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.386207 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.385383 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b9a53126-f278-4a9d-9735-5f73dd0c1964-host-cni-netd\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.386974 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.385388 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a0b23658-26fb-454a-a42e-88432bf12b0a-dbus\") pod \"global-pull-secret-syncer-dgk2w\" (UID: \"a0b23658-26fb-454a-a42e-88432bf12b0a\") " pod="kube-system/global-pull-secret-syncer-dgk2w" Jul 10 16:54:17.386974 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.385434 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1d85dbfa-1b16-4e26-b207-6bb7064345ee-hosts-file\") pod \"node-resolver-6kk9g\" (UID: \"1d85dbfa-1b16-4e26-b207-6bb7064345ee\") " pod="openshift-dns/node-resolver-6kk9g" Jul 10 16:54:17.386974 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.385442 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b9a53126-f278-4a9d-9735-5f73dd0c1964-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.386974 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.385453 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfk5q\" (UniqueName: \"kubernetes.io/projected/16033c9b-cf7b-4afc-abe6-df0ded50ff80-kube-api-access-sfk5q\") pod \"network-check-target-jvkw6\" (UID: \"16033c9b-cf7b-4afc-abe6-df0ded50ff80\") " pod="openshift-network-diagnostics/network-check-target-jvkw6" Jul 10 16:54:17.386974 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.385478 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ed7de755-820a-4395-8b5f-ea6abfb86699-etc-sysconfig\") pod \"tuned-djhll\" (UID: \"ed7de755-820a-4395-8b5f-ea6abfb86699\") " pod="openshift-cluster-node-tuning-operator/tuned-djhll" Jul 10 16:54:17.386974 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.385499 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b9a53126-f278-4a9d-9735-5f73dd0c1964-run-systemd\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.386974 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.385506 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/dc38a8bd-42b1-4809-88ce-66aa03182c8c-sys-fs\") pod \"aws-ebs-csi-driver-node-l8tnf\" (UID: \"dc38a8bd-42b1-4809-88ce-66aa03182c8c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8tnf" Jul 10 16:54:17.386974 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.385523 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b9a53126-f278-4a9d-9735-5f73dd0c1964-host-run-ovn-kubernetes\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.386974 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.385541 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2h2db\" (UniqueName: \"kubernetes.io/projected/cea5b1b2-0b38-41e0-ac8d-7e573be5ddb8-kube-api-access-2h2db\") pod \"iptables-alerter-sdvff\" (UID: \"cea5b1b2-0b38-41e0-ac8d-7e573be5ddb8\") " pod="openshift-network-operator/iptables-alerter-sdvff" Jul 10 16:54:17.386974 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.385578 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b9a53126-f278-4a9d-9735-5f73dd0c1964-systemd-units\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.386974 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.385636 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b9a53126-f278-4a9d-9735-5f73dd0c1964-etc-openvswitch\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.386974 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.385692 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b9a53126-f278-4a9d-9735-5f73dd0c1964-run-systemd\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.386974 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.385729 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b9a53126-f278-4a9d-9735-5f73dd0c1964-host-run-ovn-kubernetes\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.386974 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.385757 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/dc38a8bd-42b1-4809-88ce-66aa03182c8c-socket-dir\") pod \"aws-ebs-csi-driver-node-l8tnf\" (UID: \"dc38a8bd-42b1-4809-88ce-66aa03182c8c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8tnf" Jul 10 16:54:17.386974 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.385801 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1-multus-daemon-config\") pod \"multus-9qtfh\" (UID: \"a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1\") " pod="openshift-multus/multus-9qtfh" Jul 10 16:54:17.386974 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.385839 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6e1806e2-a453-4929-8632-267349af86c7-os-release\") pod \"multus-additional-cni-plugins-5d4nh\" (UID: \"6e1806e2-a453-4929-8632-267349af86c7\") " pod="openshift-multus/multus-additional-cni-plugins-5d4nh" Jul 10 16:54:17.386974 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.385865 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ed7de755-820a-4395-8b5f-ea6abfb86699-lib-modules\") pod \"tuned-djhll\" (UID: \"ed7de755-820a-4395-8b5f-ea6abfb86699\") " pod="openshift-cluster-node-tuning-operator/tuned-djhll" Jul 10 16:54:17.387609 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.385891 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ed7de755-820a-4395-8b5f-ea6abfb86699-etc-tuned\") pod \"tuned-djhll\" (UID: \"ed7de755-820a-4395-8b5f-ea6abfb86699\") " pod="openshift-cluster-node-tuning-operator/tuned-djhll" Jul 10 16:54:17.387609 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.385916 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/08351a06-c5eb-4c51-923c-283f5b28a7d2-agent-certs\") pod \"konnectivity-agent-t6rhv\" (UID: \"08351a06-c5eb-4c51-923c-283f5b28a7d2\") " pod="kube-system/konnectivity-agent-t6rhv" Jul 10 16:54:17.387609 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.385930 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/dc38a8bd-42b1-4809-88ce-66aa03182c8c-socket-dir\") pod \"aws-ebs-csi-driver-node-l8tnf\" (UID: \"dc38a8bd-42b1-4809-88ce-66aa03182c8c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8tnf" Jul 10 16:54:17.387609 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.385939 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/dc38a8bd-42b1-4809-88ce-66aa03182c8c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-l8tnf\" (UID: \"dc38a8bd-42b1-4809-88ce-66aa03182c8c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8tnf" Jul 10 16:54:17.387609 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.385966 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1-multus-cni-dir\") pod \"multus-9qtfh\" (UID: \"a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1\") " pod="openshift-multus/multus-9qtfh" Jul 10 16:54:17.387609 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.385991 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1-host-var-lib-kubelet\") pod \"multus-9qtfh\" (UID: \"a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1\") " pod="openshift-multus/multus-9qtfh" Jul 10 16:54:17.387609 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.386018 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d43ce12f-a8fd-4a94-bb94-09e6562e1250-metrics-certs\") pod \"network-metrics-daemon-jr8jk\" (UID: \"d43ce12f-a8fd-4a94-bb94-09e6562e1250\") " pod="openshift-multus/network-metrics-daemon-jr8jk" Jul 10 16:54:17.387609 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.386043 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ed7de755-820a-4395-8b5f-ea6abfb86699-etc-sysctl-d\") pod \"tuned-djhll\" (UID: \"ed7de755-820a-4395-8b5f-ea6abfb86699\") " pod="openshift-cluster-node-tuning-operator/tuned-djhll" Jul 10 16:54:17.387609 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.386066 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ed7de755-820a-4395-8b5f-ea6abfb86699-tmp\") pod \"tuned-djhll\" (UID: \"ed7de755-820a-4395-8b5f-ea6abfb86699\") " pod="openshift-cluster-node-tuning-operator/tuned-djhll" Jul 10 16:54:17.387609 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.386080 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cea5b1b2-0b38-41e0-ac8d-7e573be5ddb8-iptables-alerter-script\") pod \"iptables-alerter-sdvff\" (UID: \"cea5b1b2-0b38-41e0-ac8d-7e573be5ddb8\") " pod="openshift-network-operator/iptables-alerter-sdvff" Jul 10 16:54:17.387609 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.386094 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b9a53126-f278-4a9d-9735-5f73dd0c1964-var-lib-openvswitch\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.387609 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.386154 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b9a53126-f278-4a9d-9735-5f73dd0c1964-run-ovn\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.387609 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.386174 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b9a53126-f278-4a9d-9735-5f73dd0c1964-var-lib-openvswitch\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.387609 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.386185 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gn762\" (UniqueName: \"kubernetes.io/projected/b9a53126-f278-4a9d-9735-5f73dd0c1964-kube-api-access-gn762\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.387609 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.386215 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqkhg\" (UniqueName: \"kubernetes.io/projected/dc38a8bd-42b1-4809-88ce-66aa03182c8c-kube-api-access-tqkhg\") pod \"aws-ebs-csi-driver-node-l8tnf\" (UID: \"dc38a8bd-42b1-4809-88ce-66aa03182c8c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8tnf" Jul 10 16:54:17.387609 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.386227 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/dc38a8bd-42b1-4809-88ce-66aa03182c8c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-l8tnf\" (UID: \"dc38a8bd-42b1-4809-88ce-66aa03182c8c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8tnf" Jul 10 16:54:17.387609 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.386241 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1-system-cni-dir\") pod \"multus-9qtfh\" (UID: \"a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1\") " pod="openshift-multus/multus-9qtfh" Jul 10 16:54:17.388252 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.386273 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1-multus-socket-dir-parent\") pod \"multus-9qtfh\" (UID: \"a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1\") " pod="openshift-multus/multus-9qtfh" Jul 10 16:54:17.388252 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.386318 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b9a53126-f278-4a9d-9735-5f73dd0c1964-run-ovn\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.388252 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.386446 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b9a53126-f278-4a9d-9735-5f73dd0c1964-node-log\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.388252 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.386472 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b9a53126-f278-4a9d-9735-5f73dd0c1964-ovnkube-script-lib\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.388252 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.386495 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d85dbfa-1b16-4e26-b207-6bb7064345ee-tmp-dir\") pod \"node-resolver-6kk9g\" (UID: \"1d85dbfa-1b16-4e26-b207-6bb7064345ee\") " pod="openshift-dns/node-resolver-6kk9g" Jul 10 16:54:17.388252 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.386524 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpbft\" (UniqueName: \"kubernetes.io/projected/d43ce12f-a8fd-4a94-bb94-09e6562e1250-kube-api-access-lpbft\") pod \"network-metrics-daemon-jr8jk\" (UID: \"d43ce12f-a8fd-4a94-bb94-09e6562e1250\") " pod="openshift-multus/network-metrics-daemon-jr8jk" Jul 10 16:54:17.388252 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.386548 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b9a53126-f278-4a9d-9735-5f73dd0c1964-node-log\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.388252 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.386555 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6e1806e2-a453-4929-8632-267349af86c7-system-cni-dir\") pod \"multus-additional-cni-plugins-5d4nh\" (UID: \"6e1806e2-a453-4929-8632-267349af86c7\") " pod="openshift-multus/multus-additional-cni-plugins-5d4nh" Jul 10 16:54:17.388252 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.386582 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhbzj\" (UniqueName: \"kubernetes.io/projected/6e1806e2-a453-4929-8632-267349af86c7-kube-api-access-xhbzj\") pod \"multus-additional-cni-plugins-5d4nh\" (UID: \"6e1806e2-a453-4929-8632-267349af86c7\") " pod="openshift-multus/multus-additional-cni-plugins-5d4nh" Jul 10 16:54:17.388252 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.386640 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ed7de755-820a-4395-8b5f-ea6abfb86699-etc-sysctl-conf\") pod \"tuned-djhll\" (UID: \"ed7de755-820a-4395-8b5f-ea6abfb86699\") " pod="openshift-cluster-node-tuning-operator/tuned-djhll" Jul 10 16:54:17.388252 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.386663 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ed7de755-820a-4395-8b5f-ea6abfb86699-host\") pod \"tuned-djhll\" (UID: \"ed7de755-820a-4395-8b5f-ea6abfb86699\") " pod="openshift-cluster-node-tuning-operator/tuned-djhll" Jul 10 16:54:17.388252 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.386686 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b9a53126-f278-4a9d-9735-5f73dd0c1964-env-overrides\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.388252 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.386860 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d85dbfa-1b16-4e26-b207-6bb7064345ee-tmp-dir\") pod \"node-resolver-6kk9g\" (UID: \"1d85dbfa-1b16-4e26-b207-6bb7064345ee\") " pod="openshift-dns/node-resolver-6kk9g" Jul 10 16:54:17.388252 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.387101 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b9a53126-f278-4a9d-9735-5f73dd0c1964-ovnkube-script-lib\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.388252 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.387809 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b9a53126-f278-4a9d-9735-5f73dd0c1964-ovn-node-metrics-cert\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.388844 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.388833 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/08351a06-c5eb-4c51-923c-283f5b28a7d2-agent-certs\") pod \"konnectivity-agent-t6rhv\" (UID: \"08351a06-c5eb-4c51-923c-283f5b28a7d2\") " pod="kube-system/konnectivity-agent-t6rhv" Jul 10 16:54:17.401681 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.401659 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-6hmpj" Jul 10 16:54:17.406644 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.406614 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gn762\" (UniqueName: \"kubernetes.io/projected/b9a53126-f278-4a9d-9735-5f73dd0c1964-kube-api-access-gn762\") pod \"ovnkube-node-gqgkg\" (UID: \"b9a53126-f278-4a9d-9735-5f73dd0c1964\") " pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.407188 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.407156 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2h2db\" (UniqueName: \"kubernetes.io/projected/cea5b1b2-0b38-41e0-ac8d-7e573be5ddb8-kube-api-access-2h2db\") pod \"iptables-alerter-sdvff\" (UID: \"cea5b1b2-0b38-41e0-ac8d-7e573be5ddb8\") " pod="openshift-network-operator/iptables-alerter-sdvff" Jul 10 16:54:17.407675 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.407653 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tt6z5\" (UniqueName: \"kubernetes.io/projected/1d85dbfa-1b16-4e26-b207-6bb7064345ee-kube-api-access-tt6z5\") pod \"node-resolver-6kk9g\" (UID: \"1d85dbfa-1b16-4e26-b207-6bb7064345ee\") " pod="openshift-dns/node-resolver-6kk9g" Jul 10 16:54:17.408381 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.408363 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqkhg\" (UniqueName: \"kubernetes.io/projected/dc38a8bd-42b1-4809-88ce-66aa03182c8c-kube-api-access-tqkhg\") pod \"aws-ebs-csi-driver-node-l8tnf\" (UID: \"dc38a8bd-42b1-4809-88ce-66aa03182c8c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8tnf" Jul 10 16:54:17.421090 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.421067 2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 16:54:17.487002 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.486950 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1-etc-kubernetes\") pod \"multus-9qtfh\" (UID: \"a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1\") " pod="openshift-multus/multus-9qtfh" Jul 10 16:54:17.487188 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.487025 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1-etc-kubernetes\") pod \"multus-9qtfh\" (UID: \"a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1\") " pod="openshift-multus/multus-9qtfh" Jul 10 16:54:17.487451 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.487403 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6e1806e2-a453-4929-8632-267349af86c7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5d4nh\" (UID: \"6e1806e2-a453-4929-8632-267349af86c7\") " pod="openshift-multus/multus-additional-cni-plugins-5d4nh" Jul 10 16:54:17.487508 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.487492 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6e1806e2-a453-4929-8632-267349af86c7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5d4nh\" (UID: \"6e1806e2-a453-4929-8632-267349af86c7\") " pod="openshift-multus/multus-additional-cni-plugins-5d4nh" Jul 10 16:54:17.487546 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.487520 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/53987bf8-a205-4326-8757-ea8b3a6c4514-serviceca\") pod \"node-ca-w9kb4\" (UID: \"53987bf8-a205-4326-8757-ea8b3a6c4514\") " pod="openshift-image-registry/node-ca-w9kb4" Jul 10 16:54:17.487590 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.487540 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1-cnibin\") pod \"multus-9qtfh\" (UID: \"a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1\") " pod="openshift-multus/multus-9qtfh" Jul 10 16:54:17.487590 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.487562 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1-os-release\") pod \"multus-9qtfh\" (UID: \"a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1\") " pod="openshift-multus/multus-9qtfh" Jul 10 16:54:17.487590 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.487581 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6e1806e2-a453-4929-8632-267349af86c7-cni-binary-copy\") pod \"multus-additional-cni-plugins-5d4nh\" (UID: \"6e1806e2-a453-4929-8632-267349af86c7\") " pod="openshift-multus/multus-additional-cni-plugins-5d4nh" Jul 10 16:54:17.487755 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.487608 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6e1806e2-a453-4929-8632-267349af86c7-cnibin\") pod \"multus-additional-cni-plugins-5d4nh\" (UID: \"6e1806e2-a453-4929-8632-267349af86c7\") " pod="openshift-multus/multus-additional-cni-plugins-5d4nh" Jul 10 16:54:17.487755 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.487634 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ed7de755-820a-4395-8b5f-ea6abfb86699-etc-systemd\") pod \"tuned-djhll\" (UID: \"ed7de755-820a-4395-8b5f-ea6abfb86699\") " pod="openshift-cluster-node-tuning-operator/tuned-djhll" Jul 10 16:54:17.487755 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.487659 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ed7de755-820a-4395-8b5f-ea6abfb86699-etc-kubernetes\") pod \"tuned-djhll\" (UID: \"ed7de755-820a-4395-8b5f-ea6abfb86699\") " pod="openshift-cluster-node-tuning-operator/tuned-djhll" Jul 10 16:54:17.487755 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.487684 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ed7de755-820a-4395-8b5f-ea6abfb86699-run\") pod \"tuned-djhll\" (UID: \"ed7de755-820a-4395-8b5f-ea6abfb86699\") " pod="openshift-cluster-node-tuning-operator/tuned-djhll" Jul 10 16:54:17.487755 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.487700 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1-os-release\") pod \"multus-9qtfh\" (UID: \"a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1\") " pod="openshift-multus/multus-9qtfh" Jul 10 16:54:17.487755 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.487707 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ed7de755-820a-4395-8b5f-ea6abfb86699-sys\") pod \"tuned-djhll\" (UID: \"ed7de755-820a-4395-8b5f-ea6abfb86699\") " pod="openshift-cluster-node-tuning-operator/tuned-djhll" Jul 10 16:54:17.487755 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.487709 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1-cnibin\") pod \"multus-9qtfh\" (UID: \"a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1\") " pod="openshift-multus/multus-9qtfh" Jul 10 16:54:17.487755 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.487751 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1-host-var-lib-cni-multus\") pod \"multus-9qtfh\" (UID: \"a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1\") " pod="openshift-multus/multus-9qtfh" Jul 10 16:54:17.488198 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.487783 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ed7de755-820a-4395-8b5f-ea6abfb86699-etc-systemd\") pod \"tuned-djhll\" (UID: \"ed7de755-820a-4395-8b5f-ea6abfb86699\") " pod="openshift-cluster-node-tuning-operator/tuned-djhll" Jul 10 16:54:17.488198 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.487789 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdq6w\" (UniqueName: \"kubernetes.io/projected/53987bf8-a205-4326-8757-ea8b3a6c4514-kube-api-access-bdq6w\") pod \"node-ca-w9kb4\" (UID: \"53987bf8-a205-4326-8757-ea8b3a6c4514\") " pod="openshift-image-registry/node-ca-w9kb4" Jul 10 16:54:17.488198 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.487795 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ed7de755-820a-4395-8b5f-ea6abfb86699-etc-kubernetes\") pod \"tuned-djhll\" (UID: \"ed7de755-820a-4395-8b5f-ea6abfb86699\") " pod="openshift-cluster-node-tuning-operator/tuned-djhll" Jul 10 16:54:17.488198 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.487756 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ed7de755-820a-4395-8b5f-ea6abfb86699-sys\") pod \"tuned-djhll\" (UID: \"ed7de755-820a-4395-8b5f-ea6abfb86699\") " pod="openshift-cluster-node-tuning-operator/tuned-djhll" Jul 10 16:54:17.488198 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.487828 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a0b23658-26fb-454a-a42e-88432bf12b0a-original-pull-secret\") pod \"global-pull-secret-syncer-dgk2w\" (UID: \"a0b23658-26fb-454a-a42e-88432bf12b0a\") " pod="kube-system/global-pull-secret-syncer-dgk2w" Jul 10 16:54:17.488198 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.487913 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6e1806e2-a453-4929-8632-267349af86c7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5d4nh\" (UID: \"6e1806e2-a453-4929-8632-267349af86c7\") " pod="openshift-multus/multus-additional-cni-plugins-5d4nh" Jul 10 16:54:17.488198 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.487956 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1-hostroot\") pod \"multus-9qtfh\" (UID: \"a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1\") " pod="openshift-multus/multus-9qtfh" Jul 10 16:54:17.488198 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.487963 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ed7de755-820a-4395-8b5f-ea6abfb86699-run\") pod \"tuned-djhll\" (UID: \"ed7de755-820a-4395-8b5f-ea6abfb86699\") " pod="openshift-cluster-node-tuning-operator/tuned-djhll" Jul 10 16:54:17.488198 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.487972 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6e1806e2-a453-4929-8632-267349af86c7-cnibin\") pod \"multus-additional-cni-plugins-5d4nh\" (UID: \"6e1806e2-a453-4929-8632-267349af86c7\") " pod="openshift-multus/multus-additional-cni-plugins-5d4nh" Jul 10 16:54:17.488198 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.488013 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1-host-var-lib-cni-multus\") pod \"multus-9qtfh\" (UID: \"a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1\") " pod="openshift-multus/multus-9qtfh" Jul 10 16:54:17.488198 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.488016 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1-hostroot\") pod \"multus-9qtfh\" (UID: \"a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1\") " pod="openshift-multus/multus-9qtfh" Jul 10 16:54:17.488198 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:17.488017 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 16:54:17.488198 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.488054 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1-host-var-lib-cni-bin\") pod \"multus-9qtfh\" (UID: \"a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1\") " pod="openshift-multus/multus-9qtfh" Jul 10 16:54:17.488198 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.488099 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1-multus-conf-dir\") pod \"multus-9qtfh\" (UID: \"a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1\") " pod="openshift-multus/multus-9qtfh" Jul 10 16:54:17.488198 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.488019 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/53987bf8-a205-4326-8757-ea8b3a6c4514-serviceca\") pod \"node-ca-w9kb4\" (UID: \"53987bf8-a205-4326-8757-ea8b3a6c4514\") " pod="openshift-image-registry/node-ca-w9kb4" Jul 10 16:54:17.488198 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.488126 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5ggj\" (UniqueName: \"kubernetes.io/projected/a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1-kube-api-access-n5ggj\") pod \"multus-9qtfh\" (UID: \"a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1\") " pod="openshift-multus/multus-9qtfh" Jul 10 16:54:17.488198 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.488128 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1-host-var-lib-cni-bin\") pod \"multus-9qtfh\" (UID: \"a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1\") " pod="openshift-multus/multus-9qtfh" Jul 10 16:54:17.488198 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.488135 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1-multus-conf-dir\") pod \"multus-9qtfh\" (UID: \"a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1\") " pod="openshift-multus/multus-9qtfh" Jul 10 16:54:17.488198 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.488154 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1-host-run-netns\") pod \"multus-9qtfh\" (UID: \"a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1\") " pod="openshift-multus/multus-9qtfh" Jul 10 16:54:17.489186 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:17.488173 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0b23658-26fb-454a-a42e-88432bf12b0a-original-pull-secret podName:a0b23658-26fb-454a-a42e-88432bf12b0a nodeName:}" failed. No retries permitted until 2026-07-10 16:54:17.988151916 +0000 UTC m=+2.077480947 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a0b23658-26fb-454a-a42e-88432bf12b0a-original-pull-secret") pod "global-pull-secret-syncer-dgk2w" (UID: "a0b23658-26fb-454a-a42e-88432bf12b0a") : object "kube-system"/"original-pull-secret" not registered Jul 10 16:54:17.489186 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.488192 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1-host-run-netns\") pod \"multus-9qtfh\" (UID: \"a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1\") " pod="openshift-multus/multus-9qtfh" Jul 10 16:54:17.489186 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.488199 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1-host-run-multus-certs\") pod \"multus-9qtfh\" (UID: \"a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1\") " pod="openshift-multus/multus-9qtfh" Jul 10 16:54:17.489186 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.488221 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a0b23658-26fb-454a-a42e-88432bf12b0a-dbus\") pod \"global-pull-secret-syncer-dgk2w\" (UID: \"a0b23658-26fb-454a-a42e-88432bf12b0a\") " pod="kube-system/global-pull-secret-syncer-dgk2w" Jul 10 16:54:17.489186 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.488250 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfk5q\" (UniqueName: \"kubernetes.io/projected/16033c9b-cf7b-4afc-abe6-df0ded50ff80-kube-api-access-sfk5q\") pod \"network-check-target-jvkw6\" (UID: \"16033c9b-cf7b-4afc-abe6-df0ded50ff80\") " pod="openshift-network-diagnostics/network-check-target-jvkw6" Jul 10 16:54:17.489186 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.488261 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1-host-run-multus-certs\") pod \"multus-9qtfh\" (UID: \"a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1\") " pod="openshift-multus/multus-9qtfh" Jul 10 16:54:17.489186 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.488275 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ed7de755-820a-4395-8b5f-ea6abfb86699-etc-sysconfig\") pod \"tuned-djhll\" (UID: \"ed7de755-820a-4395-8b5f-ea6abfb86699\") " pod="openshift-cluster-node-tuning-operator/tuned-djhll" Jul 10 16:54:17.489186 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.488303 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1-multus-daemon-config\") pod \"multus-9qtfh\" (UID: \"a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1\") " pod="openshift-multus/multus-9qtfh" Jul 10 16:54:17.489186 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.488327 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6e1806e2-a453-4929-8632-267349af86c7-os-release\") pod \"multus-additional-cni-plugins-5d4nh\" (UID: \"6e1806e2-a453-4929-8632-267349af86c7\") " pod="openshift-multus/multus-additional-cni-plugins-5d4nh" Jul 10 16:54:17.489186 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.488350 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ed7de755-820a-4395-8b5f-ea6abfb86699-lib-modules\") pod \"tuned-djhll\" (UID: \"ed7de755-820a-4395-8b5f-ea6abfb86699\") " pod="openshift-cluster-node-tuning-operator/tuned-djhll" Jul 10 16:54:17.489186 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.488374 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a0b23658-26fb-454a-a42e-88432bf12b0a-dbus\") pod \"global-pull-secret-syncer-dgk2w\" (UID: \"a0b23658-26fb-454a-a42e-88432bf12b0a\") " pod="kube-system/global-pull-secret-syncer-dgk2w" Jul 10 16:54:17.489186 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.488395 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ed7de755-820a-4395-8b5f-ea6abfb86699-etc-sysconfig\") pod \"tuned-djhll\" (UID: \"ed7de755-820a-4395-8b5f-ea6abfb86699\") " pod="openshift-cluster-node-tuning-operator/tuned-djhll" Jul 10 16:54:17.489186 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.488398 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ed7de755-820a-4395-8b5f-ea6abfb86699-etc-tuned\") pod \"tuned-djhll\" (UID: \"ed7de755-820a-4395-8b5f-ea6abfb86699\") " pod="openshift-cluster-node-tuning-operator/tuned-djhll" Jul 10 16:54:17.489186 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.488441 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1-multus-cni-dir\") pod \"multus-9qtfh\" (UID: \"a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1\") " pod="openshift-multus/multus-9qtfh" Jul 10 16:54:17.489186 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.488441 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6e1806e2-a453-4929-8632-267349af86c7-os-release\") pod \"multus-additional-cni-plugins-5d4nh\" (UID: \"6e1806e2-a453-4929-8632-267349af86c7\") " pod="openshift-multus/multus-additional-cni-plugins-5d4nh" Jul 10 16:54:17.489186 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.488468 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1-host-var-lib-kubelet\") pod \"multus-9qtfh\" (UID: \"a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1\") " pod="openshift-multus/multus-9qtfh" Jul 10 16:54:17.489186 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.488477 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ed7de755-820a-4395-8b5f-ea6abfb86699-lib-modules\") pod \"tuned-djhll\" (UID: \"ed7de755-820a-4395-8b5f-ea6abfb86699\") " pod="openshift-cluster-node-tuning-operator/tuned-djhll" Jul 10 16:54:17.489817 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.488488 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1-multus-cni-dir\") pod \"multus-9qtfh\" (UID: \"a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1\") " pod="openshift-multus/multus-9qtfh" Jul 10 16:54:17.489817 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.488495 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d43ce12f-a8fd-4a94-bb94-09e6562e1250-metrics-certs\") pod \"network-metrics-daemon-jr8jk\" (UID: \"d43ce12f-a8fd-4a94-bb94-09e6562e1250\") " pod="openshift-multus/network-metrics-daemon-jr8jk" Jul 10 16:54:17.489817 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.488528 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1-host-var-lib-kubelet\") pod \"multus-9qtfh\" (UID: \"a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1\") " pod="openshift-multus/multus-9qtfh" Jul 10 16:54:17.489817 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.488527 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ed7de755-820a-4395-8b5f-ea6abfb86699-etc-sysctl-d\") pod \"tuned-djhll\" (UID: \"ed7de755-820a-4395-8b5f-ea6abfb86699\") " pod="openshift-cluster-node-tuning-operator/tuned-djhll" Jul 10 16:54:17.489817 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:17.488564 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 16:54:17.489817 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.488569 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ed7de755-820a-4395-8b5f-ea6abfb86699-tmp\") pod \"tuned-djhll\" (UID: \"ed7de755-820a-4395-8b5f-ea6abfb86699\") " pod="openshift-cluster-node-tuning-operator/tuned-djhll" Jul 10 16:54:17.489817 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.488602 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1-system-cni-dir\") pod \"multus-9qtfh\" (UID: \"a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1\") " pod="openshift-multus/multus-9qtfh" Jul 10 16:54:17.489817 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:17.488623 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d43ce12f-a8fd-4a94-bb94-09e6562e1250-metrics-certs podName:d43ce12f-a8fd-4a94-bb94-09e6562e1250 nodeName:}" failed. No retries permitted until 2026-07-10 16:54:17.988605377 +0000 UTC m=+2.077934411 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d43ce12f-a8fd-4a94-bb94-09e6562e1250-metrics-certs") pod "network-metrics-daemon-jr8jk" (UID: "d43ce12f-a8fd-4a94-bb94-09e6562e1250") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 16:54:17.489817 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.488631 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ed7de755-820a-4395-8b5f-ea6abfb86699-etc-sysctl-d\") pod \"tuned-djhll\" (UID: \"ed7de755-820a-4395-8b5f-ea6abfb86699\") " pod="openshift-cluster-node-tuning-operator/tuned-djhll" Jul 10 16:54:17.489817 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.488637 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1-system-cni-dir\") pod \"multus-9qtfh\" (UID: \"a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1\") " pod="openshift-multus/multus-9qtfh" Jul 10 16:54:17.489817 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.488654 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1-multus-socket-dir-parent\") pod \"multus-9qtfh\" (UID: \"a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1\") " pod="openshift-multus/multus-9qtfh" Jul 10 16:54:17.489817 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.488689 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpbft\" (UniqueName: \"kubernetes.io/projected/d43ce12f-a8fd-4a94-bb94-09e6562e1250-kube-api-access-lpbft\") pod \"network-metrics-daemon-jr8jk\" (UID: \"d43ce12f-a8fd-4a94-bb94-09e6562e1250\") " pod="openshift-multus/network-metrics-daemon-jr8jk" Jul 10 16:54:17.489817 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.488698 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1-multus-socket-dir-parent\") pod \"multus-9qtfh\" (UID: \"a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1\") " pod="openshift-multus/multus-9qtfh" Jul 10 16:54:17.489817 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.488714 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6e1806e2-a453-4929-8632-267349af86c7-system-cni-dir\") pod \"multus-additional-cni-plugins-5d4nh\" (UID: \"6e1806e2-a453-4929-8632-267349af86c7\") " pod="openshift-multus/multus-additional-cni-plugins-5d4nh" Jul 10 16:54:17.489817 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.488739 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhbzj\" (UniqueName: \"kubernetes.io/projected/6e1806e2-a453-4929-8632-267349af86c7-kube-api-access-xhbzj\") pod \"multus-additional-cni-plugins-5d4nh\" (UID: \"6e1806e2-a453-4929-8632-267349af86c7\") " pod="openshift-multus/multus-additional-cni-plugins-5d4nh" Jul 10 16:54:17.489817 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.488835 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ed7de755-820a-4395-8b5f-ea6abfb86699-etc-sysctl-conf\") pod \"tuned-djhll\" (UID: \"ed7de755-820a-4395-8b5f-ea6abfb86699\") " pod="openshift-cluster-node-tuning-operator/tuned-djhll" Jul 10 16:54:17.489817 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.488863 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ed7de755-820a-4395-8b5f-ea6abfb86699-host\") pod \"tuned-djhll\" (UID: \"ed7de755-820a-4395-8b5f-ea6abfb86699\") " pod="openshift-cluster-node-tuning-operator/tuned-djhll" Jul 10 16:54:17.490490 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.488888 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/53987bf8-a205-4326-8757-ea8b3a6c4514-host\") pod \"node-ca-w9kb4\" (UID: \"53987bf8-a205-4326-8757-ea8b3a6c4514\") " pod="openshift-image-registry/node-ca-w9kb4" Jul 10 16:54:17.490490 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.488893 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1-multus-daemon-config\") pod \"multus-9qtfh\" (UID: \"a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1\") " pod="openshift-multus/multus-9qtfh" Jul 10 16:54:17.490490 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.488915 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a0b23658-26fb-454a-a42e-88432bf12b0a-kubelet-config\") pod \"global-pull-secret-syncer-dgk2w\" (UID: \"a0b23658-26fb-454a-a42e-88432bf12b0a\") " pod="kube-system/global-pull-secret-syncer-dgk2w" Jul 10 16:54:17.490490 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.488977 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a0b23658-26fb-454a-a42e-88432bf12b0a-kubelet-config\") pod \"global-pull-secret-syncer-dgk2w\" (UID: \"a0b23658-26fb-454a-a42e-88432bf12b0a\") " pod="kube-system/global-pull-secret-syncer-dgk2w" Jul 10 16:54:17.490490 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.488997 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6e1806e2-a453-4929-8632-267349af86c7-cni-binary-copy\") pod \"multus-additional-cni-plugins-5d4nh\" (UID: \"6e1806e2-a453-4929-8632-267349af86c7\") " pod="openshift-multus/multus-additional-cni-plugins-5d4nh" Jul 10 16:54:17.490490 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.489006 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6e1806e2-a453-4929-8632-267349af86c7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5d4nh\" (UID: \"6e1806e2-a453-4929-8632-267349af86c7\") " pod="openshift-multus/multus-additional-cni-plugins-5d4nh" Jul 10 16:54:17.490490 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.489036 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ed7de755-820a-4395-8b5f-ea6abfb86699-host\") pod \"tuned-djhll\" (UID: \"ed7de755-820a-4395-8b5f-ea6abfb86699\") " pod="openshift-cluster-node-tuning-operator/tuned-djhll" Jul 10 16:54:17.490490 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.489046 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ed7de755-820a-4395-8b5f-ea6abfb86699-etc-sysctl-conf\") pod \"tuned-djhll\" (UID: \"ed7de755-820a-4395-8b5f-ea6abfb86699\") " pod="openshift-cluster-node-tuning-operator/tuned-djhll" Jul 10 16:54:17.490490 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.489060 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/53987bf8-a205-4326-8757-ea8b3a6c4514-host\") pod \"node-ca-w9kb4\" (UID: \"53987bf8-a205-4326-8757-ea8b3a6c4514\") " pod="openshift-image-registry/node-ca-w9kb4" Jul 10 16:54:17.490490 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.489071 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6e1806e2-a453-4929-8632-267349af86c7-system-cni-dir\") pod \"multus-additional-cni-plugins-5d4nh\" (UID: \"6e1806e2-a453-4929-8632-267349af86c7\") " pod="openshift-multus/multus-additional-cni-plugins-5d4nh" Jul 10 16:54:17.490490 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.489077 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1-cni-binary-copy\") pod \"multus-9qtfh\" (UID: \"a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1\") " pod="openshift-multus/multus-9qtfh" Jul 10 16:54:17.490490 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.489216 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1-host-run-k8s-cni-cncf-io\") pod \"multus-9qtfh\" (UID: \"a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1\") " pod="openshift-multus/multus-9qtfh" Jul 10 16:54:17.490490 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.489259 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ed7de755-820a-4395-8b5f-ea6abfb86699-var-lib-kubelet\") pod \"tuned-djhll\" (UID: \"ed7de755-820a-4395-8b5f-ea6abfb86699\") " pod="openshift-cluster-node-tuning-operator/tuned-djhll" Jul 10 16:54:17.490490 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.489289 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flfl2\" (UniqueName: \"kubernetes.io/projected/ed7de755-820a-4395-8b5f-ea6abfb86699-kube-api-access-flfl2\") pod \"tuned-djhll\" (UID: \"ed7de755-820a-4395-8b5f-ea6abfb86699\") " pod="openshift-cluster-node-tuning-operator/tuned-djhll" Jul 10 16:54:17.490490 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.489333 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6e1806e2-a453-4929-8632-267349af86c7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5d4nh\" (UID: \"6e1806e2-a453-4929-8632-267349af86c7\") " pod="openshift-multus/multus-additional-cni-plugins-5d4nh" Jul 10 16:54:17.490490 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.489350 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ed7de755-820a-4395-8b5f-ea6abfb86699-var-lib-kubelet\") pod \"tuned-djhll\" (UID: \"ed7de755-820a-4395-8b5f-ea6abfb86699\") " pod="openshift-cluster-node-tuning-operator/tuned-djhll" Jul 10 16:54:17.490490 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.489360 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ed7de755-820a-4395-8b5f-ea6abfb86699-etc-modprobe-d\") pod \"tuned-djhll\" (UID: \"ed7de755-820a-4395-8b5f-ea6abfb86699\") " pod="openshift-cluster-node-tuning-operator/tuned-djhll" Jul 10 16:54:17.491079 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.489438 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1-host-run-k8s-cni-cncf-io\") pod \"multus-9qtfh\" (UID: \"a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1\") " pod="openshift-multus/multus-9qtfh" Jul 10 16:54:17.491079 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.489452 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ed7de755-820a-4395-8b5f-ea6abfb86699-etc-modprobe-d\") pod \"tuned-djhll\" (UID: \"ed7de755-820a-4395-8b5f-ea6abfb86699\") " pod="openshift-cluster-node-tuning-operator/tuned-djhll" Jul 10 16:54:17.491079 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.489554 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6e1806e2-a453-4929-8632-267349af86c7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5d4nh\" (UID: \"6e1806e2-a453-4929-8632-267349af86c7\") " pod="openshift-multus/multus-additional-cni-plugins-5d4nh" Jul 10 16:54:17.491079 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.489868 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1-cni-binary-copy\") pod \"multus-9qtfh\" (UID: \"a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1\") " pod="openshift-multus/multus-9qtfh" Jul 10 16:54:17.491079 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.491042 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ed7de755-820a-4395-8b5f-ea6abfb86699-etc-tuned\") pod \"tuned-djhll\" (UID: \"ed7de755-820a-4395-8b5f-ea6abfb86699\") " pod="openshift-cluster-node-tuning-operator/tuned-djhll" Jul 10 16:54:17.491262 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.491064 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ed7de755-820a-4395-8b5f-ea6abfb86699-tmp\") pod \"tuned-djhll\" (UID: \"ed7de755-820a-4395-8b5f-ea6abfb86699\") " pod="openshift-cluster-node-tuning-operator/tuned-djhll" Jul 10 16:54:17.501782 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.501749 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdq6w\" (UniqueName: \"kubernetes.io/projected/53987bf8-a205-4326-8757-ea8b3a6c4514-kube-api-access-bdq6w\") pod \"node-ca-w9kb4\" (UID: \"53987bf8-a205-4326-8757-ea8b3a6c4514\") " pod="openshift-image-registry/node-ca-w9kb4" Jul 10 16:54:17.503399 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:17.503368 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 16:54:17.503399 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:17.503398 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 16:54:17.503581 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:17.503412 2578 projected.go:196] Error preparing data for projected volume kube-api-access-sfk5q for pod openshift-network-diagnostics/network-check-target-jvkw6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 16:54:17.503581 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:17.503496 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/16033c9b-cf7b-4afc-abe6-df0ded50ff80-kube-api-access-sfk5q podName:16033c9b-cf7b-4afc-abe6-df0ded50ff80 nodeName:}" failed. No retries permitted until 2026-07-10 16:54:18.003477936 +0000 UTC m=+2.092806981 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-sfk5q" (UniqueName: "kubernetes.io/projected/16033c9b-cf7b-4afc-abe6-df0ded50ff80-kube-api-access-sfk5q") pod "network-check-target-jvkw6" (UID: "16033c9b-cf7b-4afc-abe6-df0ded50ff80") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 16:54:17.505200 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.505178 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5ggj\" (UniqueName: \"kubernetes.io/projected/a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1-kube-api-access-n5ggj\") pod \"multus-9qtfh\" (UID: \"a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1\") " pod="openshift-multus/multus-9qtfh" Jul 10 16:54:17.506233 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.506198 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpbft\" (UniqueName: \"kubernetes.io/projected/d43ce12f-a8fd-4a94-bb94-09e6562e1250-kube-api-access-lpbft\") pod \"network-metrics-daemon-jr8jk\" (UID: \"d43ce12f-a8fd-4a94-bb94-09e6562e1250\") " pod="openshift-multus/network-metrics-daemon-jr8jk" Jul 10 16:54:17.506905 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.506811 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhbzj\" (UniqueName: \"kubernetes.io/projected/6e1806e2-a453-4929-8632-267349af86c7-kube-api-access-xhbzj\") pod \"multus-additional-cni-plugins-5d4nh\" (UID: \"6e1806e2-a453-4929-8632-267349af86c7\") " pod="openshift-multus/multus-additional-cni-plugins-5d4nh" Jul 10 16:54:17.507287 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.507270 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-flfl2\" (UniqueName: \"kubernetes.io/projected/ed7de755-820a-4395-8b5f-ea6abfb86699-kube-api-access-flfl2\") pod \"tuned-djhll\" (UID: \"ed7de755-820a-4395-8b5f-ea6abfb86699\") " pod="openshift-cluster-node-tuning-operator/tuned-djhll" Jul 10 16:54:17.518149 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:17.518120 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac3f323c6b38ca654d9468732cf2193c.slice/crio-6a196354071095922fa71c2d2cf03dc4c2d6f1f7517fa8733210c0371fe66a27 WatchSource:0}: Error finding container 6a196354071095922fa71c2d2cf03dc4c2d6f1f7517fa8733210c0371fe66a27: Status 404 returned error can't find the container with id 6a196354071095922fa71c2d2cf03dc4c2d6f1f7517fa8733210c0371fe66a27 Jul 10 16:54:17.518622 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:17.518606 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod374e81600297ab5c614c91e79771b865.slice/crio-5195f207d37a32b453f671fb061b642f567552930800ae21cf098d7c21ea762f WatchSource:0}: Error finding container 5195f207d37a32b453f671fb061b642f567552930800ae21cf098d7c21ea762f: Status 404 returned error can't find the container with id 5195f207d37a32b453f671fb061b642f567552930800ae21cf098d7c21ea762f Jul 10 16:54:17.522464 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.522446 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 16:54:17.607738 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.607706 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:17.616540 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:17.616479 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9a53126_f278_4a9d_9735_5f73dd0c1964.slice/crio-81fdbf1c219472452ca21a6cb8344d1ba5876f173df1e1708a63de92aaecb21e WatchSource:0}: Error finding container 81fdbf1c219472452ca21a6cb8344d1ba5876f173df1e1708a63de92aaecb21e: Status 404 returned error can't find the container with id 81fdbf1c219472452ca21a6cb8344d1ba5876f173df1e1708a63de92aaecb21e Jul 10 16:54:17.623075 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.623041 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6kk9g" Jul 10 16:54:17.630748 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:17.630725 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d85dbfa_1b16_4e26_b207_6bb7064345ee.slice/crio-a5d0a2dcb292125f2b47a89e0f3780f8606e222a4adeb02d5e65bea7a697ace8 WatchSource:0}: Error finding container a5d0a2dcb292125f2b47a89e0f3780f8606e222a4adeb02d5e65bea7a697ace8: Status 404 returned error can't find the container with id a5d0a2dcb292125f2b47a89e0f3780f8606e222a4adeb02d5e65bea7a697ace8 Jul 10 16:54:17.630838 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.630750 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-sdvff" Jul 10 16:54:17.638410 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:17.638386 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcea5b1b2_0b38_41e0_ac8d_7e573be5ddb8.slice/crio-70bac432a054ed6fe06784fb112b1531a34bbc782cabde9d7e373ea50fd9b216 WatchSource:0}: Error finding container 70bac432a054ed6fe06784fb112b1531a34bbc782cabde9d7e373ea50fd9b216: Status 404 returned error can't find the container with id 70bac432a054ed6fe06784fb112b1531a34bbc782cabde9d7e373ea50fd9b216 Jul 10 16:54:17.664843 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.664810 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-t6rhv" Jul 10 16:54:17.674845 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:17.674819 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08351a06_c5eb_4c51_923c_283f5b28a7d2.slice/crio-35140ed47d3d8eaeeb1bfd3669bc886912501af187eb6e4a08ce61f5211f558b WatchSource:0}: Error finding container 35140ed47d3d8eaeeb1bfd3669bc886912501af187eb6e4a08ce61f5211f558b: Status 404 returned error can't find the container with id 35140ed47d3d8eaeeb1bfd3669bc886912501af187eb6e4a08ce61f5211f558b Jul 10 16:54:17.683534 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.683513 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8tnf" Jul 10 16:54:17.691947 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:17.691922 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc38a8bd_42b1_4809_88ce_66aa03182c8c.slice/crio-369e2fa44042b10b18efce84707dea559d98852ef1e65f5d0a31f656b85cfd76 WatchSource:0}: Error finding container 369e2fa44042b10b18efce84707dea559d98852ef1e65f5d0a31f656b85cfd76: Status 404 returned error can't find the container with id 369e2fa44042b10b18efce84707dea559d98852ef1e65f5d0a31f656b85cfd76 Jul 10 16:54:17.702290 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.702268 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9qtfh" Jul 10 16:54:17.711101 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:17.711072 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda26f297b_fb0e_4bd2_8cab_e3ec3aff00c1.slice/crio-3ab3239553bc919496bb11d4adde496d46252fdef063848ae9fe988ca6f19982 WatchSource:0}: Error finding container 3ab3239553bc919496bb11d4adde496d46252fdef063848ae9fe988ca6f19982: Status 404 returned error can't find the container with id 3ab3239553bc919496bb11d4adde496d46252fdef063848ae9fe988ca6f19982 Jul 10 16:54:17.721776 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.721754 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5d4nh" Jul 10 16:54:17.729033 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:17.729006 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e1806e2_a453_4929_8632_267349af86c7.slice/crio-225f08521b30059726aede60906b42d18e6fda3347e365b90fd5a53060fdafd3 WatchSource:0}: Error finding container 225f08521b30059726aede60906b42d18e6fda3347e365b90fd5a53060fdafd3: Status 404 returned error can't find the container with id 225f08521b30059726aede60906b42d18e6fda3347e365b90fd5a53060fdafd3 Jul 10 16:54:17.735295 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.735274 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-djhll" Jul 10 16:54:17.741275 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.741249 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-w9kb4" Jul 10 16:54:17.742371 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:17.742350 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded7de755_820a_4395_8b5f_ea6abfb86699.slice/crio-2aef94bf4547858f6b35730d6a6c2c0aa77ba89e5760a9f9e2c1eedc6d4d8c34 WatchSource:0}: Error finding container 2aef94bf4547858f6b35730d6a6c2c0aa77ba89e5760a9f9e2c1eedc6d4d8c34: Status 404 returned error can't find the container with id 2aef94bf4547858f6b35730d6a6c2c0aa77ba89e5760a9f9e2c1eedc6d4d8c34 Jul 10 16:54:17.749569 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:17.749545 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53987bf8_a205_4326_8757_ea8b3a6c4514.slice/crio-16286c648d477ebf099097c6b3aba69b1d4db9a190e0d7722e23e998c067266e WatchSource:0}: Error finding container 16286c648d477ebf099097c6b3aba69b1d4db9a190e0d7722e23e998c067266e: Status 404 returned error can't find the container with id 16286c648d477ebf099097c6b3aba69b1d4db9a190e0d7722e23e998c067266e Jul 10 16:54:17.992151 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.992061 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d43ce12f-a8fd-4a94-bb94-09e6562e1250-metrics-certs\") pod \"network-metrics-daemon-jr8jk\" (UID: \"d43ce12f-a8fd-4a94-bb94-09e6562e1250\") " pod="openshift-multus/network-metrics-daemon-jr8jk" Jul 10 16:54:17.992313 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:17.992176 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a0b23658-26fb-454a-a42e-88432bf12b0a-original-pull-secret\") pod \"global-pull-secret-syncer-dgk2w\" (UID: \"a0b23658-26fb-454a-a42e-88432bf12b0a\") " pod="kube-system/global-pull-secret-syncer-dgk2w" Jul 10 16:54:17.992313 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:17.992195 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 16:54:17.992313 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:17.992264 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d43ce12f-a8fd-4a94-bb94-09e6562e1250-metrics-certs podName:d43ce12f-a8fd-4a94-bb94-09e6562e1250 nodeName:}" failed. No retries permitted until 2026-07-10 16:54:18.992243403 +0000 UTC m=+3.081572437 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d43ce12f-a8fd-4a94-bb94-09e6562e1250-metrics-certs") pod "network-metrics-daemon-jr8jk" (UID: "d43ce12f-a8fd-4a94-bb94-09e6562e1250") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 16:54:17.992313 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:17.992288 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 16:54:17.992546 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:17.992326 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0b23658-26fb-454a-a42e-88432bf12b0a-original-pull-secret podName:a0b23658-26fb-454a-a42e-88432bf12b0a nodeName:}" failed. No retries permitted until 2026-07-10 16:54:18.992313703 +0000 UTC m=+3.081642736 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a0b23658-26fb-454a-a42e-88432bf12b0a-original-pull-secret") pod "global-pull-secret-syncer-dgk2w" (UID: "a0b23658-26fb-454a-a42e-88432bf12b0a") : object "kube-system"/"original-pull-secret" not registered Jul 10 16:54:18.093030 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:18.092979 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfk5q\" (UniqueName: \"kubernetes.io/projected/16033c9b-cf7b-4afc-abe6-df0ded50ff80-kube-api-access-sfk5q\") pod \"network-check-target-jvkw6\" (UID: \"16033c9b-cf7b-4afc-abe6-df0ded50ff80\") " pod="openshift-network-diagnostics/network-check-target-jvkw6" Jul 10 16:54:18.093214 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:18.093159 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 16:54:18.093214 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:18.093187 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 16:54:18.093214 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:18.093198 2578 projected.go:196] Error preparing data for projected volume kube-api-access-sfk5q for pod openshift-network-diagnostics/network-check-target-jvkw6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 16:54:18.093335 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:18.093249 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/16033c9b-cf7b-4afc-abe6-df0ded50ff80-kube-api-access-sfk5q podName:16033c9b-cf7b-4afc-abe6-df0ded50ff80 nodeName:}" failed. No retries permitted until 2026-07-10 16:54:19.093234919 +0000 UTC m=+3.182563951 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-sfk5q" (UniqueName: "kubernetes.io/projected/16033c9b-cf7b-4afc-abe6-df0ded50ff80-kube-api-access-sfk5q") pod "network-check-target-jvkw6" (UID: "16033c9b-cf7b-4afc-abe6-df0ded50ff80") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 16:54:18.302590 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:18.302552 2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 16:54:18.404369 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:18.404298 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 16:49:17 +0000 UTC" deadline="2028-03-09 04:22:39.406243223 +0000 UTC" Jul 10 16:54:18.404369 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:18.404364 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14579h28m21.001883806s" Jul 10 16:54:18.422644 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:18.422610 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgk2w" Jul 10 16:54:18.422804 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:18.422757 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dgk2w" podUID="a0b23658-26fb-454a-a42e-88432bf12b0a" Jul 10 16:54:18.450832 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:18.450420 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" event={"ID":"b9a53126-f278-4a9d-9735-5f73dd0c1964","Type":"ContainerStarted","Data":"81fdbf1c219472452ca21a6cb8344d1ba5876f173df1e1708a63de92aaecb21e"} Jul 10 16:54:18.456421 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:18.456312 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-7.ec2.internal" event={"ID":"ac3f323c6b38ca654d9468732cf2193c","Type":"ContainerStarted","Data":"6a196354071095922fa71c2d2cf03dc4c2d6f1f7517fa8733210c0371fe66a27"} Jul 10 16:54:18.460429 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:18.460337 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-sdvff" event={"ID":"cea5b1b2-0b38-41e0-ac8d-7e573be5ddb8","Type":"ContainerStarted","Data":"70bac432a054ed6fe06784fb112b1531a34bbc782cabde9d7e373ea50fd9b216"} Jul 10 16:54:18.465259 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:18.465172 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6kk9g" event={"ID":"1d85dbfa-1b16-4e26-b207-6bb7064345ee","Type":"ContainerStarted","Data":"a5d0a2dcb292125f2b47a89e0f3780f8606e222a4adeb02d5e65bea7a697ace8"} Jul 10 16:54:18.472748 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:18.472707 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-7.ec2.internal" event={"ID":"374e81600297ab5c614c91e79771b865","Type":"ContainerStarted","Data":"5195f207d37a32b453f671fb061b642f567552930800ae21cf098d7c21ea762f"} Jul 10 16:54:18.479672 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:18.479632 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-w9kb4" event={"ID":"53987bf8-a205-4326-8757-ea8b3a6c4514","Type":"ContainerStarted","Data":"16286c648d477ebf099097c6b3aba69b1d4db9a190e0d7722e23e998c067266e"} Jul 10 16:54:18.499254 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:18.499213 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-djhll" event={"ID":"ed7de755-820a-4395-8b5f-ea6abfb86699","Type":"ContainerStarted","Data":"2aef94bf4547858f6b35730d6a6c2c0aa77ba89e5760a9f9e2c1eedc6d4d8c34"} Jul 10 16:54:18.532328 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:18.532284 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5d4nh" event={"ID":"6e1806e2-a453-4929-8632-267349af86c7","Type":"ContainerStarted","Data":"225f08521b30059726aede60906b42d18e6fda3347e365b90fd5a53060fdafd3"} Jul 10 16:54:18.573395 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:18.573257 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9qtfh" event={"ID":"a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1","Type":"ContainerStarted","Data":"3ab3239553bc919496bb11d4adde496d46252fdef063848ae9fe988ca6f19982"} Jul 10 16:54:18.596446 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:18.596395 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8tnf" event={"ID":"dc38a8bd-42b1-4809-88ce-66aa03182c8c","Type":"ContainerStarted","Data":"369e2fa44042b10b18efce84707dea559d98852ef1e65f5d0a31f656b85cfd76"} Jul 10 16:54:18.603352 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:18.603313 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-t6rhv" event={"ID":"08351a06-c5eb-4c51-923c-283f5b28a7d2","Type":"ContainerStarted","Data":"35140ed47d3d8eaeeb1bfd3669bc886912501af187eb6e4a08ce61f5211f558b"} Jul 10 16:54:19.001014 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:19.000976 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a0b23658-26fb-454a-a42e-88432bf12b0a-original-pull-secret\") pod \"global-pull-secret-syncer-dgk2w\" (UID: \"a0b23658-26fb-454a-a42e-88432bf12b0a\") " pod="kube-system/global-pull-secret-syncer-dgk2w" Jul 10 16:54:19.001215 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:19.001066 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d43ce12f-a8fd-4a94-bb94-09e6562e1250-metrics-certs\") pod \"network-metrics-daemon-jr8jk\" (UID: \"d43ce12f-a8fd-4a94-bb94-09e6562e1250\") " pod="openshift-multus/network-metrics-daemon-jr8jk" Jul 10 16:54:19.001215 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:19.001210 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 16:54:19.001319 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:19.001269 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d43ce12f-a8fd-4a94-bb94-09e6562e1250-metrics-certs podName:d43ce12f-a8fd-4a94-bb94-09e6562e1250 nodeName:}" failed. No retries permitted until 2026-07-10 16:54:21.001251225 +0000 UTC m=+5.090580255 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d43ce12f-a8fd-4a94-bb94-09e6562e1250-metrics-certs") pod "network-metrics-daemon-jr8jk" (UID: "d43ce12f-a8fd-4a94-bb94-09e6562e1250") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 16:54:19.001650 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:19.001630 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 16:54:19.001720 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:19.001685 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0b23658-26fb-454a-a42e-88432bf12b0a-original-pull-secret podName:a0b23658-26fb-454a-a42e-88432bf12b0a nodeName:}" failed. No retries permitted until 2026-07-10 16:54:21.001670898 +0000 UTC m=+5.090999932 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a0b23658-26fb-454a-a42e-88432bf12b0a-original-pull-secret") pod "global-pull-secret-syncer-dgk2w" (UID: "a0b23658-26fb-454a-a42e-88432bf12b0a") : object "kube-system"/"original-pull-secret" not registered Jul 10 16:54:19.101733 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:19.101686 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfk5q\" (UniqueName: \"kubernetes.io/projected/16033c9b-cf7b-4afc-abe6-df0ded50ff80-kube-api-access-sfk5q\") pod \"network-check-target-jvkw6\" (UID: \"16033c9b-cf7b-4afc-abe6-df0ded50ff80\") " pod="openshift-network-diagnostics/network-check-target-jvkw6" Jul 10 16:54:19.101917 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:19.101869 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 16:54:19.101917 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:19.101893 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 16:54:19.101917 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:19.101907 2578 projected.go:196] Error preparing data for projected volume kube-api-access-sfk5q for pod openshift-network-diagnostics/network-check-target-jvkw6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 16:54:19.102087 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:19.101973 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/16033c9b-cf7b-4afc-abe6-df0ded50ff80-kube-api-access-sfk5q podName:16033c9b-cf7b-4afc-abe6-df0ded50ff80 nodeName:}" failed. No retries permitted until 2026-07-10 16:54:21.101954358 +0000 UTC m=+5.191283407 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-sfk5q" (UniqueName: "kubernetes.io/projected/16033c9b-cf7b-4afc-abe6-df0ded50ff80-kube-api-access-sfk5q") pod "network-check-target-jvkw6" (UID: "16033c9b-cf7b-4afc-abe6-df0ded50ff80") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 16:54:19.404686 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:19.404636 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 16:49:17 +0000 UTC" deadline="2027-12-05 20:01:41.393233021 +0000 UTC" Jul 10 16:54:19.404686 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:19.404682 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12315h7m21.988556274s" Jul 10 16:54:19.420794 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:19.420041 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jr8jk" Jul 10 16:54:19.420794 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:19.420213 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jr8jk" podUID="d43ce12f-a8fd-4a94-bb94-09e6562e1250" Jul 10 16:54:19.420794 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:19.420645 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jvkw6" Jul 10 16:54:19.420794 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:19.420737 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jvkw6" podUID="16033c9b-cf7b-4afc-abe6-df0ded50ff80" Jul 10 16:54:19.805710 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:19.805396 2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 16:54:20.421991 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:20.421959 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgk2w" Jul 10 16:54:20.422480 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:20.422116 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dgk2w" podUID="a0b23658-26fb-454a-a42e-88432bf12b0a" Jul 10 16:54:21.017807 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:21.017765 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a0b23658-26fb-454a-a42e-88432bf12b0a-original-pull-secret\") pod \"global-pull-secret-syncer-dgk2w\" (UID: \"a0b23658-26fb-454a-a42e-88432bf12b0a\") " pod="kube-system/global-pull-secret-syncer-dgk2w" Jul 10 16:54:21.017993 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:21.017853 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d43ce12f-a8fd-4a94-bb94-09e6562e1250-metrics-certs\") pod \"network-metrics-daemon-jr8jk\" (UID: \"d43ce12f-a8fd-4a94-bb94-09e6562e1250\") " pod="openshift-multus/network-metrics-daemon-jr8jk" Jul 10 16:54:21.018142 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:21.017996 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 16:54:21.018142 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:21.018071 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d43ce12f-a8fd-4a94-bb94-09e6562e1250-metrics-certs podName:d43ce12f-a8fd-4a94-bb94-09e6562e1250 nodeName:}" failed. No retries permitted until 2026-07-10 16:54:25.01803505 +0000 UTC m=+9.107364092 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d43ce12f-a8fd-4a94-bb94-09e6562e1250-metrics-certs") pod "network-metrics-daemon-jr8jk" (UID: "d43ce12f-a8fd-4a94-bb94-09e6562e1250") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 16:54:21.018523 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:21.018485 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 16:54:21.018620 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:21.018539 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0b23658-26fb-454a-a42e-88432bf12b0a-original-pull-secret podName:a0b23658-26fb-454a-a42e-88432bf12b0a nodeName:}" failed. No retries permitted until 2026-07-10 16:54:25.018524043 +0000 UTC m=+9.107853073 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a0b23658-26fb-454a-a42e-88432bf12b0a-original-pull-secret") pod "global-pull-secret-syncer-dgk2w" (UID: "a0b23658-26fb-454a-a42e-88432bf12b0a") : object "kube-system"/"original-pull-secret" not registered Jul 10 16:54:21.119798 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:21.119157 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfk5q\" (UniqueName: \"kubernetes.io/projected/16033c9b-cf7b-4afc-abe6-df0ded50ff80-kube-api-access-sfk5q\") pod \"network-check-target-jvkw6\" (UID: \"16033c9b-cf7b-4afc-abe6-df0ded50ff80\") " pod="openshift-network-diagnostics/network-check-target-jvkw6" Jul 10 16:54:21.119798 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:21.119378 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 16:54:21.119798 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:21.119396 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 16:54:21.119798 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:21.119408 2578 projected.go:196] Error preparing data for projected volume kube-api-access-sfk5q for pod openshift-network-diagnostics/network-check-target-jvkw6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 16:54:21.119798 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:21.119465 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/16033c9b-cf7b-4afc-abe6-df0ded50ff80-kube-api-access-sfk5q podName:16033c9b-cf7b-4afc-abe6-df0ded50ff80 nodeName:}" failed. No retries permitted until 2026-07-10 16:54:25.119446898 +0000 UTC m=+9.208775928 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-sfk5q" (UniqueName: "kubernetes.io/projected/16033c9b-cf7b-4afc-abe6-df0ded50ff80-kube-api-access-sfk5q") pod "network-check-target-jvkw6" (UID: "16033c9b-cf7b-4afc-abe6-df0ded50ff80") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 16:54:21.420663 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:21.420452 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jvkw6" Jul 10 16:54:21.420663 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:21.420476 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jr8jk" Jul 10 16:54:21.420663 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:21.420558 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jvkw6" podUID="16033c9b-cf7b-4afc-abe6-df0ded50ff80" Jul 10 16:54:21.420881 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:21.420673 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jr8jk" podUID="d43ce12f-a8fd-4a94-bb94-09e6562e1250" Jul 10 16:54:22.422964 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:22.422477 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgk2w" Jul 10 16:54:22.422964 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:22.422601 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dgk2w" podUID="a0b23658-26fb-454a-a42e-88432bf12b0a" Jul 10 16:54:23.420155 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:23.420118 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jvkw6" Jul 10 16:54:23.420342 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:23.420264 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jvkw6" podUID="16033c9b-cf7b-4afc-abe6-df0ded50ff80" Jul 10 16:54:23.420668 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:23.420636 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jr8jk" Jul 10 16:54:23.420774 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:23.420751 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jr8jk" podUID="d43ce12f-a8fd-4a94-bb94-09e6562e1250" Jul 10 16:54:24.420515 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:24.420262 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgk2w" Jul 10 16:54:24.420915 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:24.420634 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dgk2w" podUID="a0b23658-26fb-454a-a42e-88432bf12b0a" Jul 10 16:54:25.051467 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:25.051425 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d43ce12f-a8fd-4a94-bb94-09e6562e1250-metrics-certs\") pod \"network-metrics-daemon-jr8jk\" (UID: \"d43ce12f-a8fd-4a94-bb94-09e6562e1250\") " pod="openshift-multus/network-metrics-daemon-jr8jk" Jul 10 16:54:25.051663 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:25.051500 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a0b23658-26fb-454a-a42e-88432bf12b0a-original-pull-secret\") pod \"global-pull-secret-syncer-dgk2w\" (UID: \"a0b23658-26fb-454a-a42e-88432bf12b0a\") " pod="kube-system/global-pull-secret-syncer-dgk2w" Jul 10 16:54:25.052307 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:25.052171 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 16:54:25.052853 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:25.052832 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d43ce12f-a8fd-4a94-bb94-09e6562e1250-metrics-certs podName:d43ce12f-a8fd-4a94-bb94-09e6562e1250 nodeName:}" failed. No retries permitted until 2026-07-10 16:54:33.052237695 +0000 UTC m=+17.141566726 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d43ce12f-a8fd-4a94-bb94-09e6562e1250-metrics-certs") pod "network-metrics-daemon-jr8jk" (UID: "d43ce12f-a8fd-4a94-bb94-09e6562e1250") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 16:54:25.060463 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:25.060429 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 16:54:25.060603 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:25.060528 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0b23658-26fb-454a-a42e-88432bf12b0a-original-pull-secret podName:a0b23658-26fb-454a-a42e-88432bf12b0a nodeName:}" failed. No retries permitted until 2026-07-10 16:54:33.060505003 +0000 UTC m=+17.149834031 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a0b23658-26fb-454a-a42e-88432bf12b0a-original-pull-secret") pod "global-pull-secret-syncer-dgk2w" (UID: "a0b23658-26fb-454a-a42e-88432bf12b0a") : object "kube-system"/"original-pull-secret" not registered Jul 10 16:54:25.152251 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:25.152207 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfk5q\" (UniqueName: \"kubernetes.io/projected/16033c9b-cf7b-4afc-abe6-df0ded50ff80-kube-api-access-sfk5q\") pod \"network-check-target-jvkw6\" (UID: \"16033c9b-cf7b-4afc-abe6-df0ded50ff80\") " pod="openshift-network-diagnostics/network-check-target-jvkw6" Jul 10 16:54:25.152437 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:25.152405 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 16:54:25.152437 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:25.152425 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 16:54:25.152437 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:25.152437 2578 projected.go:196] Error preparing data for projected volume kube-api-access-sfk5q for pod openshift-network-diagnostics/network-check-target-jvkw6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 16:54:25.152560 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:25.152497 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/16033c9b-cf7b-4afc-abe6-df0ded50ff80-kube-api-access-sfk5q podName:16033c9b-cf7b-4afc-abe6-df0ded50ff80 nodeName:}" failed. No retries permitted until 2026-07-10 16:54:33.152478981 +0000 UTC m=+17.241808008 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-sfk5q" (UniqueName: "kubernetes.io/projected/16033c9b-cf7b-4afc-abe6-df0ded50ff80-kube-api-access-sfk5q") pod "network-check-target-jvkw6" (UID: "16033c9b-cf7b-4afc-abe6-df0ded50ff80") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 16:54:25.419967 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:25.419870 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jvkw6" Jul 10 16:54:25.420147 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:25.419984 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jvkw6" podUID="16033c9b-cf7b-4afc-abe6-df0ded50ff80" Jul 10 16:54:25.420477 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:25.420356 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jr8jk" Jul 10 16:54:25.420477 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:25.420444 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jr8jk" podUID="d43ce12f-a8fd-4a94-bb94-09e6562e1250" Jul 10 16:54:26.420647 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:26.420618 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgk2w" Jul 10 16:54:26.421177 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:26.420702 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dgk2w" podUID="a0b23658-26fb-454a-a42e-88432bf12b0a" Jul 10 16:54:27.419701 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:27.419659 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jvkw6" Jul 10 16:54:27.419871 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:27.419779 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jvkw6" podUID="16033c9b-cf7b-4afc-abe6-df0ded50ff80" Jul 10 16:54:27.420332 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:27.420182 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jr8jk" Jul 10 16:54:27.420332 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:27.420285 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jr8jk" podUID="d43ce12f-a8fd-4a94-bb94-09e6562e1250" Jul 10 16:54:28.420260 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:28.420227 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgk2w" Jul 10 16:54:28.420698 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:28.420353 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dgk2w" podUID="a0b23658-26fb-454a-a42e-88432bf12b0a" Jul 10 16:54:29.420332 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:29.420289 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jr8jk" Jul 10 16:54:29.420749 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:29.420340 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jvkw6" Jul 10 16:54:29.420749 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:29.420433 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jr8jk" podUID="d43ce12f-a8fd-4a94-bb94-09e6562e1250" Jul 10 16:54:29.420749 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:29.420566 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jvkw6" podUID="16033c9b-cf7b-4afc-abe6-df0ded50ff80" Jul 10 16:54:30.421800 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:30.421755 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgk2w" Jul 10 16:54:30.422328 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:30.421880 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dgk2w" podUID="a0b23658-26fb-454a-a42e-88432bf12b0a" Jul 10 16:54:31.419818 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:31.419783 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jr8jk" Jul 10 16:54:31.419988 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:31.419891 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jr8jk" podUID="d43ce12f-a8fd-4a94-bb94-09e6562e1250" Jul 10 16:54:31.420144 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:31.420125 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jvkw6" Jul 10 16:54:31.420250 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:31.420226 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jvkw6" podUID="16033c9b-cf7b-4afc-abe6-df0ded50ff80" Jul 10 16:54:32.420489 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:32.420454 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgk2w" Jul 10 16:54:32.420834 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:32.420558 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dgk2w" podUID="a0b23658-26fb-454a-a42e-88432bf12b0a" Jul 10 16:54:33.116832 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:33.116795 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a0b23658-26fb-454a-a42e-88432bf12b0a-original-pull-secret\") pod \"global-pull-secret-syncer-dgk2w\" (UID: \"a0b23658-26fb-454a-a42e-88432bf12b0a\") " pod="kube-system/global-pull-secret-syncer-dgk2w" Jul 10 16:54:33.117030 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:33.116850 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d43ce12f-a8fd-4a94-bb94-09e6562e1250-metrics-certs\") pod \"network-metrics-daemon-jr8jk\" (UID: \"d43ce12f-a8fd-4a94-bb94-09e6562e1250\") " pod="openshift-multus/network-metrics-daemon-jr8jk" Jul 10 16:54:33.117030 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:33.116969 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 16:54:33.117030 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:33.116971 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 16:54:33.117030 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:33.117031 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d43ce12f-a8fd-4a94-bb94-09e6562e1250-metrics-certs podName:d43ce12f-a8fd-4a94-bb94-09e6562e1250 nodeName:}" failed. No retries permitted until 2026-07-10 16:54:49.117008961 +0000 UTC m=+33.206337990 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d43ce12f-a8fd-4a94-bb94-09e6562e1250-metrics-certs") pod "network-metrics-daemon-jr8jk" (UID: "d43ce12f-a8fd-4a94-bb94-09e6562e1250") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 16:54:33.117267 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:33.117064 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0b23658-26fb-454a-a42e-88432bf12b0a-original-pull-secret podName:a0b23658-26fb-454a-a42e-88432bf12b0a nodeName:}" failed. No retries permitted until 2026-07-10 16:54:49.117041677 +0000 UTC m=+33.206370705 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a0b23658-26fb-454a-a42e-88432bf12b0a-original-pull-secret") pod "global-pull-secret-syncer-dgk2w" (UID: "a0b23658-26fb-454a-a42e-88432bf12b0a") : object "kube-system"/"original-pull-secret" not registered Jul 10 16:54:33.218064 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:33.218015 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfk5q\" (UniqueName: \"kubernetes.io/projected/16033c9b-cf7b-4afc-abe6-df0ded50ff80-kube-api-access-sfk5q\") pod \"network-check-target-jvkw6\" (UID: \"16033c9b-cf7b-4afc-abe6-df0ded50ff80\") " pod="openshift-network-diagnostics/network-check-target-jvkw6" Jul 10 16:54:33.218225 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:33.218189 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 16:54:33.218225 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:33.218207 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 16:54:33.218225 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:33.218221 2578 projected.go:196] Error preparing data for projected volume kube-api-access-sfk5q for pod openshift-network-diagnostics/network-check-target-jvkw6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 16:54:33.218378 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:33.218276 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/16033c9b-cf7b-4afc-abe6-df0ded50ff80-kube-api-access-sfk5q podName:16033c9b-cf7b-4afc-abe6-df0ded50ff80 nodeName:}" failed. No retries permitted until 2026-07-10 16:54:49.218260846 +0000 UTC m=+33.307589877 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-sfk5q" (UniqueName: "kubernetes.io/projected/16033c9b-cf7b-4afc-abe6-df0ded50ff80-kube-api-access-sfk5q") pod "network-check-target-jvkw6" (UID: "16033c9b-cf7b-4afc-abe6-df0ded50ff80") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 16:54:33.420284 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:33.420188 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jvkw6" Jul 10 16:54:33.420436 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:33.420195 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jr8jk" Jul 10 16:54:33.420436 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:33.420337 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jvkw6" podUID="16033c9b-cf7b-4afc-abe6-df0ded50ff80" Jul 10 16:54:33.420436 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:33.420378 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jr8jk" podUID="d43ce12f-a8fd-4a94-bb94-09e6562e1250" Jul 10 16:54:34.419655 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:34.419617 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgk2w" Jul 10 16:54:34.419855 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:34.419752 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dgk2w" podUID="a0b23658-26fb-454a-a42e-88432bf12b0a" Jul 10 16:54:35.420265 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:35.420228 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jr8jk" Jul 10 16:54:35.420731 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:35.420226 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jvkw6" Jul 10 16:54:35.420731 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:35.420378 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jr8jk" podUID="d43ce12f-a8fd-4a94-bb94-09e6562e1250" Jul 10 16:54:35.420731 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:35.420425 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jvkw6" podUID="16033c9b-cf7b-4afc-abe6-df0ded50ff80" Jul 10 16:54:36.421023 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:36.420983 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgk2w" Jul 10 16:54:36.421404 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:36.421126 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dgk2w" podUID="a0b23658-26fb-454a-a42e-88432bf12b0a" Jul 10 16:54:37.420316 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:37.420280 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jvkw6" Jul 10 16:54:37.420495 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:37.420422 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jvkw6" podUID="16033c9b-cf7b-4afc-abe6-df0ded50ff80" Jul 10 16:54:37.420495 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:37.420480 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jr8jk" Jul 10 16:54:37.420626 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:37.420606 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jr8jk" podUID="d43ce12f-a8fd-4a94-bb94-09e6562e1250" Jul 10 16:54:38.419936 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:38.419905 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgk2w" Jul 10 16:54:38.420436 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:38.420020 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dgk2w" podUID="a0b23658-26fb-454a-a42e-88432bf12b0a" Jul 10 16:54:39.420144 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:39.419836 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jvkw6" Jul 10 16:54:39.420685 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:39.419892 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jr8jk" Jul 10 16:54:39.420685 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:39.420223 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jvkw6" podUID="16033c9b-cf7b-4afc-abe6-df0ded50ff80" Jul 10 16:54:39.420685 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:39.420290 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jr8jk" podUID="d43ce12f-a8fd-4a94-bb94-09e6562e1250" Jul 10 16:54:39.657715 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:39.657677 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9qtfh" event={"ID":"a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1","Type":"ContainerStarted","Data":"120f20f55f5f73a37f2718a969ad38de6d9a58b3033bab43de9f2492539418ee"} Jul 10 16:54:39.659095 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:39.659066 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8tnf" event={"ID":"dc38a8bd-42b1-4809-88ce-66aa03182c8c","Type":"ContainerStarted","Data":"203d38d1b03bbf0f43aa3f8f5f21ae0c4316d32533851ad17b80d77c48073105"} Jul 10 16:54:39.660354 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:39.660315 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-t6rhv" event={"ID":"08351a06-c5eb-4c51-923c-283f5b28a7d2","Type":"ContainerStarted","Data":"0625940f87a5e22031e5a2b9f84bfcbf1f3e402175357a95902c61b1d16e2eab"} Jul 10 16:54:39.663099 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:39.663081 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gqgkg_b9a53126-f278-4a9d-9735-5f73dd0c1964/ovn-acl-logging/0.log" Jul 10 16:54:39.663415 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:39.663395 2578 generic.go:356] "Generic (PLEG): container finished" podID="b9a53126-f278-4a9d-9735-5f73dd0c1964" containerID="f7cdda8242a3b4650459dcf01073be4cbe1a8a9554b2f1c64a5d21fbf75e41a1" exitCode=1 Jul 10 16:54:39.663490 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:39.663472 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" event={"ID":"b9a53126-f278-4a9d-9735-5f73dd0c1964","Type":"ContainerStarted","Data":"2b0070dd3f9c90d88204ecdb95a47108e3e0b0d48d3ba730a0278cca929cb7c4"} Jul 10 16:54:39.663543 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:39.663527 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" event={"ID":"b9a53126-f278-4a9d-9735-5f73dd0c1964","Type":"ContainerStarted","Data":"bdf57055c036e27abf037aead69bf274a036470581f5791c60bf3dd7faf39c43"} Jul 10 16:54:39.663586 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:39.663567 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" event={"ID":"b9a53126-f278-4a9d-9735-5f73dd0c1964","Type":"ContainerStarted","Data":"8b6c7293a61d9f4e0a473060274b5e59051134c3031a02f5cf7c87a8d5c0d1fb"} Jul 10 16:54:39.663618 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:39.663599 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" event={"ID":"b9a53126-f278-4a9d-9735-5f73dd0c1964","Type":"ContainerStarted","Data":"673f59b5d5f81342d97f94679ee66b2ae9bfcf32411ae8ed750c38a891a75a50"} Jul 10 16:54:39.663649 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:39.663635 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" event={"ID":"b9a53126-f278-4a9d-9735-5f73dd0c1964","Type":"ContainerDied","Data":"f7cdda8242a3b4650459dcf01073be4cbe1a8a9554b2f1c64a5d21fbf75e41a1"} Jul 10 16:54:39.663694 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:39.663679 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" event={"ID":"b9a53126-f278-4a9d-9735-5f73dd0c1964","Type":"ContainerStarted","Data":"ce0d70f462ad8e9ce169e7d8337f400b1a638514b43b207cea3e5fd373437bc2"} Jul 10 16:54:39.664863 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:39.664842 2578 generic.go:356] "Generic (PLEG): container finished" podID="ac3f323c6b38ca654d9468732cf2193c" containerID="1b3ff21ff6a445d6ff1eea1640ecc12c0145773bee407e6ec7b2ee92d43bad17" exitCode=0 Jul 10 16:54:39.664919 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:39.664875 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-7.ec2.internal" event={"ID":"ac3f323c6b38ca654d9468732cf2193c","Type":"ContainerDied","Data":"1b3ff21ff6a445d6ff1eea1640ecc12c0145773bee407e6ec7b2ee92d43bad17"} Jul 10 16:54:39.666266 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:39.666242 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6kk9g" event={"ID":"1d85dbfa-1b16-4e26-b207-6bb7064345ee","Type":"ContainerStarted","Data":"ce9442596305b977cc267647e62e736d0b94dfb1a971c3ad5ed1c214cb537eb9"} Jul 10 16:54:39.667676 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:39.667657 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-7.ec2.internal" event={"ID":"374e81600297ab5c614c91e79771b865","Type":"ContainerStarted","Data":"f4225835e78ece9dca8cffa06adaf19fe7617cfad1fa079c43b406c51af4f864"} Jul 10 16:54:39.668988 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:39.668968 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-w9kb4" event={"ID":"53987bf8-a205-4326-8757-ea8b3a6c4514","Type":"ContainerStarted","Data":"2667b5d41f116fc6814da4849b16bb8a1e59be3a2cf7e8373dfa099b8473e6f8"} Jul 10 16:54:39.670458 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:39.670438 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-djhll" event={"ID":"ed7de755-820a-4395-8b5f-ea6abfb86699","Type":"ContainerStarted","Data":"6ebca2d769239fe4b1591051ab2fa7d648dd8190181c4bf45782540817c792bc"} Jul 10 16:54:39.671852 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:39.671833 2578 generic.go:356] "Generic (PLEG): container finished" podID="6e1806e2-a453-4929-8632-267349af86c7" containerID="be5f1cf2f356b79cce454d3c86118f2979635a0abf03cbae447666f16b1b98c0" exitCode=0 Jul 10 16:54:39.671930 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:39.671858 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5d4nh" event={"ID":"6e1806e2-a453-4929-8632-267349af86c7","Type":"ContainerDied","Data":"be5f1cf2f356b79cce454d3c86118f2979635a0abf03cbae447666f16b1b98c0"} Jul 10 16:54:39.675860 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:39.675826 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-9qtfh" podStartSLOduration=2.669796526 podStartE2EDuration="23.675811197s" totalImagesPullingTime="21.006014671s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 16:54:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:54:39.675232816 +0000 UTC m=+23.764561865" watchObservedRunningTime="2026-07-10 16:54:39.675811197 +0000 UTC m=+23.765140251" Jul 10 16:54:39.704823 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:39.704742 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-6kk9g" podStartSLOduration=2.805303438 podStartE2EDuration="23.704726883s" totalImagesPullingTime="20.899423445s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 16:54:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:54:39.690030785 +0000 UTC m=+23.779359836" watchObservedRunningTime="2026-07-10 16:54:39.704726883 +0000 UTC m=+23.794055936" Jul 10 16:54:39.705151 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:39.705113 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-t6rhv" podStartSLOduration=2.885111675 podStartE2EDuration="23.705104295s" totalImagesPullingTime="20.81999262s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:54:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:54:39.704173115 +0000 UTC m=+23.793502164" watchObservedRunningTime="2026-07-10 16:54:39.705104295 +0000 UTC m=+23.794433347" Jul 10 16:54:39.729454 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:39.729387 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-djhll" podStartSLOduration=1.9448137380000001 podStartE2EDuration="22.729366713s" totalImagesPullingTime="20.784552975s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 16:54:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:54:39.728683453 +0000 UTC m=+23.818012503" watchObservedRunningTime="2026-07-10 16:54:39.729366713 +0000 UTC m=+23.818695763" Jul 10 16:54:39.742745 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:39.742704 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-7.ec2.internal" podStartSLOduration=22.742689143 podStartE2EDuration="22.742689143s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 16:54:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:54:39.742645833 +0000 UTC m=+23.831974883" watchObservedRunningTime="2026-07-10 16:54:39.742689143 +0000 UTC m=+23.832018193" Jul 10 16:54:39.794202 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:39.794163 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-w9kb4" podStartSLOduration=2.018689244 podStartE2EDuration="22.794147803s" totalImagesPullingTime="20.775458559s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 16:54:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:54:39.793770138 +0000 UTC m=+23.883099190" watchObservedRunningTime="2026-07-10 16:54:39.794147803 +0000 UTC m=+23.883476853" Jul 10 16:54:40.263511 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:40.263430 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-t6rhv" Jul 10 16:54:40.263995 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:40.263976 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-t6rhv" Jul 10 16:54:40.420835 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:40.420800 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgk2w" Jul 10 16:54:40.421290 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:40.420948 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dgk2w" podUID="a0b23658-26fb-454a-a42e-88432bf12b0a" Jul 10 16:54:40.508176 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:40.508149 2578 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 10 16:54:40.676079 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:40.676025 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8tnf" event={"ID":"dc38a8bd-42b1-4809-88ce-66aa03182c8c","Type":"ContainerStarted","Data":"edbc3d7ad169bf45b93ff1340aab55f2eceb9645c44632ecea0b6b60bc4a7931"} Jul 10 16:54:40.677792 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:40.677771 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-7.ec2.internal" event={"ID":"ac3f323c6b38ca654d9468732cf2193c","Type":"ContainerStarted","Data":"89d06c4c42894203b655bd931fc484cf07e525b35b86f7f15ee3ab000646d3e4"} Jul 10 16:54:40.679299 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:40.679264 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-sdvff" event={"ID":"cea5b1b2-0b38-41e0-ac8d-7e573be5ddb8","Type":"ContainerStarted","Data":"0d737a5ebef8dbe3513faf2292478519442df6efcf0105f57d2fdfaa337327ab"} Jul 10 16:54:40.680067 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:40.680026 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-t6rhv" Jul 10 16:54:40.680467 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:40.680452 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-t6rhv" Jul 10 16:54:40.706391 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:40.706350 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-7.ec2.internal" podStartSLOduration=23.706335391 podStartE2EDuration="23.706335391s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 16:54:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:54:40.692586141 +0000 UTC m=+24.781915190" watchObservedRunningTime="2026-07-10 16:54:40.706335391 +0000 UTC m=+24.795664440" Jul 10 16:54:40.720919 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:40.720879 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-sdvff" podStartSLOduration=3.864219433 podStartE2EDuration="24.720864513s" totalImagesPullingTime="20.85664508s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 16:54:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:54:40.720364902 +0000 UTC m=+24.809693952" watchObservedRunningTime="2026-07-10 16:54:40.720864513 +0000 UTC m=+24.810193563" Jul 10 16:54:41.359729 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:41.359609 2578 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-10T16:54:40.508169284Z","UUID":"e1aa7921-45df-4617-9e52-9b6966937d5f","Handler":null,"Name":"","Endpoint":""} Jul 10 16:54:41.362260 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:41.362235 2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 10 16:54:41.362400 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:41.362277 2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 10 16:54:41.420324 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:41.420292 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jvkw6" Jul 10 16:54:41.420494 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:41.420413 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jvkw6" podUID="16033c9b-cf7b-4afc-abe6-df0ded50ff80" Jul 10 16:54:41.420494 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:41.420448 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jr8jk" Jul 10 16:54:41.420586 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:41.420532 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jr8jk" podUID="d43ce12f-a8fd-4a94-bb94-09e6562e1250" Jul 10 16:54:42.420247 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:42.419991 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgk2w" Jul 10 16:54:42.420669 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:42.420352 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dgk2w" podUID="a0b23658-26fb-454a-a42e-88432bf12b0a" Jul 10 16:54:42.685704 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:42.685608 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8tnf" event={"ID":"dc38a8bd-42b1-4809-88ce-66aa03182c8c","Type":"ContainerStarted","Data":"ca40b07ef52cf678293ac47d08703f7d3cdd5cb97cc21106e4c0e21dd0e45ba2"} Jul 10 16:54:42.688381 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:42.688360 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gqgkg_b9a53126-f278-4a9d-9735-5f73dd0c1964/ovn-acl-logging/0.log" Jul 10 16:54:42.688839 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:42.688810 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" event={"ID":"b9a53126-f278-4a9d-9735-5f73dd0c1964","Type":"ContainerStarted","Data":"031ee0f5f69b68c6d8ddf72d47577207cdcc28ae252e2125b8b1189fe45b9589"} Jul 10 16:54:42.704539 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:42.704489 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8tnf" podStartSLOduration=3.324646946 podStartE2EDuration="26.70447674s" totalImagesPullingTime="23.379829794s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 16:54:16 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:54:42.704178978 +0000 UTC m=+26.793508030" watchObservedRunningTime="2026-07-10 16:54:42.70447674 +0000 UTC m=+26.793805789" Jul 10 16:54:43.420462 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:43.420430 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jvkw6" Jul 10 16:54:43.420938 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:43.420431 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jr8jk" Jul 10 16:54:43.420938 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:43.420640 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jr8jk" podUID="d43ce12f-a8fd-4a94-bb94-09e6562e1250" Jul 10 16:54:43.420938 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:43.420526 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jvkw6" podUID="16033c9b-cf7b-4afc-abe6-df0ded50ff80" Jul 10 16:54:44.419751 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:44.419721 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgk2w" Jul 10 16:54:44.419926 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:44.419843 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dgk2w" podUID="a0b23658-26fb-454a-a42e-88432bf12b0a" Jul 10 16:54:45.420274 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:45.420072 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jr8jk" Jul 10 16:54:45.420940 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:45.420096 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jvkw6" Jul 10 16:54:45.420940 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:45.420395 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jr8jk" podUID="d43ce12f-a8fd-4a94-bb94-09e6562e1250" Jul 10 16:54:45.420940 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:45.420434 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jvkw6" podUID="16033c9b-cf7b-4afc-abe6-df0ded50ff80" Jul 10 16:54:45.697393 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:45.697358 2578 generic.go:356] "Generic (PLEG): container finished" podID="6e1806e2-a453-4929-8632-267349af86c7" containerID="f7ab28ddbc0050b0616efbb8ff1e9b8a5bf88e1b473677b97f187572fee4e4a9" exitCode=0 Jul 10 16:54:45.697561 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:45.697448 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5d4nh" event={"ID":"6e1806e2-a453-4929-8632-267349af86c7","Type":"ContainerDied","Data":"f7ab28ddbc0050b0616efbb8ff1e9b8a5bf88e1b473677b97f187572fee4e4a9"} Jul 10 16:54:45.701147 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:45.701124 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gqgkg_b9a53126-f278-4a9d-9735-5f73dd0c1964/ovn-acl-logging/0.log" Jul 10 16:54:45.701468 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:45.701447 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" event={"ID":"b9a53126-f278-4a9d-9735-5f73dd0c1964","Type":"ContainerStarted","Data":"3c451afbe1d53e9795942e6bce570d2fcd48e5cfa1b448d3d1d69e2e27a7dccf"} Jul 10 16:54:45.701702 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:45.701682 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:45.701855 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:45.701842 2578 scope.go:117] "RemoveContainer" containerID="f7cdda8242a3b4650459dcf01073be4cbe1a8a9554b2f1c64a5d21fbf75e41a1" Jul 10 16:54:45.721506 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:45.721484 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:46.420687 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:46.420648 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgk2w" Jul 10 16:54:46.421184 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:46.420773 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dgk2w" podUID="a0b23658-26fb-454a-a42e-88432bf12b0a" Jul 10 16:54:46.708338 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:46.708287 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gqgkg_b9a53126-f278-4a9d-9735-5f73dd0c1964/ovn-acl-logging/0.log" Jul 10 16:54:46.708679 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:46.708651 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" event={"ID":"b9a53126-f278-4a9d-9735-5f73dd0c1964","Type":"ContainerStarted","Data":"786c2e8903bd3b8dbf8b7238f414358797d5bd2f8f2670660cd89469d443c70b"} Jul 10 16:54:46.708886 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:46.708861 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:46.709007 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:46.708937 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:46.725097 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:46.725068 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:54:46.739237 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:46.739186 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" podStartSLOduration=9.789227466 podStartE2EDuration="30.739170828s" totalImagesPullingTime="20.949943362s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 16:54:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:54:46.73888528 +0000 UTC m=+30.828214332" watchObservedRunningTime="2026-07-10 16:54:46.739170828 +0000 UTC m=+30.828499926" Jul 10 16:54:47.080806 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:47.080769 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dgk2w"] Jul 10 16:54:47.080987 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:47.080940 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgk2w" Jul 10 16:54:47.081097 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:47.081070 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dgk2w" podUID="a0b23658-26fb-454a-a42e-88432bf12b0a" Jul 10 16:54:47.083742 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:47.083719 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jvkw6"] Jul 10 16:54:47.083869 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:47.083831 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jvkw6" Jul 10 16:54:47.083954 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:47.083932 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jvkw6" podUID="16033c9b-cf7b-4afc-abe6-df0ded50ff80" Jul 10 16:54:47.084270 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:47.084248 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jr8jk"] Jul 10 16:54:47.084372 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:47.084357 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jr8jk" Jul 10 16:54:47.084514 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:47.084491 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jr8jk" podUID="d43ce12f-a8fd-4a94-bb94-09e6562e1250" Jul 10 16:54:47.712587 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:47.712551 2578 generic.go:356] "Generic (PLEG): container finished" podID="6e1806e2-a453-4929-8632-267349af86c7" containerID="fac774e1bfd449484d65d51b9bd78009b992920db3774af68865523405639c48" exitCode=0 Jul 10 16:54:47.713043 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:47.712628 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5d4nh" event={"ID":"6e1806e2-a453-4929-8632-267349af86c7","Type":"ContainerDied","Data":"fac774e1bfd449484d65d51b9bd78009b992920db3774af68865523405639c48"} Jul 10 16:54:48.419943 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:48.419754 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jr8jk" Jul 10 16:54:48.420135 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:48.419763 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgk2w" Jul 10 16:54:48.420135 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:48.420066 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jr8jk" podUID="d43ce12f-a8fd-4a94-bb94-09e6562e1250" Jul 10 16:54:48.420135 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:48.420120 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dgk2w" podUID="a0b23658-26fb-454a-a42e-88432bf12b0a" Jul 10 16:54:48.716897 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:48.716855 2578 generic.go:356] "Generic (PLEG): container finished" podID="6e1806e2-a453-4929-8632-267349af86c7" containerID="fe4eaa798b3b16e481cdf0a12c2e625f5012ce822edf64832c11292cfe911509" exitCode=0 Jul 10 16:54:48.717432 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:48.716947 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5d4nh" event={"ID":"6e1806e2-a453-4929-8632-267349af86c7","Type":"ContainerDied","Data":"fe4eaa798b3b16e481cdf0a12c2e625f5012ce822edf64832c11292cfe911509"} Jul 10 16:54:49.133871 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:49.133833 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d43ce12f-a8fd-4a94-bb94-09e6562e1250-metrics-certs\") pod \"network-metrics-daemon-jr8jk\" (UID: \"d43ce12f-a8fd-4a94-bb94-09e6562e1250\") " pod="openshift-multus/network-metrics-daemon-jr8jk" Jul 10 16:54:49.134020 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:49.133892 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a0b23658-26fb-454a-a42e-88432bf12b0a-original-pull-secret\") pod \"global-pull-secret-syncer-dgk2w\" (UID: \"a0b23658-26fb-454a-a42e-88432bf12b0a\") " pod="kube-system/global-pull-secret-syncer-dgk2w" Jul 10 16:54:49.134020 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:49.133987 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 16:54:49.134020 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:49.133991 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 16:54:49.134157 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:49.134040 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0b23658-26fb-454a-a42e-88432bf12b0a-original-pull-secret podName:a0b23658-26fb-454a-a42e-88432bf12b0a nodeName:}" failed. No retries permitted until 2026-07-10 16:55:21.134026624 +0000 UTC m=+65.223355652 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a0b23658-26fb-454a-a42e-88432bf12b0a-original-pull-secret") pod "global-pull-secret-syncer-dgk2w" (UID: "a0b23658-26fb-454a-a42e-88432bf12b0a") : object "kube-system"/"original-pull-secret" not registered Jul 10 16:54:49.134157 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:49.134078 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d43ce12f-a8fd-4a94-bb94-09e6562e1250-metrics-certs podName:d43ce12f-a8fd-4a94-bb94-09e6562e1250 nodeName:}" failed. No retries permitted until 2026-07-10 16:55:21.134068345 +0000 UTC m=+65.223397376 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d43ce12f-a8fd-4a94-bb94-09e6562e1250-metrics-certs") pod "network-metrics-daemon-jr8jk" (UID: "d43ce12f-a8fd-4a94-bb94-09e6562e1250") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 16:54:49.234676 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:49.234634 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfk5q\" (UniqueName: \"kubernetes.io/projected/16033c9b-cf7b-4afc-abe6-df0ded50ff80-kube-api-access-sfk5q\") pod \"network-check-target-jvkw6\" (UID: \"16033c9b-cf7b-4afc-abe6-df0ded50ff80\") " pod="openshift-network-diagnostics/network-check-target-jvkw6" Jul 10 16:54:49.234859 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:49.234836 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 16:54:49.234922 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:49.234869 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 16:54:49.234922 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:49.234883 2578 projected.go:196] Error preparing data for projected volume kube-api-access-sfk5q for pod openshift-network-diagnostics/network-check-target-jvkw6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 16:54:49.235023 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:49.234946 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/16033c9b-cf7b-4afc-abe6-df0ded50ff80-kube-api-access-sfk5q podName:16033c9b-cf7b-4afc-abe6-df0ded50ff80 nodeName:}" failed. No retries permitted until 2026-07-10 16:55:21.234929659 +0000 UTC m=+65.324258700 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-sfk5q" (UniqueName: "kubernetes.io/projected/16033c9b-cf7b-4afc-abe6-df0ded50ff80-kube-api-access-sfk5q") pod "network-check-target-jvkw6" (UID: "16033c9b-cf7b-4afc-abe6-df0ded50ff80") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 16:54:49.419600 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:49.419523 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jvkw6" Jul 10 16:54:49.419754 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:49.419635 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jvkw6" podUID="16033c9b-cf7b-4afc-abe6-df0ded50ff80" Jul 10 16:54:50.424184 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:50.424153 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jr8jk" Jul 10 16:54:50.424184 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:50.424187 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgk2w" Jul 10 16:54:50.424886 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:50.424295 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jr8jk" podUID="d43ce12f-a8fd-4a94-bb94-09e6562e1250" Jul 10 16:54:50.424886 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:50.424422 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dgk2w" podUID="a0b23658-26fb-454a-a42e-88432bf12b0a" Jul 10 16:54:51.419484 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:51.419454 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jvkw6" Jul 10 16:54:51.419661 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:51.419578 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jvkw6" podUID="16033c9b-cf7b-4afc-abe6-df0ded50ff80" Jul 10 16:54:52.231871 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.231839 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-7.ec2.internal" event="NodeReady" Jul 10 16:54:52.232334 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.231992 2578 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 10 16:54:52.268852 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.268816 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zfbfz"] Jul 10 16:54:52.271256 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.271229 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zfbfz" Jul 10 16:54:52.273123 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.273100 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-657d95d548-bj62l"] Jul 10 16:54:52.274762 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.274743 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-657d95d548-bj62l" Jul 10 16:54:52.275728 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.275701 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 10 16:54:52.275893 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.275801 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-mxq54\"" Jul 10 16:54:52.276001 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.275931 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 10 16:54:52.276001 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.275711 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 10 16:54:52.276401 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.276253 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 10 16:54:52.278111 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.277261 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fxc54"] Jul 10 16:54:52.278111 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.277320 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 10 16:54:52.278351 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.278327 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 10 16:54:52.278614 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.278597 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 10 16:54:52.278937 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.278922 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-bb9cb67d8-ddrbp"] Jul 10 16:54:52.279090 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.279075 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fxc54" Jul 10 16:54:52.279547 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.279529 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-phvw6\"" Jul 10 16:54:52.280769 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.280744 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-d5bgt"] Jul 10 16:54:52.281100 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.281084 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-bb9cb67d8-ddrbp" Jul 10 16:54:52.286405 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.286383 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 10 16:54:52.286570 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.286549 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-j6txs\"" Jul 10 16:54:52.286905 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.286754 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 10 16:54:52.286905 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.286770 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 10 16:54:52.286905 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.286842 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 10 16:54:52.290096 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.290075 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 10 16:54:52.290901 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.290872 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 10 16:54:52.291118 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.291093 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-5j4ws"] Jul 10 16:54:52.291218 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.291175 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 10 16:54:52.291274 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.291257 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 10 16:54:52.291443 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.291427 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 10 16:54:52.291570 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.291507 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 10 16:54:52.291629 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.291593 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-l7wpk\"" Jul 10 16:54:52.291938 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.291915 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-d5bgt" Jul 10 16:54:52.293326 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.292892 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-8r74k"] Jul 10 16:54:52.294000 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.293983 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-5j4ws" Jul 10 16:54:52.295264 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.295245 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 10 16:54:52.295719 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.295670 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-96td8\"" Jul 10 16:54:52.296022 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.295832 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 10 16:54:52.296182 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.296120 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zfbfz"] Jul 10 16:54:52.296182 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.296171 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fxc54"] Jul 10 16:54:52.296298 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.296196 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-8r74k"] Jul 10 16:54:52.296298 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.296225 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 10 16:54:52.296392 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.296315 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 10 16:54:52.296392 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.296355 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-8r74k" Jul 10 16:54:52.297147 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.297129 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-5j4ws"] Jul 10 16:54:52.297483 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.297459 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 10 16:54:52.297647 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.297633 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 10 16:54:52.298951 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.298934 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-w6xjc\"" Jul 10 16:54:52.300532 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.300513 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-bb9cb67d8-ddrbp"] Jul 10 16:54:52.302239 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.301185 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 10 16:54:52.302239 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.301490 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 10 16:54:52.302239 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.301547 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 10 16:54:52.302239 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.301778 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 10 16:54:52.302864 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.302844 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-g5nt8"] Jul 10 16:54:52.304005 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.303989 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 10 16:54:52.304859 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.304838 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-g5nt8" Jul 10 16:54:52.305292 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.305271 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-jk625\"" Jul 10 16:54:52.306865 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.306309 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 10 16:54:52.306865 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.306490 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-xc54t"] Jul 10 16:54:52.308269 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.308250 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 10 16:54:52.308449 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.308388 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ccqfl\"" Jul 10 16:54:52.308524 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.308482 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-gqqc7"] Jul 10 16:54:52.308569 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.308554 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-xc54t" Jul 10 16:54:52.310202 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.310186 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-5nbcv"] Jul 10 16:54:52.310740 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.310396 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-gqqc7" Jul 10 16:54:52.312348 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.312331 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-hwfb8"] Jul 10 16:54:52.312488 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.312470 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-5nbcv" Jul 10 16:54:52.314943 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.314918 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hwfb8" Jul 10 16:54:52.318762 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.318691 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 10 16:54:52.319172 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.319152 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 10 16:54:52.319935 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.319917 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-657d95d548-bj62l"] Jul 10 16:54:52.322115 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.322093 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 10 16:54:52.325640 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.325623 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 10 16:54:52.325886 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.325654 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 10 16:54:52.325970 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.325713 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 10 16:54:52.325970 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.325719 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-mx8qr\"" Jul 10 16:54:52.326326 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.326295 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 10 16:54:52.326410 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.326328 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-nmpnr\"" Jul 10 16:54:52.326747 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.326728 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-b5v82\"" Jul 10 16:54:52.327767 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.327738 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 10 16:54:52.328140 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.328121 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 10 16:54:52.328408 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.328392 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 10 16:54:52.328542 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.328521 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 10 16:54:52.328683 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.328603 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-z5mrn\"" Jul 10 16:54:52.328763 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.328734 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 10 16:54:52.333289 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.333205 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 10 16:54:52.334699 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.334681 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-g5nt8"] Jul 10 16:54:52.335102 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.335085 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-d5bgt"] Jul 10 16:54:52.335728 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.335715 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hwfb8"] Jul 10 16:54:52.336843 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.336563 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-xc54t"] Jul 10 16:54:52.337102 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.337083 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 10 16:54:52.337540 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.337520 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-5nbcv"] Jul 10 16:54:52.358802 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.358775 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/73c4ea4d-fc8b-4fca-b4bd-2fc9102d4347-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-8r74k\" (UID: \"73c4ea4d-fc8b-4fca-b4bd-2fc9102d4347\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-8r74k" Jul 10 16:54:52.358943 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.358820 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/40988796-3d52-4414-89dc-53c322b4d514-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-zfbfz\" (UID: \"40988796-3d52-4414-89dc-53c322b4d514\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zfbfz" Jul 10 16:54:52.358943 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.358845 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/04e15cd0-bcac-4375-97d5-b135b8ef9997-trusted-ca\") pod \"image-registry-657d95d548-bj62l\" (UID: \"04e15cd0-bcac-4375-97d5-b135b8ef9997\") " pod="openshift-image-registry/image-registry-657d95d548-bj62l" Jul 10 16:54:52.358943 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.358869 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ef2eb691-dbbf-4cfc-9fc1-bba4cd71895b-snapshots\") pod \"insights-operator-68d9dd94cf-d5bgt\" (UID: \"ef2eb691-dbbf-4cfc-9fc1-bba4cd71895b\") " pod="openshift-insights/insights-operator-68d9dd94cf-d5bgt" Jul 10 16:54:52.358943 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.358887 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/40988796-3d52-4414-89dc-53c322b4d514-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-zfbfz\" (UID: \"40988796-3d52-4414-89dc-53c322b4d514\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zfbfz" Jul 10 16:54:52.359150 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.358953 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/27742f15-b602-4554-84dc-ff9d46a9adda-config\") pod \"console-operator-7866497494-5j4ws\" (UID: \"27742f15-b602-4554-84dc-ff9d46a9adda\") " pod="openshift-console-operator/console-operator-7866497494-5j4ws" Jul 10 16:54:52.359150 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.358988 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/04e15cd0-bcac-4375-97d5-b135b8ef9997-image-registry-private-configuration\") pod \"image-registry-657d95d548-bj62l\" (UID: \"04e15cd0-bcac-4375-97d5-b135b8ef9997\") " pod="openshift-image-registry/image-registry-657d95d548-bj62l" Jul 10 16:54:52.359150 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.359017 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgc22\" (UniqueName: \"kubernetes.io/projected/c0ba50ec-0b78-46ce-9b5c-e644af63989d-kube-api-access-wgc22\") pod \"cluster-samples-operator-65db858894-fxc54\" (UID: \"c0ba50ec-0b78-46ce-9b5c-e644af63989d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fxc54" Jul 10 16:54:52.359150 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.359073 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/04e15cd0-bcac-4375-97d5-b135b8ef9997-bound-sa-token\") pod \"image-registry-657d95d548-bj62l\" (UID: \"04e15cd0-bcac-4375-97d5-b135b8ef9997\") " pod="openshift-image-registry/image-registry-657d95d548-bj62l" Jul 10 16:54:52.359150 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.359102 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84-metrics-certs\") pod \"router-default-bb9cb67d8-ddrbp\" (UID: \"3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84\") " pod="openshift-ingress/router-default-bb9cb67d8-ddrbp" Jul 10 16:54:52.359150 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.359127 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-gqqc7"] Jul 10 16:54:52.359412 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.359130 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqcxv\" (UniqueName: \"kubernetes.io/projected/40988796-3d52-4414-89dc-53c322b4d514-kube-api-access-cqcxv\") pod \"cluster-monitoring-operator-5f7f494dd4-zfbfz\" (UID: \"40988796-3d52-4414-89dc-53c322b4d514\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zfbfz" Jul 10 16:54:52.359412 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.359226 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/04e15cd0-bcac-4375-97d5-b135b8ef9997-ca-trust-extracted\") pod \"image-registry-657d95d548-bj62l\" (UID: \"04e15cd0-bcac-4375-97d5-b135b8ef9997\") " pod="openshift-image-registry/image-registry-657d95d548-bj62l" Jul 10 16:54:52.359412 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.359253 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6xwk\" (UniqueName: \"kubernetes.io/projected/04e15cd0-bcac-4375-97d5-b135b8ef9997-kube-api-access-q6xwk\") pod \"image-registry-657d95d548-bj62l\" (UID: \"04e15cd0-bcac-4375-97d5-b135b8ef9997\") " pod="openshift-image-registry/image-registry-657d95d548-bj62l" Jul 10 16:54:52.359412 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.359272 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ef2eb691-dbbf-4cfc-9fc1-bba4cd71895b-tmp\") pod \"insights-operator-68d9dd94cf-d5bgt\" (UID: \"ef2eb691-dbbf-4cfc-9fc1-bba4cd71895b\") " pod="openshift-insights/insights-operator-68d9dd94cf-d5bgt" Jul 10 16:54:52.359412 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.359295 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84-stats-auth\") pod \"router-default-bb9cb67d8-ddrbp\" (UID: \"3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84\") " pod="openshift-ingress/router-default-bb9cb67d8-ddrbp" Jul 10 16:54:52.359412 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.359324 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8zmk\" (UniqueName: \"kubernetes.io/projected/73c4ea4d-fc8b-4fca-b4bd-2fc9102d4347-kube-api-access-h8zmk\") pod \"kube-storage-version-migrator-operator-799cb7f958-8r74k\" (UID: \"73c4ea4d-fc8b-4fca-b4bd-2fc9102d4347\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-8r74k" Jul 10 16:54:52.359412 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.359352 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ef2eb691-dbbf-4cfc-9fc1-bba4cd71895b-serving-cert\") pod \"insights-operator-68d9dd94cf-d5bgt\" (UID: \"ef2eb691-dbbf-4cfc-9fc1-bba4cd71895b\") " pod="openshift-insights/insights-operator-68d9dd94cf-d5bgt" Jul 10 16:54:52.359412 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.359372 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54d4j\" (UniqueName: \"kubernetes.io/projected/ef2eb691-dbbf-4cfc-9fc1-bba4cd71895b-kube-api-access-54d4j\") pod \"insights-operator-68d9dd94cf-d5bgt\" (UID: \"ef2eb691-dbbf-4cfc-9fc1-bba4cd71895b\") " pod="openshift-insights/insights-operator-68d9dd94cf-d5bgt" Jul 10 16:54:52.359412 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.359389 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pr6jw\" (UniqueName: \"kubernetes.io/projected/3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84-kube-api-access-pr6jw\") pod \"router-default-bb9cb67d8-ddrbp\" (UID: \"3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84\") " pod="openshift-ingress/router-default-bb9cb67d8-ddrbp" Jul 10 16:54:52.359412 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.359413 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef2eb691-dbbf-4cfc-9fc1-bba4cd71895b-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-d5bgt\" (UID: \"ef2eb691-dbbf-4cfc-9fc1-bba4cd71895b\") " pod="openshift-insights/insights-operator-68d9dd94cf-d5bgt" Jul 10 16:54:52.359859 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.359432 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dswfl\" (UniqueName: \"kubernetes.io/projected/27742f15-b602-4554-84dc-ff9d46a9adda-kube-api-access-dswfl\") pod \"console-operator-7866497494-5j4ws\" (UID: \"27742f15-b602-4554-84dc-ff9d46a9adda\") " pod="openshift-console-operator/console-operator-7866497494-5j4ws" Jul 10 16:54:52.359859 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.359448 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84-service-ca-bundle\") pod \"router-default-bb9cb67d8-ddrbp\" (UID: \"3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84\") " pod="openshift-ingress/router-default-bb9cb67d8-ddrbp" Jul 10 16:54:52.359859 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.359464 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c0ba50ec-0b78-46ce-9b5c-e644af63989d-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-fxc54\" (UID: \"c0ba50ec-0b78-46ce-9b5c-e644af63989d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fxc54" Jul 10 16:54:52.359859 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.359488 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/04e15cd0-bcac-4375-97d5-b135b8ef9997-registry-tls\") pod \"image-registry-657d95d548-bj62l\" (UID: \"04e15cd0-bcac-4375-97d5-b135b8ef9997\") " pod="openshift-image-registry/image-registry-657d95d548-bj62l" Jul 10 16:54:52.359859 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.359506 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/27742f15-b602-4554-84dc-ff9d46a9adda-trusted-ca\") pod \"console-operator-7866497494-5j4ws\" (UID: \"27742f15-b602-4554-84dc-ff9d46a9adda\") " pod="openshift-console-operator/console-operator-7866497494-5j4ws" Jul 10 16:54:52.359859 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.359524 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/04e15cd0-bcac-4375-97d5-b135b8ef9997-installation-pull-secrets\") pod \"image-registry-657d95d548-bj62l\" (UID: \"04e15cd0-bcac-4375-97d5-b135b8ef9997\") " pod="openshift-image-registry/image-registry-657d95d548-bj62l" Jul 10 16:54:52.359859 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.359550 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/73c4ea4d-fc8b-4fca-b4bd-2fc9102d4347-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-8r74k\" (UID: \"73c4ea4d-fc8b-4fca-b4bd-2fc9102d4347\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-8r74k" Jul 10 16:54:52.359859 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.359573 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84-default-certificate\") pod \"router-default-bb9cb67d8-ddrbp\" (UID: \"3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84\") " pod="openshift-ingress/router-default-bb9cb67d8-ddrbp" Jul 10 16:54:52.359859 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.359590 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/04e15cd0-bcac-4375-97d5-b135b8ef9997-registry-certificates\") pod \"image-registry-657d95d548-bj62l\" (UID: \"04e15cd0-bcac-4375-97d5-b135b8ef9997\") " pod="openshift-image-registry/image-registry-657d95d548-bj62l" Jul 10 16:54:52.359859 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.359618 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef2eb691-dbbf-4cfc-9fc1-bba4cd71895b-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-d5bgt\" (UID: \"ef2eb691-dbbf-4cfc-9fc1-bba4cd71895b\") " pod="openshift-insights/insights-operator-68d9dd94cf-d5bgt" Jul 10 16:54:52.359859 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.359640 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/27742f15-b602-4554-84dc-ff9d46a9adda-serving-cert\") pod \"console-operator-7866497494-5j4ws\" (UID: \"27742f15-b602-4554-84dc-ff9d46a9adda\") " pod="openshift-console-operator/console-operator-7866497494-5j4ws" Jul 10 16:54:52.423626 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.423548 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgk2w" Jul 10 16:54:52.423778 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.423732 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jr8jk" Jul 10 16:54:52.427399 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.427375 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 10 16:54:52.427891 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.427871 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 10 16:54:52.428182 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.428161 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qdmq6\"" Jul 10 16:54:52.460025 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.459961 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/99905b79-72f4-478d-9c3a-05967f561ee2-config\") pod \"service-ca-operator-7695fc6c8c-gqqc7\" (UID: \"99905b79-72f4-478d-9c3a-05967f561ee2\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-gqqc7" Jul 10 16:54:52.460352 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.460324 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/27742f15-b602-4554-84dc-ff9d46a9adda-config\") pod \"console-operator-7866497494-5j4ws\" (UID: \"27742f15-b602-4554-84dc-ff9d46a9adda\") " pod="openshift-console-operator/console-operator-7866497494-5j4ws" Jul 10 16:54:52.460464 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.460379 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/04e15cd0-bcac-4375-97d5-b135b8ef9997-image-registry-private-configuration\") pod \"image-registry-657d95d548-bj62l\" (UID: \"04e15cd0-bcac-4375-97d5-b135b8ef9997\") " pod="openshift-image-registry/image-registry-657d95d548-bj62l" Jul 10 16:54:52.460464 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.460411 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgc22\" (UniqueName: \"kubernetes.io/projected/c0ba50ec-0b78-46ce-9b5c-e644af63989d-kube-api-access-wgc22\") pod \"cluster-samples-operator-65db858894-fxc54\" (UID: \"c0ba50ec-0b78-46ce-9b5c-e644af63989d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fxc54" Jul 10 16:54:52.460464 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.460446 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/04e15cd0-bcac-4375-97d5-b135b8ef9997-bound-sa-token\") pod \"image-registry-657d95d548-bj62l\" (UID: \"04e15cd0-bcac-4375-97d5-b135b8ef9997\") " pod="openshift-image-registry/image-registry-657d95d548-bj62l" Jul 10 16:54:52.460655 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.460475 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84-metrics-certs\") pod \"router-default-bb9cb67d8-ddrbp\" (UID: \"3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84\") " pod="openshift-ingress/router-default-bb9cb67d8-ddrbp" Jul 10 16:54:52.460655 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.460500 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqcxv\" (UniqueName: \"kubernetes.io/projected/40988796-3d52-4414-89dc-53c322b4d514-kube-api-access-cqcxv\") pod \"cluster-monitoring-operator-5f7f494dd4-zfbfz\" (UID: \"40988796-3d52-4414-89dc-53c322b4d514\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zfbfz" Jul 10 16:54:52.460655 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.460530 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/04e15cd0-bcac-4375-97d5-b135b8ef9997-ca-trust-extracted\") pod \"image-registry-657d95d548-bj62l\" (UID: \"04e15cd0-bcac-4375-97d5-b135b8ef9997\") " pod="openshift-image-registry/image-registry-657d95d548-bj62l" Jul 10 16:54:52.460655 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.460554 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/99905b79-72f4-478d-9c3a-05967f561ee2-serving-cert\") pod \"service-ca-operator-7695fc6c8c-gqqc7\" (UID: \"99905b79-72f4-478d-9c3a-05967f561ee2\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-gqqc7" Jul 10 16:54:52.460655 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.460583 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6xwk\" (UniqueName: \"kubernetes.io/projected/04e15cd0-bcac-4375-97d5-b135b8ef9997-kube-api-access-q6xwk\") pod \"image-registry-657d95d548-bj62l\" (UID: \"04e15cd0-bcac-4375-97d5-b135b8ef9997\") " pod="openshift-image-registry/image-registry-657d95d548-bj62l" Jul 10 16:54:52.460876 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:52.460690 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 16:54:52.460876 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:52.460756 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84-metrics-certs podName:3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84 nodeName:}" failed. No retries permitted until 2026-07-10 16:54:52.960736711 +0000 UTC m=+37.050065739 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84-metrics-certs") pod "router-default-bb9cb67d8-ddrbp" (UID: "3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84") : secret "router-metrics-certs-default" not found Jul 10 16:54:52.460876 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.460807 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xv96\" (UniqueName: \"kubernetes.io/projected/99905b79-72f4-478d-9c3a-05967f561ee2-kube-api-access-7xv96\") pod \"service-ca-operator-7695fc6c8c-gqqc7\" (UID: \"99905b79-72f4-478d-9c3a-05967f561ee2\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-gqqc7" Jul 10 16:54:52.460876 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.460848 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ef2eb691-dbbf-4cfc-9fc1-bba4cd71895b-tmp\") pod \"insights-operator-68d9dd94cf-d5bgt\" (UID: \"ef2eb691-dbbf-4cfc-9fc1-bba4cd71895b\") " pod="openshift-insights/insights-operator-68d9dd94cf-d5bgt" Jul 10 16:54:52.461098 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.460881 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84-stats-auth\") pod \"router-default-bb9cb67d8-ddrbp\" (UID: \"3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84\") " pod="openshift-ingress/router-default-bb9cb67d8-ddrbp" Jul 10 16:54:52.461098 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.460911 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8zmk\" (UniqueName: \"kubernetes.io/projected/73c4ea4d-fc8b-4fca-b4bd-2fc9102d4347-kube-api-access-h8zmk\") pod \"kube-storage-version-migrator-operator-799cb7f958-8r74k\" (UID: \"73c4ea4d-fc8b-4fca-b4bd-2fc9102d4347\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-8r74k" Jul 10 16:54:52.461098 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.460940 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ef2eb691-dbbf-4cfc-9fc1-bba4cd71895b-serving-cert\") pod \"insights-operator-68d9dd94cf-d5bgt\" (UID: \"ef2eb691-dbbf-4cfc-9fc1-bba4cd71895b\") " pod="openshift-insights/insights-operator-68d9dd94cf-d5bgt" Jul 10 16:54:52.461098 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.460965 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54d4j\" (UniqueName: \"kubernetes.io/projected/ef2eb691-dbbf-4cfc-9fc1-bba4cd71895b-kube-api-access-54d4j\") pod \"insights-operator-68d9dd94cf-d5bgt\" (UID: \"ef2eb691-dbbf-4cfc-9fc1-bba4cd71895b\") " pod="openshift-insights/insights-operator-68d9dd94cf-d5bgt" Jul 10 16:54:52.461098 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.460991 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/55d7dbb1-c8db-40fc-b416-7b781aae9aab-metrics-tls\") pod \"dns-default-g5nt8\" (UID: \"55d7dbb1-c8db-40fc-b416-7b781aae9aab\") " pod="openshift-dns/dns-default-g5nt8" Jul 10 16:54:52.461098 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.461017 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pr6jw\" (UniqueName: \"kubernetes.io/projected/3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84-kube-api-access-pr6jw\") pod \"router-default-bb9cb67d8-ddrbp\" (UID: \"3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84\") " pod="openshift-ingress/router-default-bb9cb67d8-ddrbp" Jul 10 16:54:52.461098 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.461042 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6qp7\" (UniqueName: \"kubernetes.io/projected/633753a4-3340-4473-9639-4299c06b53a8-kube-api-access-b6qp7\") pod \"ingress-canary-hwfb8\" (UID: \"633753a4-3340-4473-9639-4299c06b53a8\") " pod="openshift-ingress-canary/ingress-canary-hwfb8" Jul 10 16:54:52.461098 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.461072 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/04e15cd0-bcac-4375-97d5-b135b8ef9997-ca-trust-extracted\") pod \"image-registry-657d95d548-bj62l\" (UID: \"04e15cd0-bcac-4375-97d5-b135b8ef9997\") " pod="openshift-image-registry/image-registry-657d95d548-bj62l" Jul 10 16:54:52.461098 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.461100 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef2eb691-dbbf-4cfc-9fc1-bba4cd71895b-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-d5bgt\" (UID: \"ef2eb691-dbbf-4cfc-9fc1-bba4cd71895b\") " pod="openshift-insights/insights-operator-68d9dd94cf-d5bgt" Jul 10 16:54:52.461567 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.461124 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55d7dbb1-c8db-40fc-b416-7b781aae9aab-tmp-dir\") pod \"dns-default-g5nt8\" (UID: \"55d7dbb1-c8db-40fc-b416-7b781aae9aab\") " pod="openshift-dns/dns-default-g5nt8" Jul 10 16:54:52.461567 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.461153 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dswfl\" (UniqueName: \"kubernetes.io/projected/27742f15-b602-4554-84dc-ff9d46a9adda-kube-api-access-dswfl\") pod \"console-operator-7866497494-5j4ws\" (UID: \"27742f15-b602-4554-84dc-ff9d46a9adda\") " pod="openshift-console-operator/console-operator-7866497494-5j4ws" Jul 10 16:54:52.461567 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.461178 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84-service-ca-bundle\") pod \"router-default-bb9cb67d8-ddrbp\" (UID: \"3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84\") " pod="openshift-ingress/router-default-bb9cb67d8-ddrbp" Jul 10 16:54:52.461567 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.461204 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c0ba50ec-0b78-46ce-9b5c-e644af63989d-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-fxc54\" (UID: \"c0ba50ec-0b78-46ce-9b5c-e644af63989d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fxc54" Jul 10 16:54:52.461567 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.461227 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/633753a4-3340-4473-9639-4299c06b53a8-cert\") pod \"ingress-canary-hwfb8\" (UID: \"633753a4-3340-4473-9639-4299c06b53a8\") " pod="openshift-ingress-canary/ingress-canary-hwfb8" Jul 10 16:54:52.461567 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.461258 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/04e15cd0-bcac-4375-97d5-b135b8ef9997-registry-tls\") pod \"image-registry-657d95d548-bj62l\" (UID: \"04e15cd0-bcac-4375-97d5-b135b8ef9997\") " pod="openshift-image-registry/image-registry-657d95d548-bj62l" Jul 10 16:54:52.461567 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.461282 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/27742f15-b602-4554-84dc-ff9d46a9adda-trusted-ca\") pod \"console-operator-7866497494-5j4ws\" (UID: \"27742f15-b602-4554-84dc-ff9d46a9adda\") " pod="openshift-console-operator/console-operator-7866497494-5j4ws" Jul 10 16:54:52.461567 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.461314 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/04e15cd0-bcac-4375-97d5-b135b8ef9997-installation-pull-secrets\") pod \"image-registry-657d95d548-bj62l\" (UID: \"04e15cd0-bcac-4375-97d5-b135b8ef9997\") " pod="openshift-image-registry/image-registry-657d95d548-bj62l" Jul 10 16:54:52.461567 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.461340 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/73c4ea4d-fc8b-4fca-b4bd-2fc9102d4347-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-8r74k\" (UID: \"73c4ea4d-fc8b-4fca-b4bd-2fc9102d4347\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-8r74k" Jul 10 16:54:52.461567 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.461365 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84-default-certificate\") pod \"router-default-bb9cb67d8-ddrbp\" (UID: \"3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84\") " pod="openshift-ingress/router-default-bb9cb67d8-ddrbp" Jul 10 16:54:52.461567 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.461392 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/04e15cd0-bcac-4375-97d5-b135b8ef9997-registry-certificates\") pod \"image-registry-657d95d548-bj62l\" (UID: \"04e15cd0-bcac-4375-97d5-b135b8ef9997\") " pod="openshift-image-registry/image-registry-657d95d548-bj62l" Jul 10 16:54:52.461567 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.461400 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ef2eb691-dbbf-4cfc-9fc1-bba4cd71895b-tmp\") pod \"insights-operator-68d9dd94cf-d5bgt\" (UID: \"ef2eb691-dbbf-4cfc-9fc1-bba4cd71895b\") " pod="openshift-insights/insights-operator-68d9dd94cf-d5bgt" Jul 10 16:54:52.461567 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.461418 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftbrt\" (UniqueName: \"kubernetes.io/projected/54cce7d7-9921-4a19-aaf7-d909fbfafa8f-kube-api-access-ftbrt\") pod \"volume-data-source-validator-78495597f6-xc54t\" (UID: \"54cce7d7-9921-4a19-aaf7-d909fbfafa8f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-xc54t" Jul 10 16:54:52.461567 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.461452 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef2eb691-dbbf-4cfc-9fc1-bba4cd71895b-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-d5bgt\" (UID: \"ef2eb691-dbbf-4cfc-9fc1-bba4cd71895b\") " pod="openshift-insights/insights-operator-68d9dd94cf-d5bgt" Jul 10 16:54:52.461567 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.461465 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/27742f15-b602-4554-84dc-ff9d46a9adda-config\") pod \"console-operator-7866497494-5j4ws\" (UID: \"27742f15-b602-4554-84dc-ff9d46a9adda\") " pod="openshift-console-operator/console-operator-7866497494-5j4ws" Jul 10 16:54:52.461567 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.461481 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/27742f15-b602-4554-84dc-ff9d46a9adda-serving-cert\") pod \"console-operator-7866497494-5j4ws\" (UID: \"27742f15-b602-4554-84dc-ff9d46a9adda\") " pod="openshift-console-operator/console-operator-7866497494-5j4ws" Jul 10 16:54:52.462389 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.461537 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/73c4ea4d-fc8b-4fca-b4bd-2fc9102d4347-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-8r74k\" (UID: \"73c4ea4d-fc8b-4fca-b4bd-2fc9102d4347\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-8r74k" Jul 10 16:54:52.462389 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.461568 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/40988796-3d52-4414-89dc-53c322b4d514-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-zfbfz\" (UID: \"40988796-3d52-4414-89dc-53c322b4d514\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zfbfz" Jul 10 16:54:52.462389 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.461600 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/04e15cd0-bcac-4375-97d5-b135b8ef9997-trusted-ca\") pod \"image-registry-657d95d548-bj62l\" (UID: \"04e15cd0-bcac-4375-97d5-b135b8ef9997\") " pod="openshift-image-registry/image-registry-657d95d548-bj62l" Jul 10 16:54:52.462389 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.461999 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef2eb691-dbbf-4cfc-9fc1-bba4cd71895b-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-d5bgt\" (UID: \"ef2eb691-dbbf-4cfc-9fc1-bba4cd71895b\") " pod="openshift-insights/insights-operator-68d9dd94cf-d5bgt" Jul 10 16:54:52.462389 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.462220 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/27742f15-b602-4554-84dc-ff9d46a9adda-trusted-ca\") pod \"console-operator-7866497494-5j4ws\" (UID: \"27742f15-b602-4554-84dc-ff9d46a9adda\") " pod="openshift-console-operator/console-operator-7866497494-5j4ws" Jul 10 16:54:52.462389 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:52.462333 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84-service-ca-bundle podName:3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84 nodeName:}" failed. No retries permitted until 2026-07-10 16:54:52.962315763 +0000 UTC m=+37.051644816 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84-service-ca-bundle") pod "router-default-bb9cb67d8-ddrbp" (UID: "3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84") : configmap references non-existent config key: service-ca.crt Jul 10 16:54:52.462681 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:52.462414 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 10 16:54:52.462681 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:52.462451 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0ba50ec-0b78-46ce-9b5c-e644af63989d-samples-operator-tls podName:c0ba50ec-0b78-46ce-9b5c-e644af63989d nodeName:}" failed. No retries permitted until 2026-07-10 16:54:52.962439166 +0000 UTC m=+37.051768194 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c0ba50ec-0b78-46ce-9b5c-e644af63989d-samples-operator-tls") pod "cluster-samples-operator-65db858894-fxc54" (UID: "c0ba50ec-0b78-46ce-9b5c-e644af63989d") : secret "samples-operator-tls" not found Jul 10 16:54:52.462681 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:52.462540 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 16:54:52.462681 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:52.462553 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-657d95d548-bj62l: secret "image-registry-tls" not found Jul 10 16:54:52.462681 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:52.462586 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/04e15cd0-bcac-4375-97d5-b135b8ef9997-registry-tls podName:04e15cd0-bcac-4375-97d5-b135b8ef9997 nodeName:}" failed. No retries permitted until 2026-07-10 16:54:52.962575709 +0000 UTC m=+37.051904740 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/04e15cd0-bcac-4375-97d5-b135b8ef9997-registry-tls") pod "image-registry-657d95d548-bj62l" (UID: "04e15cd0-bcac-4375-97d5-b135b8ef9997") : secret "image-registry-tls" not found Jul 10 16:54:52.462681 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.462580 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/04e15cd0-bcac-4375-97d5-b135b8ef9997-trusted-ca\") pod \"image-registry-657d95d548-bj62l\" (UID: \"04e15cd0-bcac-4375-97d5-b135b8ef9997\") " pod="openshift-image-registry/image-registry-657d95d548-bj62l" Jul 10 16:54:52.463320 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.463296 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/04e15cd0-bcac-4375-97d5-b135b8ef9997-registry-certificates\") pod \"image-registry-657d95d548-bj62l\" (UID: \"04e15cd0-bcac-4375-97d5-b135b8ef9997\") " pod="openshift-image-registry/image-registry-657d95d548-bj62l" Jul 10 16:54:52.463406 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.463307 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/73c4ea4d-fc8b-4fca-b4bd-2fc9102d4347-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-8r74k\" (UID: \"73c4ea4d-fc8b-4fca-b4bd-2fc9102d4347\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-8r74k" Jul 10 16:54:52.463406 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:52.463384 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 16:54:52.463532 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:52.463426 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40988796-3d52-4414-89dc-53c322b4d514-cluster-monitoring-operator-tls podName:40988796-3d52-4414-89dc-53c322b4d514 nodeName:}" failed. No retries permitted until 2026-07-10 16:54:52.963412567 +0000 UTC m=+37.052741597 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/40988796-3d52-4414-89dc-53c322b4d514-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-zfbfz" (UID: "40988796-3d52-4414-89dc-53c322b4d514") : secret "cluster-monitoring-operator-tls" not found Jul 10 16:54:52.464960 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.463857 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8vnr\" (UniqueName: \"kubernetes.io/projected/09ea20af-03b7-43ae-9f88-cf2054329229-kube-api-access-t8vnr\") pod \"network-check-source-869dcfcdcd-5nbcv\" (UID: \"09ea20af-03b7-43ae-9f88-cf2054329229\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-5nbcv" Jul 10 16:54:52.464960 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.463942 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/55d7dbb1-c8db-40fc-b416-7b781aae9aab-config-volume\") pod \"dns-default-g5nt8\" (UID: \"55d7dbb1-c8db-40fc-b416-7b781aae9aab\") " pod="openshift-dns/dns-default-g5nt8" Jul 10 16:54:52.464960 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.463973 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4sjdx\" (UniqueName: \"kubernetes.io/projected/55d7dbb1-c8db-40fc-b416-7b781aae9aab-kube-api-access-4sjdx\") pod \"dns-default-g5nt8\" (UID: \"55d7dbb1-c8db-40fc-b416-7b781aae9aab\") " pod="openshift-dns/dns-default-g5nt8" Jul 10 16:54:52.464960 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.464006 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ef2eb691-dbbf-4cfc-9fc1-bba4cd71895b-snapshots\") pod \"insights-operator-68d9dd94cf-d5bgt\" (UID: \"ef2eb691-dbbf-4cfc-9fc1-bba4cd71895b\") " pod="openshift-insights/insights-operator-68d9dd94cf-d5bgt" Jul 10 16:54:52.464960 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.464035 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/40988796-3d52-4414-89dc-53c322b4d514-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-zfbfz\" (UID: \"40988796-3d52-4414-89dc-53c322b4d514\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zfbfz" Jul 10 16:54:52.464960 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.464139 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef2eb691-dbbf-4cfc-9fc1-bba4cd71895b-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-d5bgt\" (UID: \"ef2eb691-dbbf-4cfc-9fc1-bba4cd71895b\") " pod="openshift-insights/insights-operator-68d9dd94cf-d5bgt" Jul 10 16:54:52.464960 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.464496 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ef2eb691-dbbf-4cfc-9fc1-bba4cd71895b-snapshots\") pod \"insights-operator-68d9dd94cf-d5bgt\" (UID: \"ef2eb691-dbbf-4cfc-9fc1-bba4cd71895b\") " pod="openshift-insights/insights-operator-68d9dd94cf-d5bgt" Jul 10 16:54:52.464960 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.464902 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/40988796-3d52-4414-89dc-53c322b4d514-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-zfbfz\" (UID: \"40988796-3d52-4414-89dc-53c322b4d514\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zfbfz" Jul 10 16:54:52.466316 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.466290 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/73c4ea4d-fc8b-4fca-b4bd-2fc9102d4347-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-8r74k\" (UID: \"73c4ea4d-fc8b-4fca-b4bd-2fc9102d4347\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-8r74k" Jul 10 16:54:52.466444 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.466299 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/04e15cd0-bcac-4375-97d5-b135b8ef9997-installation-pull-secrets\") pod \"image-registry-657d95d548-bj62l\" (UID: \"04e15cd0-bcac-4375-97d5-b135b8ef9997\") " pod="openshift-image-registry/image-registry-657d95d548-bj62l" Jul 10 16:54:52.466913 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.466890 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ef2eb691-dbbf-4cfc-9fc1-bba4cd71895b-serving-cert\") pod \"insights-operator-68d9dd94cf-d5bgt\" (UID: \"ef2eb691-dbbf-4cfc-9fc1-bba4cd71895b\") " pod="openshift-insights/insights-operator-68d9dd94cf-d5bgt" Jul 10 16:54:52.467522 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.467497 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84-stats-auth\") pod \"router-default-bb9cb67d8-ddrbp\" (UID: \"3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84\") " pod="openshift-ingress/router-default-bb9cb67d8-ddrbp" Jul 10 16:54:52.467817 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.467795 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84-default-certificate\") pod \"router-default-bb9cb67d8-ddrbp\" (UID: \"3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84\") " pod="openshift-ingress/router-default-bb9cb67d8-ddrbp" Jul 10 16:54:52.468404 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.468380 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/04e15cd0-bcac-4375-97d5-b135b8ef9997-image-registry-private-configuration\") pod \"image-registry-657d95d548-bj62l\" (UID: \"04e15cd0-bcac-4375-97d5-b135b8ef9997\") " pod="openshift-image-registry/image-registry-657d95d548-bj62l" Jul 10 16:54:52.468913 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.468892 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/27742f15-b602-4554-84dc-ff9d46a9adda-serving-cert\") pod \"console-operator-7866497494-5j4ws\" (UID: \"27742f15-b602-4554-84dc-ff9d46a9adda\") " pod="openshift-console-operator/console-operator-7866497494-5j4ws" Jul 10 16:54:52.469212 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.469188 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqcxv\" (UniqueName: \"kubernetes.io/projected/40988796-3d52-4414-89dc-53c322b4d514-kube-api-access-cqcxv\") pod \"cluster-monitoring-operator-5f7f494dd4-zfbfz\" (UID: \"40988796-3d52-4414-89dc-53c322b4d514\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zfbfz" Jul 10 16:54:52.471393 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.470791 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgc22\" (UniqueName: \"kubernetes.io/projected/c0ba50ec-0b78-46ce-9b5c-e644af63989d-kube-api-access-wgc22\") pod \"cluster-samples-operator-65db858894-fxc54\" (UID: \"c0ba50ec-0b78-46ce-9b5c-e644af63989d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fxc54" Jul 10 16:54:52.479127 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.473629 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dswfl\" (UniqueName: \"kubernetes.io/projected/27742f15-b602-4554-84dc-ff9d46a9adda-kube-api-access-dswfl\") pod \"console-operator-7866497494-5j4ws\" (UID: \"27742f15-b602-4554-84dc-ff9d46a9adda\") " pod="openshift-console-operator/console-operator-7866497494-5j4ws" Jul 10 16:54:52.479127 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.474556 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/04e15cd0-bcac-4375-97d5-b135b8ef9997-bound-sa-token\") pod \"image-registry-657d95d548-bj62l\" (UID: \"04e15cd0-bcac-4375-97d5-b135b8ef9997\") " pod="openshift-image-registry/image-registry-657d95d548-bj62l" Jul 10 16:54:52.479127 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.475062 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-54d4j\" (UniqueName: \"kubernetes.io/projected/ef2eb691-dbbf-4cfc-9fc1-bba4cd71895b-kube-api-access-54d4j\") pod \"insights-operator-68d9dd94cf-d5bgt\" (UID: \"ef2eb691-dbbf-4cfc-9fc1-bba4cd71895b\") " pod="openshift-insights/insights-operator-68d9dd94cf-d5bgt" Jul 10 16:54:52.479127 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.475702 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pr6jw\" (UniqueName: \"kubernetes.io/projected/3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84-kube-api-access-pr6jw\") pod \"router-default-bb9cb67d8-ddrbp\" (UID: \"3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84\") " pod="openshift-ingress/router-default-bb9cb67d8-ddrbp" Jul 10 16:54:52.479127 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.478963 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8zmk\" (UniqueName: \"kubernetes.io/projected/73c4ea4d-fc8b-4fca-b4bd-2fc9102d4347-kube-api-access-h8zmk\") pod \"kube-storage-version-migrator-operator-799cb7f958-8r74k\" (UID: \"73c4ea4d-fc8b-4fca-b4bd-2fc9102d4347\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-8r74k" Jul 10 16:54:52.479127 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.479078 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6xwk\" (UniqueName: \"kubernetes.io/projected/04e15cd0-bcac-4375-97d5-b135b8ef9997-kube-api-access-q6xwk\") pod \"image-registry-657d95d548-bj62l\" (UID: \"04e15cd0-bcac-4375-97d5-b135b8ef9997\") " pod="openshift-image-registry/image-registry-657d95d548-bj62l" Jul 10 16:54:52.565246 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.565202 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4sjdx\" (UniqueName: \"kubernetes.io/projected/55d7dbb1-c8db-40fc-b416-7b781aae9aab-kube-api-access-4sjdx\") pod \"dns-default-g5nt8\" (UID: \"55d7dbb1-c8db-40fc-b416-7b781aae9aab\") " pod="openshift-dns/dns-default-g5nt8" Jul 10 16:54:52.565246 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.565244 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/99905b79-72f4-478d-9c3a-05967f561ee2-config\") pod \"service-ca-operator-7695fc6c8c-gqqc7\" (UID: \"99905b79-72f4-478d-9c3a-05967f561ee2\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-gqqc7" Jul 10 16:54:52.565465 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.565342 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/99905b79-72f4-478d-9c3a-05967f561ee2-serving-cert\") pod \"service-ca-operator-7695fc6c8c-gqqc7\" (UID: \"99905b79-72f4-478d-9c3a-05967f561ee2\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-gqqc7" Jul 10 16:54:52.565465 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.565367 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xv96\" (UniqueName: \"kubernetes.io/projected/99905b79-72f4-478d-9c3a-05967f561ee2-kube-api-access-7xv96\") pod \"service-ca-operator-7695fc6c8c-gqqc7\" (UID: \"99905b79-72f4-478d-9c3a-05967f561ee2\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-gqqc7" Jul 10 16:54:52.565465 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.565405 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/55d7dbb1-c8db-40fc-b416-7b781aae9aab-metrics-tls\") pod \"dns-default-g5nt8\" (UID: \"55d7dbb1-c8db-40fc-b416-7b781aae9aab\") " pod="openshift-dns/dns-default-g5nt8" Jul 10 16:54:52.565465 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.565429 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6qp7\" (UniqueName: \"kubernetes.io/projected/633753a4-3340-4473-9639-4299c06b53a8-kube-api-access-b6qp7\") pod \"ingress-canary-hwfb8\" (UID: \"633753a4-3340-4473-9639-4299c06b53a8\") " pod="openshift-ingress-canary/ingress-canary-hwfb8" Jul 10 16:54:52.565465 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.565455 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55d7dbb1-c8db-40fc-b416-7b781aae9aab-tmp-dir\") pod \"dns-default-g5nt8\" (UID: \"55d7dbb1-c8db-40fc-b416-7b781aae9aab\") " pod="openshift-dns/dns-default-g5nt8" Jul 10 16:54:52.565704 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.565502 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/633753a4-3340-4473-9639-4299c06b53a8-cert\") pod \"ingress-canary-hwfb8\" (UID: \"633753a4-3340-4473-9639-4299c06b53a8\") " pod="openshift-ingress-canary/ingress-canary-hwfb8" Jul 10 16:54:52.565704 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.565547 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftbrt\" (UniqueName: \"kubernetes.io/projected/54cce7d7-9921-4a19-aaf7-d909fbfafa8f-kube-api-access-ftbrt\") pod \"volume-data-source-validator-78495597f6-xc54t\" (UID: \"54cce7d7-9921-4a19-aaf7-d909fbfafa8f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-xc54t" Jul 10 16:54:52.565704 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:52.565564 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 16:54:52.565704 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.565615 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8vnr\" (UniqueName: \"kubernetes.io/projected/09ea20af-03b7-43ae-9f88-cf2054329229-kube-api-access-t8vnr\") pod \"network-check-source-869dcfcdcd-5nbcv\" (UID: \"09ea20af-03b7-43ae-9f88-cf2054329229\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-5nbcv" Jul 10 16:54:52.565704 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:52.565633 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55d7dbb1-c8db-40fc-b416-7b781aae9aab-metrics-tls podName:55d7dbb1-c8db-40fc-b416-7b781aae9aab nodeName:}" failed. No retries permitted until 2026-07-10 16:54:53.065613816 +0000 UTC m=+37.154942847 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/55d7dbb1-c8db-40fc-b416-7b781aae9aab-metrics-tls") pod "dns-default-g5nt8" (UID: "55d7dbb1-c8db-40fc-b416-7b781aae9aab") : secret "dns-default-metrics-tls" not found Jul 10 16:54:52.565704 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.565678 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/55d7dbb1-c8db-40fc-b416-7b781aae9aab-config-volume\") pod \"dns-default-g5nt8\" (UID: \"55d7dbb1-c8db-40fc-b416-7b781aae9aab\") " pod="openshift-dns/dns-default-g5nt8" Jul 10 16:54:52.565987 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.565805 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55d7dbb1-c8db-40fc-b416-7b781aae9aab-tmp-dir\") pod \"dns-default-g5nt8\" (UID: \"55d7dbb1-c8db-40fc-b416-7b781aae9aab\") " pod="openshift-dns/dns-default-g5nt8" Jul 10 16:54:52.565987 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.565903 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/99905b79-72f4-478d-9c3a-05967f561ee2-config\") pod \"service-ca-operator-7695fc6c8c-gqqc7\" (UID: \"99905b79-72f4-478d-9c3a-05967f561ee2\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-gqqc7" Jul 10 16:54:52.566339 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.566290 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/55d7dbb1-c8db-40fc-b416-7b781aae9aab-config-volume\") pod \"dns-default-g5nt8\" (UID: \"55d7dbb1-c8db-40fc-b416-7b781aae9aab\") " pod="openshift-dns/dns-default-g5nt8" Jul 10 16:54:52.568180 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.568151 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/99905b79-72f4-478d-9c3a-05967f561ee2-serving-cert\") pod \"service-ca-operator-7695fc6c8c-gqqc7\" (UID: \"99905b79-72f4-478d-9c3a-05967f561ee2\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-gqqc7" Jul 10 16:54:52.568345 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.568324 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/633753a4-3340-4473-9639-4299c06b53a8-cert\") pod \"ingress-canary-hwfb8\" (UID: \"633753a4-3340-4473-9639-4299c06b53a8\") " pod="openshift-ingress-canary/ingress-canary-hwfb8" Jul 10 16:54:52.577461 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.577437 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4sjdx\" (UniqueName: \"kubernetes.io/projected/55d7dbb1-c8db-40fc-b416-7b781aae9aab-kube-api-access-4sjdx\") pod \"dns-default-g5nt8\" (UID: \"55d7dbb1-c8db-40fc-b416-7b781aae9aab\") " pod="openshift-dns/dns-default-g5nt8" Jul 10 16:54:52.577742 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.577716 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftbrt\" (UniqueName: \"kubernetes.io/projected/54cce7d7-9921-4a19-aaf7-d909fbfafa8f-kube-api-access-ftbrt\") pod \"volume-data-source-validator-78495597f6-xc54t\" (UID: \"54cce7d7-9921-4a19-aaf7-d909fbfafa8f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-xc54t" Jul 10 16:54:52.578824 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.578801 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xv96\" (UniqueName: \"kubernetes.io/projected/99905b79-72f4-478d-9c3a-05967f561ee2-kube-api-access-7xv96\") pod \"service-ca-operator-7695fc6c8c-gqqc7\" (UID: \"99905b79-72f4-478d-9c3a-05967f561ee2\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-gqqc7" Jul 10 16:54:52.578932 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.578910 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6qp7\" (UniqueName: \"kubernetes.io/projected/633753a4-3340-4473-9639-4299c06b53a8-kube-api-access-b6qp7\") pod \"ingress-canary-hwfb8\" (UID: \"633753a4-3340-4473-9639-4299c06b53a8\") " pod="openshift-ingress-canary/ingress-canary-hwfb8" Jul 10 16:54:52.579574 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.579553 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8vnr\" (UniqueName: \"kubernetes.io/projected/09ea20af-03b7-43ae-9f88-cf2054329229-kube-api-access-t8vnr\") pod \"network-check-source-869dcfcdcd-5nbcv\" (UID: \"09ea20af-03b7-43ae-9f88-cf2054329229\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-5nbcv" Jul 10 16:54:52.634494 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.634244 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-d5bgt" Jul 10 16:54:52.643471 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.643446 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-5j4ws" Jul 10 16:54:52.650682 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.650662 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-8r74k" Jul 10 16:54:52.666350 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.666325 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-xc54t" Jul 10 16:54:52.674794 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.674739 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-gqqc7" Jul 10 16:54:52.682811 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.682792 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-5nbcv" Jul 10 16:54:52.688903 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.688883 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hwfb8" Jul 10 16:54:52.968891 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.968801 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84-service-ca-bundle\") pod \"router-default-bb9cb67d8-ddrbp\" (UID: \"3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84\") " pod="openshift-ingress/router-default-bb9cb67d8-ddrbp" Jul 10 16:54:52.968891 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.968849 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c0ba50ec-0b78-46ce-9b5c-e644af63989d-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-fxc54\" (UID: \"c0ba50ec-0b78-46ce-9b5c-e644af63989d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fxc54" Jul 10 16:54:52.968891 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.968877 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/04e15cd0-bcac-4375-97d5-b135b8ef9997-registry-tls\") pod \"image-registry-657d95d548-bj62l\" (UID: \"04e15cd0-bcac-4375-97d5-b135b8ef9997\") " pod="openshift-image-registry/image-registry-657d95d548-bj62l" Jul 10 16:54:52.969196 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.968917 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/40988796-3d52-4414-89dc-53c322b4d514-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-zfbfz\" (UID: \"40988796-3d52-4414-89dc-53c322b4d514\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zfbfz" Jul 10 16:54:52.969196 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:52.968997 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 10 16:54:52.969196 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:52.969016 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84-service-ca-bundle podName:3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84 nodeName:}" failed. No retries permitted until 2026-07-10 16:54:53.968994837 +0000 UTC m=+38.058323872 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84-service-ca-bundle") pod "router-default-bb9cb67d8-ddrbp" (UID: "3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84") : configmap references non-existent config key: service-ca.crt Jul 10 16:54:52.969196 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:52.969062 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0ba50ec-0b78-46ce-9b5c-e644af63989d-samples-operator-tls podName:c0ba50ec-0b78-46ce-9b5c-e644af63989d nodeName:}" failed. No retries permitted until 2026-07-10 16:54:53.969034677 +0000 UTC m=+38.058363707 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c0ba50ec-0b78-46ce-9b5c-e644af63989d-samples-operator-tls") pod "cluster-samples-operator-65db858894-fxc54" (UID: "c0ba50ec-0b78-46ce-9b5c-e644af63989d") : secret "samples-operator-tls" not found Jul 10 16:54:52.969196 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:52.968998 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84-metrics-certs\") pod \"router-default-bb9cb67d8-ddrbp\" (UID: \"3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84\") " pod="openshift-ingress/router-default-bb9cb67d8-ddrbp" Jul 10 16:54:52.969196 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:52.969100 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 16:54:52.969196 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:52.969151 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84-metrics-certs podName:3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84 nodeName:}" failed. No retries permitted until 2026-07-10 16:54:53.96913608 +0000 UTC m=+38.058465121 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84-metrics-certs") pod "router-default-bb9cb67d8-ddrbp" (UID: "3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84") : secret "router-metrics-certs-default" not found Jul 10 16:54:52.969196 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:52.969187 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 16:54:52.969196 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:52.969196 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 16:54:52.969514 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:52.969207 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-657d95d548-bj62l: secret "image-registry-tls" not found Jul 10 16:54:52.969514 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:52.969238 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40988796-3d52-4414-89dc-53c322b4d514-cluster-monitoring-operator-tls podName:40988796-3d52-4414-89dc-53c322b4d514 nodeName:}" failed. No retries permitted until 2026-07-10 16:54:53.969228042 +0000 UTC m=+38.058557073 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/40988796-3d52-4414-89dc-53c322b4d514-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-zfbfz" (UID: "40988796-3d52-4414-89dc-53c322b4d514") : secret "cluster-monitoring-operator-tls" not found Jul 10 16:54:52.969514 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:52.969262 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/04e15cd0-bcac-4375-97d5-b135b8ef9997-registry-tls podName:04e15cd0-bcac-4375-97d5-b135b8ef9997 nodeName:}" failed. No retries permitted until 2026-07-10 16:54:53.96924646 +0000 UTC m=+38.058575492 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/04e15cd0-bcac-4375-97d5-b135b8ef9997-registry-tls") pod "image-registry-657d95d548-bj62l" (UID: "04e15cd0-bcac-4375-97d5-b135b8ef9997") : secret "image-registry-tls" not found Jul 10 16:54:53.069745 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:53.069708 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/55d7dbb1-c8db-40fc-b416-7b781aae9aab-metrics-tls\") pod \"dns-default-g5nt8\" (UID: \"55d7dbb1-c8db-40fc-b416-7b781aae9aab\") " pod="openshift-dns/dns-default-g5nt8" Jul 10 16:54:53.069920 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:53.069898 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 16:54:53.069983 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:53.069972 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55d7dbb1-c8db-40fc-b416-7b781aae9aab-metrics-tls podName:55d7dbb1-c8db-40fc-b416-7b781aae9aab nodeName:}" failed. No retries permitted until 2026-07-10 16:54:54.069947264 +0000 UTC m=+38.159276291 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/55d7dbb1-c8db-40fc-b416-7b781aae9aab-metrics-tls") pod "dns-default-g5nt8" (UID: "55d7dbb1-c8db-40fc-b416-7b781aae9aab") : secret "dns-default-metrics-tls" not found Jul 10 16:54:53.420312 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:53.419579 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jvkw6" Jul 10 16:54:53.430733 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:53.429732 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jn8d4\"" Jul 10 16:54:53.505546 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:53.505512 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hwfb8"] Jul 10 16:54:53.510116 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:53.510088 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-gqqc7"] Jul 10 16:54:53.510753 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:53.510727 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod633753a4_3340_4473_9639_4299c06b53a8.slice/crio-f70da7c7d68c64864652970bd3d8d05c12921fd398ba4e34e7e6e7d0e24cf40f WatchSource:0}: Error finding container f70da7c7d68c64864652970bd3d8d05c12921fd398ba4e34e7e6e7d0e24cf40f: Status 404 returned error can't find the container with id f70da7c7d68c64864652970bd3d8d05c12921fd398ba4e34e7e6e7d0e24cf40f Jul 10 16:54:53.515113 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:53.515035 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-5nbcv"] Jul 10 16:54:53.518032 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:53.518012 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-8r74k"] Jul 10 16:54:53.518134 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:53.518078 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-xc54t"] Jul 10 16:54:53.521537 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:53.519466 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73c4ea4d_fc8b_4fca_b4bd_2fc9102d4347.slice/crio-9e85b45a109c6ed28bbc346d0bb4cc346d776ac6dfbd1c91020cdd9248ca85df WatchSource:0}: Error finding container 9e85b45a109c6ed28bbc346d0bb4cc346d776ac6dfbd1c91020cdd9248ca85df: Status 404 returned error can't find the container with id 9e85b45a109c6ed28bbc346d0bb4cc346d776ac6dfbd1c91020cdd9248ca85df Jul 10 16:54:53.525857 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:53.525747 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-d5bgt"] Jul 10 16:54:53.531788 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:53.531762 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef2eb691_dbbf_4cfc_9fc1_bba4cd71895b.slice/crio-8268ae0a93e2a05958c6b3ecafac5ca68b4d4b095e961dd4cb2f99c38fdb7858 WatchSource:0}: Error finding container 8268ae0a93e2a05958c6b3ecafac5ca68b4d4b095e961dd4cb2f99c38fdb7858: Status 404 returned error can't find the container with id 8268ae0a93e2a05958c6b3ecafac5ca68b4d4b095e961dd4cb2f99c38fdb7858 Jul 10 16:54:53.538397 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:53.538333 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-5j4ws"] Jul 10 16:54:53.542901 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:54:53.542875 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27742f15_b602_4554_84dc_ff9d46a9adda.slice/crio-4056010de3fb35fe03ea11e2b4664cca95e96e1642dae6090899f8ead34e6cdd WatchSource:0}: Error finding container 4056010de3fb35fe03ea11e2b4664cca95e96e1642dae6090899f8ead34e6cdd: Status 404 returned error can't find the container with id 4056010de3fb35fe03ea11e2b4664cca95e96e1642dae6090899f8ead34e6cdd Jul 10 16:54:53.730418 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:53.730377 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hwfb8" event={"ID":"633753a4-3340-4473-9639-4299c06b53a8","Type":"ContainerStarted","Data":"f70da7c7d68c64864652970bd3d8d05c12921fd398ba4e34e7e6e7d0e24cf40f"} Jul 10 16:54:53.736803 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:53.736770 2578 generic.go:356] "Generic (PLEG): container finished" podID="6e1806e2-a453-4929-8632-267349af86c7" containerID="20d69b11dc5c1047cdf9aa97db06e80c49a7a1142c22741fdad3716c188f4c84" exitCode=0 Jul 10 16:54:53.736987 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:53.736861 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5d4nh" event={"ID":"6e1806e2-a453-4929-8632-267349af86c7","Type":"ContainerDied","Data":"20d69b11dc5c1047cdf9aa97db06e80c49a7a1142c22741fdad3716c188f4c84"} Jul 10 16:54:53.738097 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:53.738065 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-xc54t" event={"ID":"54cce7d7-9921-4a19-aaf7-d909fbfafa8f","Type":"ContainerStarted","Data":"d022cc420010a910aa8d1c9ac44ed024b4797d02194a218861591102f1f387b0"} Jul 10 16:54:53.739144 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:53.739122 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-gqqc7" event={"ID":"99905b79-72f4-478d-9c3a-05967f561ee2","Type":"ContainerStarted","Data":"e6f422f8bb2bd98dd32be109082454a57c5c6528d09fd6e0025234239e4ede95"} Jul 10 16:54:53.740106 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:53.740081 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-5nbcv" event={"ID":"09ea20af-03b7-43ae-9f88-cf2054329229","Type":"ContainerStarted","Data":"740ef66127cffbb5c085ef5768839ee3baa41459ea632ce26e3095baed6f3092"} Jul 10 16:54:53.740974 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:53.740952 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-d5bgt" event={"ID":"ef2eb691-dbbf-4cfc-9fc1-bba4cd71895b","Type":"ContainerStarted","Data":"8268ae0a93e2a05958c6b3ecafac5ca68b4d4b095e961dd4cb2f99c38fdb7858"} Jul 10 16:54:53.742283 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:53.742257 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-5j4ws" event={"ID":"27742f15-b602-4554-84dc-ff9d46a9adda","Type":"ContainerStarted","Data":"4056010de3fb35fe03ea11e2b4664cca95e96e1642dae6090899f8ead34e6cdd"} Jul 10 16:54:53.743496 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:53.743476 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-8r74k" event={"ID":"73c4ea4d-fc8b-4fca-b4bd-2fc9102d4347","Type":"ContainerStarted","Data":"9e85b45a109c6ed28bbc346d0bb4cc346d776ac6dfbd1c91020cdd9248ca85df"} Jul 10 16:54:53.980356 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:53.980152 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84-service-ca-bundle\") pod \"router-default-bb9cb67d8-ddrbp\" (UID: \"3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84\") " pod="openshift-ingress/router-default-bb9cb67d8-ddrbp" Jul 10 16:54:53.980521 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:53.980361 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84-service-ca-bundle podName:3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84 nodeName:}" failed. No retries permitted until 2026-07-10 16:54:55.98033718 +0000 UTC m=+40.069666210 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84-service-ca-bundle") pod "router-default-bb9cb67d8-ddrbp" (UID: "3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84") : configmap references non-existent config key: service-ca.crt Jul 10 16:54:53.980521 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:53.980408 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c0ba50ec-0b78-46ce-9b5c-e644af63989d-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-fxc54\" (UID: \"c0ba50ec-0b78-46ce-9b5c-e644af63989d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fxc54" Jul 10 16:54:53.980521 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:53.980446 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/04e15cd0-bcac-4375-97d5-b135b8ef9997-registry-tls\") pod \"image-registry-657d95d548-bj62l\" (UID: \"04e15cd0-bcac-4375-97d5-b135b8ef9997\") " pod="openshift-image-registry/image-registry-657d95d548-bj62l" Jul 10 16:54:53.980521 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:53.980494 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/40988796-3d52-4414-89dc-53c322b4d514-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-zfbfz\" (UID: \"40988796-3d52-4414-89dc-53c322b4d514\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zfbfz" Jul 10 16:54:53.980693 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:53.980556 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 10 16:54:53.980693 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:53.980574 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 16:54:53.980693 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:53.980584 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 16:54:53.980693 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:53.980604 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-657d95d548-bj62l: secret "image-registry-tls" not found Jul 10 16:54:53.980693 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:53.980615 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40988796-3d52-4414-89dc-53c322b4d514-cluster-monitoring-operator-tls podName:40988796-3d52-4414-89dc-53c322b4d514 nodeName:}" failed. No retries permitted until 2026-07-10 16:54:55.980602146 +0000 UTC m=+40.069931175 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/40988796-3d52-4414-89dc-53c322b4d514-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-zfbfz" (UID: "40988796-3d52-4414-89dc-53c322b4d514") : secret "cluster-monitoring-operator-tls" not found Jul 10 16:54:53.980693 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:53.980633 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0ba50ec-0b78-46ce-9b5c-e644af63989d-samples-operator-tls podName:c0ba50ec-0b78-46ce-9b5c-e644af63989d nodeName:}" failed. No retries permitted until 2026-07-10 16:54:55.980623007 +0000 UTC m=+40.069952036 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c0ba50ec-0b78-46ce-9b5c-e644af63989d-samples-operator-tls") pod "cluster-samples-operator-65db858894-fxc54" (UID: "c0ba50ec-0b78-46ce-9b5c-e644af63989d") : secret "samples-operator-tls" not found Jul 10 16:54:53.980693 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:53.980657 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/04e15cd0-bcac-4375-97d5-b135b8ef9997-registry-tls podName:04e15cd0-bcac-4375-97d5-b135b8ef9997 nodeName:}" failed. No retries permitted until 2026-07-10 16:54:55.980639036 +0000 UTC m=+40.069968070 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/04e15cd0-bcac-4375-97d5-b135b8ef9997-registry-tls") pod "image-registry-657d95d548-bj62l" (UID: "04e15cd0-bcac-4375-97d5-b135b8ef9997") : secret "image-registry-tls" not found Jul 10 16:54:53.980945 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:53.980747 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84-metrics-certs\") pod \"router-default-bb9cb67d8-ddrbp\" (UID: \"3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84\") " pod="openshift-ingress/router-default-bb9cb67d8-ddrbp" Jul 10 16:54:53.980945 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:53.980851 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 16:54:53.980945 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:53.980897 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84-metrics-certs podName:3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84 nodeName:}" failed. No retries permitted until 2026-07-10 16:54:55.980883274 +0000 UTC m=+40.070212303 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84-metrics-certs") pod "router-default-bb9cb67d8-ddrbp" (UID: "3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84") : secret "router-metrics-certs-default" not found Jul 10 16:54:54.081561 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:54.081514 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/55d7dbb1-c8db-40fc-b416-7b781aae9aab-metrics-tls\") pod \"dns-default-g5nt8\" (UID: \"55d7dbb1-c8db-40fc-b416-7b781aae9aab\") " pod="openshift-dns/dns-default-g5nt8" Jul 10 16:54:54.081736 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:54.081714 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 16:54:54.081797 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:54.081781 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55d7dbb1-c8db-40fc-b416-7b781aae9aab-metrics-tls podName:55d7dbb1-c8db-40fc-b416-7b781aae9aab nodeName:}" failed. No retries permitted until 2026-07-10 16:54:56.08176119 +0000 UTC m=+40.171090218 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/55d7dbb1-c8db-40fc-b416-7b781aae9aab-metrics-tls") pod "dns-default-g5nt8" (UID: "55d7dbb1-c8db-40fc-b416-7b781aae9aab") : secret "dns-default-metrics-tls" not found Jul 10 16:54:54.758467 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:54.757196 2578 generic.go:356] "Generic (PLEG): container finished" podID="6e1806e2-a453-4929-8632-267349af86c7" containerID="9566855d079938047a6fd4d7b68c40aa9fc97607dfbeced92f6f036ec31fd1e0" exitCode=0 Jul 10 16:54:54.758467 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:54.757283 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5d4nh" event={"ID":"6e1806e2-a453-4929-8632-267349af86c7","Type":"ContainerDied","Data":"9566855d079938047a6fd4d7b68c40aa9fc97607dfbeced92f6f036ec31fd1e0"} Jul 10 16:54:55.764982 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:55.764941 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5d4nh" event={"ID":"6e1806e2-a453-4929-8632-267349af86c7","Type":"ContainerStarted","Data":"bd3ef53ec37ac40b2dde8bb6451f50930def1880a6371e7460e421b2c1df46c0"} Jul 10 16:54:55.791562 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:55.791501 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-5d4nh" podStartSLOduration=7.030465124 podStartE2EDuration="39.791479004s" totalImagesPullingTime="32.76101388s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 16:54:16 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:54:55.789765549 +0000 UTC m=+39.879094613" watchObservedRunningTime="2026-07-10 16:54:55.791479004 +0000 UTC m=+39.880808055" Jul 10 16:54:56.006141 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:56.006099 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84-metrics-certs\") pod \"router-default-bb9cb67d8-ddrbp\" (UID: \"3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84\") " pod="openshift-ingress/router-default-bb9cb67d8-ddrbp" Jul 10 16:54:56.006324 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:56.006190 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84-service-ca-bundle\") pod \"router-default-bb9cb67d8-ddrbp\" (UID: \"3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84\") " pod="openshift-ingress/router-default-bb9cb67d8-ddrbp" Jul 10 16:54:56.006324 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:56.006220 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c0ba50ec-0b78-46ce-9b5c-e644af63989d-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-fxc54\" (UID: \"c0ba50ec-0b78-46ce-9b5c-e644af63989d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fxc54" Jul 10 16:54:56.006324 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:56.006256 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/04e15cd0-bcac-4375-97d5-b135b8ef9997-registry-tls\") pod \"image-registry-657d95d548-bj62l\" (UID: \"04e15cd0-bcac-4375-97d5-b135b8ef9997\") " pod="openshift-image-registry/image-registry-657d95d548-bj62l" Jul 10 16:54:56.006324 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:56.006299 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/40988796-3d52-4414-89dc-53c322b4d514-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-zfbfz\" (UID: \"40988796-3d52-4414-89dc-53c322b4d514\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zfbfz" Jul 10 16:54:56.006485 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:56.006452 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 16:54:56.006523 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:56.006516 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40988796-3d52-4414-89dc-53c322b4d514-cluster-monitoring-operator-tls podName:40988796-3d52-4414-89dc-53c322b4d514 nodeName:}" failed. No retries permitted until 2026-07-10 16:55:00.006496323 +0000 UTC m=+44.095825357 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/40988796-3d52-4414-89dc-53c322b4d514-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-zfbfz" (UID: "40988796-3d52-4414-89dc-53c322b4d514") : secret "cluster-monitoring-operator-tls" not found Jul 10 16:54:56.006594 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:56.006581 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 16:54:56.006630 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:56.006619 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84-metrics-certs podName:3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84 nodeName:}" failed. No retries permitted until 2026-07-10 16:55:00.006606693 +0000 UTC m=+44.095935724 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84-metrics-certs") pod "router-default-bb9cb67d8-ddrbp" (UID: "3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84") : secret "router-metrics-certs-default" not found Jul 10 16:54:56.006702 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:56.006690 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84-service-ca-bundle podName:3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84 nodeName:}" failed. No retries permitted until 2026-07-10 16:55:00.006675644 +0000 UTC m=+44.096004678 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84-service-ca-bundle") pod "router-default-bb9cb67d8-ddrbp" (UID: "3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84") : configmap references non-existent config key: service-ca.crt Jul 10 16:54:56.007401 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:56.006757 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 10 16:54:56.007401 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:56.007326 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0ba50ec-0b78-46ce-9b5c-e644af63989d-samples-operator-tls podName:c0ba50ec-0b78-46ce-9b5c-e644af63989d nodeName:}" failed. No retries permitted until 2026-07-10 16:55:00.007305926 +0000 UTC m=+44.096634957 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c0ba50ec-0b78-46ce-9b5c-e644af63989d-samples-operator-tls") pod "cluster-samples-operator-65db858894-fxc54" (UID: "c0ba50ec-0b78-46ce-9b5c-e644af63989d") : secret "samples-operator-tls" not found Jul 10 16:54:56.007401 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:56.006852 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 16:54:56.007401 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:56.007349 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-657d95d548-bj62l: secret "image-registry-tls" not found Jul 10 16:54:56.007401 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:56.007381 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/04e15cd0-bcac-4375-97d5-b135b8ef9997-registry-tls podName:04e15cd0-bcac-4375-97d5-b135b8ef9997 nodeName:}" failed. No retries permitted until 2026-07-10 16:55:00.007370486 +0000 UTC m=+44.096699514 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/04e15cd0-bcac-4375-97d5-b135b8ef9997-registry-tls") pod "image-registry-657d95d548-bj62l" (UID: "04e15cd0-bcac-4375-97d5-b135b8ef9997") : secret "image-registry-tls" not found Jul 10 16:54:56.107838 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:54:56.107798 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/55d7dbb1-c8db-40fc-b416-7b781aae9aab-metrics-tls\") pod \"dns-default-g5nt8\" (UID: \"55d7dbb1-c8db-40fc-b416-7b781aae9aab\") " pod="openshift-dns/dns-default-g5nt8" Jul 10 16:54:56.107999 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:56.107962 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 16:54:56.108126 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:54:56.108029 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55d7dbb1-c8db-40fc-b416-7b781aae9aab-metrics-tls podName:55d7dbb1-c8db-40fc-b416-7b781aae9aab nodeName:}" failed. No retries permitted until 2026-07-10 16:55:00.108012725 +0000 UTC m=+44.197341753 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/55d7dbb1-c8db-40fc-b416-7b781aae9aab-metrics-tls") pod "dns-default-g5nt8" (UID: "55d7dbb1-c8db-40fc-b416-7b781aae9aab") : secret "dns-default-metrics-tls" not found Jul 10 16:55:00.041378 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:00.041327 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84-metrics-certs\") pod \"router-default-bb9cb67d8-ddrbp\" (UID: \"3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84\") " pod="openshift-ingress/router-default-bb9cb67d8-ddrbp" Jul 10 16:55:00.041989 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:00.041407 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84-service-ca-bundle\") pod \"router-default-bb9cb67d8-ddrbp\" (UID: \"3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84\") " pod="openshift-ingress/router-default-bb9cb67d8-ddrbp" Jul 10 16:55:00.041989 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:00.041432 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c0ba50ec-0b78-46ce-9b5c-e644af63989d-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-fxc54\" (UID: \"c0ba50ec-0b78-46ce-9b5c-e644af63989d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fxc54" Jul 10 16:55:00.041989 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:00.041464 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/04e15cd0-bcac-4375-97d5-b135b8ef9997-registry-tls\") pod \"image-registry-657d95d548-bj62l\" (UID: \"04e15cd0-bcac-4375-97d5-b135b8ef9997\") " pod="openshift-image-registry/image-registry-657d95d548-bj62l" Jul 10 16:55:00.041989 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:55:00.041480 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 16:55:00.041989 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:00.041536 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/40988796-3d52-4414-89dc-53c322b4d514-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-zfbfz\" (UID: \"40988796-3d52-4414-89dc-53c322b4d514\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zfbfz" Jul 10 16:55:00.041989 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:55:00.041579 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84-service-ca-bundle podName:3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84 nodeName:}" failed. No retries permitted until 2026-07-10 16:55:08.041560619 +0000 UTC m=+52.130889651 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84-service-ca-bundle") pod "router-default-bb9cb67d8-ddrbp" (UID: "3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84") : configmap references non-existent config key: service-ca.crt Jul 10 16:55:00.041989 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:55:00.041595 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 10 16:55:00.041989 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:55:00.041605 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84-metrics-certs podName:3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84 nodeName:}" failed. No retries permitted until 2026-07-10 16:55:08.041596408 +0000 UTC m=+52.130925437 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84-metrics-certs") pod "router-default-bb9cb67d8-ddrbp" (UID: "3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84") : secret "router-metrics-certs-default" not found Jul 10 16:55:00.041989 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:55:00.041643 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 16:55:00.041989 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:55:00.041658 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 16:55:00.041989 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:55:00.041659 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-657d95d548-bj62l: secret "image-registry-tls" not found Jul 10 16:55:00.041989 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:55:00.041670 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0ba50ec-0b78-46ce-9b5c-e644af63989d-samples-operator-tls podName:c0ba50ec-0b78-46ce-9b5c-e644af63989d nodeName:}" failed. No retries permitted until 2026-07-10 16:55:08.041650941 +0000 UTC m=+52.130979985 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c0ba50ec-0b78-46ce-9b5c-e644af63989d-samples-operator-tls") pod "cluster-samples-operator-65db858894-fxc54" (UID: "c0ba50ec-0b78-46ce-9b5c-e644af63989d") : secret "samples-operator-tls" not found Jul 10 16:55:00.041989 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:55:00.041684 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40988796-3d52-4414-89dc-53c322b4d514-cluster-monitoring-operator-tls podName:40988796-3d52-4414-89dc-53c322b4d514 nodeName:}" failed. No retries permitted until 2026-07-10 16:55:08.041677592 +0000 UTC m=+52.131006620 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/40988796-3d52-4414-89dc-53c322b4d514-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-zfbfz" (UID: "40988796-3d52-4414-89dc-53c322b4d514") : secret "cluster-monitoring-operator-tls" not found Jul 10 16:55:00.041989 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:55:00.041703 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/04e15cd0-bcac-4375-97d5-b135b8ef9997-registry-tls podName:04e15cd0-bcac-4375-97d5-b135b8ef9997 nodeName:}" failed. No retries permitted until 2026-07-10 16:55:08.041688642 +0000 UTC m=+52.131017685 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/04e15cd0-bcac-4375-97d5-b135b8ef9997-registry-tls") pod "image-registry-657d95d548-bj62l" (UID: "04e15cd0-bcac-4375-97d5-b135b8ef9997") : secret "image-registry-tls" not found Jul 10 16:55:00.142841 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:00.142799 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/55d7dbb1-c8db-40fc-b416-7b781aae9aab-metrics-tls\") pod \"dns-default-g5nt8\" (UID: \"55d7dbb1-c8db-40fc-b416-7b781aae9aab\") " pod="openshift-dns/dns-default-g5nt8" Jul 10 16:55:00.143033 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:55:00.143018 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 16:55:00.143116 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:55:00.143091 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55d7dbb1-c8db-40fc-b416-7b781aae9aab-metrics-tls podName:55d7dbb1-c8db-40fc-b416-7b781aae9aab nodeName:}" failed. No retries permitted until 2026-07-10 16:55:08.143077689 +0000 UTC m=+52.232406717 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/55d7dbb1-c8db-40fc-b416-7b781aae9aab-metrics-tls") pod "dns-default-g5nt8" (UID: "55d7dbb1-c8db-40fc-b416-7b781aae9aab") : secret "dns-default-metrics-tls" not found Jul 10 16:55:01.785040 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:01.784988 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hwfb8" event={"ID":"633753a4-3340-4473-9639-4299c06b53a8","Type":"ContainerStarted","Data":"be1e85697e8cfaaa4d132e71ffe4c624785b2c40ce19f8a1d423a1e357351b08"} Jul 10 16:55:01.787649 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:01.787130 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-xc54t" event={"ID":"54cce7d7-9921-4a19-aaf7-d909fbfafa8f","Type":"ContainerStarted","Data":"6f6649a3462c1672c4665cd19876e45f202a90951e88345e28f3746166e14e5b"} Jul 10 16:55:01.789248 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:01.788792 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-gqqc7" event={"ID":"99905b79-72f4-478d-9c3a-05967f561ee2","Type":"ContainerStarted","Data":"8908e12948680a50a9b9964d60704482611fdecf428dfe6a760b1e2497f78404"} Jul 10 16:55:01.790800 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:01.790776 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-5nbcv" event={"ID":"09ea20af-03b7-43ae-9f88-cf2054329229","Type":"ContainerStarted","Data":"309a35cc431aacf3372336a03ea164f7224d8187d0558bd8ba3f4acf51106761"} Jul 10 16:55:01.792593 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:01.792571 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-d5bgt" event={"ID":"ef2eb691-dbbf-4cfc-9fc1-bba4cd71895b","Type":"ContainerStarted","Data":"c57d2934a4aa2732d024f901cb72d3b8e2e516e6e7718ad5793df227bb80818b"} Jul 10 16:55:01.794541 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:01.794518 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-5j4ws_27742f15-b602-4554-84dc-ff9d46a9adda/console-operator/0.log" Jul 10 16:55:01.794642 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:01.794555 2578 generic.go:356] "Generic (PLEG): container finished" podID="27742f15-b602-4554-84dc-ff9d46a9adda" containerID="6a60497e48f5753343fc561e7ebf2f749866528ac1ea3dd1ba93e745d4cc63f4" exitCode=255 Jul 10 16:55:01.794642 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:01.794629 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-5j4ws" event={"ID":"27742f15-b602-4554-84dc-ff9d46a9adda","Type":"ContainerDied","Data":"6a60497e48f5753343fc561e7ebf2f749866528ac1ea3dd1ba93e745d4cc63f4"} Jul 10 16:55:01.794880 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:01.794866 2578 scope.go:117] "RemoveContainer" containerID="6a60497e48f5753343fc561e7ebf2f749866528ac1ea3dd1ba93e745d4cc63f4" Jul 10 16:55:01.798799 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:01.798774 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-8r74k" event={"ID":"73c4ea4d-fc8b-4fca-b4bd-2fc9102d4347","Type":"ContainerStarted","Data":"3e2bdf4d4a69716d1609be8ee0a8ed17c97319d1cccfe84673313904ece865d4"} Jul 10 16:55:01.803073 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:01.802820 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-hwfb8" podStartSLOduration=1.9481046 podStartE2EDuration="9.802801872s" totalImagesPullingTime="7.854697272s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:54:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:55:01.801957651 +0000 UTC m=+45.891286700" watchObservedRunningTime="2026-07-10 16:55:01.802801872 +0000 UTC m=+45.892130902" Jul 10 16:55:01.819732 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:01.819673 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-d5bgt" podStartSLOduration=13.978484141 podStartE2EDuration="21.819651153s" totalImagesPullingTime="7.841167012s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:54:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:55:01.818402578 +0000 UTC m=+45.907731628" watchObservedRunningTime="2026-07-10 16:55:01.819651153 +0000 UTC m=+45.908980203" Jul 10 16:55:01.853281 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:01.853225 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-8r74k" podStartSLOduration=14.002851327 podStartE2EDuration="21.853202983s" totalImagesPullingTime="7.850351656s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:54:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:55:01.852007935 +0000 UTC m=+45.941336987" watchObservedRunningTime="2026-07-10 16:55:01.853202983 +0000 UTC m=+45.942532033" Jul 10 16:55:01.871628 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:01.871524 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-xc54t" podStartSLOduration=14.151194797 podStartE2EDuration="21.871503242s" totalImagesPullingTime="7.720308445s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:54:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:55:01.870031278 +0000 UTC m=+45.959360330" watchObservedRunningTime="2026-07-10 16:55:01.871503242 +0000 UTC m=+45.960832293" Jul 10 16:55:01.893726 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:01.893669 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-gqqc7" podStartSLOduration=12.039454218 podStartE2EDuration="19.893649219s" totalImagesPullingTime="7.854195001s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:54:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:55:01.890804152 +0000 UTC m=+45.980133198" watchObservedRunningTime="2026-07-10 16:55:01.893649219 +0000 UTC m=+45.982978271" Jul 10 16:55:01.912448 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:01.912393 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-5nbcv" podStartSLOduration=11.055656271 podStartE2EDuration="18.912371111s" totalImagesPullingTime="7.85671484s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:54:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:55:01.910902514 +0000 UTC m=+46.000231564" watchObservedRunningTime="2026-07-10 16:55:01.912371111 +0000 UTC m=+46.001700164" Jul 10 16:55:02.644154 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:02.644115 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-5j4ws" Jul 10 16:55:02.644292 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:02.644169 2578 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-5j4ws" Jul 10 16:55:02.804340 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:02.804309 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-5j4ws_27742f15-b602-4554-84dc-ff9d46a9adda/console-operator/1.log" Jul 10 16:55:02.804774 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:02.804734 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-5j4ws_27742f15-b602-4554-84dc-ff9d46a9adda/console-operator/0.log" Jul 10 16:55:02.804830 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:02.804767 2578 generic.go:356] "Generic (PLEG): container finished" podID="27742f15-b602-4554-84dc-ff9d46a9adda" containerID="48fb817c00618ec6a1745d7c284428d34b11e85cd73a109f800874f572416ffd" exitCode=255 Jul 10 16:55:02.804881 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:02.804863 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-5j4ws" event={"ID":"27742f15-b602-4554-84dc-ff9d46a9adda","Type":"ContainerDied","Data":"48fb817c00618ec6a1745d7c284428d34b11e85cd73a109f800874f572416ffd"} Jul 10 16:55:02.804944 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:02.804931 2578 scope.go:117] "RemoveContainer" containerID="6a60497e48f5753343fc561e7ebf2f749866528ac1ea3dd1ba93e745d4cc63f4" Jul 10 16:55:02.805218 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:02.805135 2578 scope.go:117] "RemoveContainer" containerID="48fb817c00618ec6a1745d7c284428d34b11e85cd73a109f800874f572416ffd" Jul 10 16:55:02.805387 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:55:02.805349 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-5j4ws_openshift-console-operator(27742f15-b602-4554-84dc-ff9d46a9adda)\"" pod="openshift-console-operator/console-operator-7866497494-5j4ws" podUID="27742f15-b602-4554-84dc-ff9d46a9adda" Jul 10 16:55:03.675946 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:03.675904 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-qzmtz"] Jul 10 16:55:03.707234 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:03.707187 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qzmtz"] Jul 10 16:55:03.707386 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:03.707330 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qzmtz" Jul 10 16:55:03.709798 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:03.709775 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 10 16:55:03.709798 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:03.709790 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vsnsq\"" Jul 10 16:55:03.709979 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:03.709781 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 10 16:55:03.809158 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:03.809130 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-5j4ws_27742f15-b602-4554-84dc-ff9d46a9adda/console-operator/1.log" Jul 10 16:55:03.809521 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:03.809460 2578 scope.go:117] "RemoveContainer" containerID="48fb817c00618ec6a1745d7c284428d34b11e85cd73a109f800874f572416ffd" Jul 10 16:55:03.809630 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:55:03.809610 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-5j4ws_openshift-console-operator(27742f15-b602-4554-84dc-ff9d46a9adda)\"" pod="openshift-console-operator/console-operator-7866497494-5j4ws" podUID="27742f15-b602-4554-84dc-ff9d46a9adda" Jul 10 16:55:03.882794 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:03.882760 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/634f2940-fb9b-4a74-ae85-4fba5ee47d8d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qzmtz\" (UID: \"634f2940-fb9b-4a74-ae85-4fba5ee47d8d\") " pod="openshift-insights/insights-runtime-extractor-qzmtz" Jul 10 16:55:03.883006 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:03.882842 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/634f2940-fb9b-4a74-ae85-4fba5ee47d8d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qzmtz\" (UID: \"634f2940-fb9b-4a74-ae85-4fba5ee47d8d\") " pod="openshift-insights/insights-runtime-extractor-qzmtz" Jul 10 16:55:03.883006 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:03.882886 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/634f2940-fb9b-4a74-ae85-4fba5ee47d8d-crio-socket\") pod \"insights-runtime-extractor-qzmtz\" (UID: \"634f2940-fb9b-4a74-ae85-4fba5ee47d8d\") " pod="openshift-insights/insights-runtime-extractor-qzmtz" Jul 10 16:55:03.883006 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:03.882932 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/634f2940-fb9b-4a74-ae85-4fba5ee47d8d-data-volume\") pod \"insights-runtime-extractor-qzmtz\" (UID: \"634f2940-fb9b-4a74-ae85-4fba5ee47d8d\") " pod="openshift-insights/insights-runtime-extractor-qzmtz" Jul 10 16:55:03.883180 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:03.883037 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkgs4\" (UniqueName: \"kubernetes.io/projected/634f2940-fb9b-4a74-ae85-4fba5ee47d8d-kube-api-access-lkgs4\") pod \"insights-runtime-extractor-qzmtz\" (UID: \"634f2940-fb9b-4a74-ae85-4fba5ee47d8d\") " pod="openshift-insights/insights-runtime-extractor-qzmtz" Jul 10 16:55:03.984520 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:03.984427 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkgs4\" (UniqueName: \"kubernetes.io/projected/634f2940-fb9b-4a74-ae85-4fba5ee47d8d-kube-api-access-lkgs4\") pod \"insights-runtime-extractor-qzmtz\" (UID: \"634f2940-fb9b-4a74-ae85-4fba5ee47d8d\") " pod="openshift-insights/insights-runtime-extractor-qzmtz" Jul 10 16:55:03.984520 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:03.984493 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/634f2940-fb9b-4a74-ae85-4fba5ee47d8d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qzmtz\" (UID: \"634f2940-fb9b-4a74-ae85-4fba5ee47d8d\") " pod="openshift-insights/insights-runtime-extractor-qzmtz" Jul 10 16:55:03.984745 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:03.984559 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/634f2940-fb9b-4a74-ae85-4fba5ee47d8d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qzmtz\" (UID: \"634f2940-fb9b-4a74-ae85-4fba5ee47d8d\") " pod="openshift-insights/insights-runtime-extractor-qzmtz" Jul 10 16:55:03.984745 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:55:03.984583 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 16:55:03.984745 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:03.984640 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/634f2940-fb9b-4a74-ae85-4fba5ee47d8d-crio-socket\") pod \"insights-runtime-extractor-qzmtz\" (UID: \"634f2940-fb9b-4a74-ae85-4fba5ee47d8d\") " pod="openshift-insights/insights-runtime-extractor-qzmtz" Jul 10 16:55:03.984745 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:55:03.984674 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 16:55:03.984745 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:55:03.984684 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/634f2940-fb9b-4a74-ae85-4fba5ee47d8d-kube-rbac-proxy-cm podName:634f2940-fb9b-4a74-ae85-4fba5ee47d8d nodeName:}" failed. No retries permitted until 2026-07-10 16:55:04.484664473 +0000 UTC m=+48.573993514 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/634f2940-fb9b-4a74-ae85-4fba5ee47d8d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qzmtz" (UID: "634f2940-fb9b-4a74-ae85-4fba5ee47d8d") : configmap "kube-rbac-proxy" not found Jul 10 16:55:03.984745 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:03.984701 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/634f2940-fb9b-4a74-ae85-4fba5ee47d8d-crio-socket\") pod \"insights-runtime-extractor-qzmtz\" (UID: \"634f2940-fb9b-4a74-ae85-4fba5ee47d8d\") " pod="openshift-insights/insights-runtime-extractor-qzmtz" Jul 10 16:55:03.984745 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:55:03.984727 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/634f2940-fb9b-4a74-ae85-4fba5ee47d8d-insights-runtime-extractor-tls podName:634f2940-fb9b-4a74-ae85-4fba5ee47d8d nodeName:}" failed. No retries permitted until 2026-07-10 16:55:04.484711519 +0000 UTC m=+48.574040553 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/634f2940-fb9b-4a74-ae85-4fba5ee47d8d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qzmtz" (UID: "634f2940-fb9b-4a74-ae85-4fba5ee47d8d") : secret "insights-runtime-extractor-tls" not found Jul 10 16:55:03.985157 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:03.984761 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/634f2940-fb9b-4a74-ae85-4fba5ee47d8d-data-volume\") pod \"insights-runtime-extractor-qzmtz\" (UID: \"634f2940-fb9b-4a74-ae85-4fba5ee47d8d\") " pod="openshift-insights/insights-runtime-extractor-qzmtz" Jul 10 16:55:03.985157 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:03.985138 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/634f2940-fb9b-4a74-ae85-4fba5ee47d8d-data-volume\") pod \"insights-runtime-extractor-qzmtz\" (UID: \"634f2940-fb9b-4a74-ae85-4fba5ee47d8d\") " pod="openshift-insights/insights-runtime-extractor-qzmtz" Jul 10 16:55:03.994132 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:03.994099 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkgs4\" (UniqueName: \"kubernetes.io/projected/634f2940-fb9b-4a74-ae85-4fba5ee47d8d-kube-api-access-lkgs4\") pod \"insights-runtime-extractor-qzmtz\" (UID: \"634f2940-fb9b-4a74-ae85-4fba5ee47d8d\") " pod="openshift-insights/insights-runtime-extractor-qzmtz" Jul 10 16:55:04.490576 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:04.490542 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/634f2940-fb9b-4a74-ae85-4fba5ee47d8d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qzmtz\" (UID: \"634f2940-fb9b-4a74-ae85-4fba5ee47d8d\") " pod="openshift-insights/insights-runtime-extractor-qzmtz" Jul 10 16:55:04.490765 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:04.490610 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/634f2940-fb9b-4a74-ae85-4fba5ee47d8d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qzmtz\" (UID: \"634f2940-fb9b-4a74-ae85-4fba5ee47d8d\") " pod="openshift-insights/insights-runtime-extractor-qzmtz" Jul 10 16:55:04.490765 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:55:04.490680 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 16:55:04.490765 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:55:04.490708 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 16:55:04.490863 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:55:04.490780 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/634f2940-fb9b-4a74-ae85-4fba5ee47d8d-kube-rbac-proxy-cm podName:634f2940-fb9b-4a74-ae85-4fba5ee47d8d nodeName:}" failed. No retries permitted until 2026-07-10 16:55:05.490765268 +0000 UTC m=+49.580094299 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/634f2940-fb9b-4a74-ae85-4fba5ee47d8d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qzmtz" (UID: "634f2940-fb9b-4a74-ae85-4fba5ee47d8d") : configmap "kube-rbac-proxy" not found Jul 10 16:55:04.490863 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:55:04.490794 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/634f2940-fb9b-4a74-ae85-4fba5ee47d8d-insights-runtime-extractor-tls podName:634f2940-fb9b-4a74-ae85-4fba5ee47d8d nodeName:}" failed. No retries permitted until 2026-07-10 16:55:05.490787948 +0000 UTC m=+49.580116978 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/634f2940-fb9b-4a74-ae85-4fba5ee47d8d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qzmtz" (UID: "634f2940-fb9b-4a74-ae85-4fba5ee47d8d") : secret "insights-runtime-extractor-tls" not found Jul 10 16:55:05.075036 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:05.075006 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6kk9g_1d85dbfa-1b16-4e26-b207-6bb7064345ee/dns-node-resolver/0.log" Jul 10 16:55:05.499890 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:05.499793 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/634f2940-fb9b-4a74-ae85-4fba5ee47d8d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qzmtz\" (UID: \"634f2940-fb9b-4a74-ae85-4fba5ee47d8d\") " pod="openshift-insights/insights-runtime-extractor-qzmtz" Jul 10 16:55:05.499890 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:05.499862 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/634f2940-fb9b-4a74-ae85-4fba5ee47d8d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qzmtz\" (UID: \"634f2940-fb9b-4a74-ae85-4fba5ee47d8d\") " pod="openshift-insights/insights-runtime-extractor-qzmtz" Jul 10 16:55:05.500120 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:55:05.499936 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 16:55:05.500120 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:55:05.499978 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 16:55:05.500120 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:55:05.500001 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/634f2940-fb9b-4a74-ae85-4fba5ee47d8d-insights-runtime-extractor-tls podName:634f2940-fb9b-4a74-ae85-4fba5ee47d8d nodeName:}" failed. No retries permitted until 2026-07-10 16:55:07.499986191 +0000 UTC m=+51.589315218 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/634f2940-fb9b-4a74-ae85-4fba5ee47d8d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qzmtz" (UID: "634f2940-fb9b-4a74-ae85-4fba5ee47d8d") : secret "insights-runtime-extractor-tls" not found Jul 10 16:55:05.500120 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:55:05.500038 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/634f2940-fb9b-4a74-ae85-4fba5ee47d8d-kube-rbac-proxy-cm podName:634f2940-fb9b-4a74-ae85-4fba5ee47d8d nodeName:}" failed. No retries permitted until 2026-07-10 16:55:07.500025993 +0000 UTC m=+51.589355020 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/634f2940-fb9b-4a74-ae85-4fba5ee47d8d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qzmtz" (UID: "634f2940-fb9b-4a74-ae85-4fba5ee47d8d") : configmap "kube-rbac-proxy" not found Jul 10 16:55:06.475420 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:06.475393 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-w9kb4_53987bf8-a205-4326-8757-ea8b3a6c4514/node-ca/0.log" Jul 10 16:55:07.082000 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:07.081969 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hwfb8_633753a4-3340-4473-9639-4299c06b53a8/serve-healthcheck-canary/0.log" Jul 10 16:55:07.514195 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:07.514109 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/634f2940-fb9b-4a74-ae85-4fba5ee47d8d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qzmtz\" (UID: \"634f2940-fb9b-4a74-ae85-4fba5ee47d8d\") " pod="openshift-insights/insights-runtime-extractor-qzmtz" Jul 10 16:55:07.514543 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:07.514250 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/634f2940-fb9b-4a74-ae85-4fba5ee47d8d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qzmtz\" (UID: \"634f2940-fb9b-4a74-ae85-4fba5ee47d8d\") " pod="openshift-insights/insights-runtime-extractor-qzmtz" Jul 10 16:55:07.514543 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:55:07.514250 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 16:55:07.514543 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:55:07.514324 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/634f2940-fb9b-4a74-ae85-4fba5ee47d8d-kube-rbac-proxy-cm podName:634f2940-fb9b-4a74-ae85-4fba5ee47d8d nodeName:}" failed. No retries permitted until 2026-07-10 16:55:11.514311664 +0000 UTC m=+55.603640692 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/634f2940-fb9b-4a74-ae85-4fba5ee47d8d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qzmtz" (UID: "634f2940-fb9b-4a74-ae85-4fba5ee47d8d") : configmap "kube-rbac-proxy" not found Jul 10 16:55:07.514543 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:55:07.514324 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 16:55:07.514543 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:55:07.514390 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/634f2940-fb9b-4a74-ae85-4fba5ee47d8d-insights-runtime-extractor-tls podName:634f2940-fb9b-4a74-ae85-4fba5ee47d8d nodeName:}" failed. No retries permitted until 2026-07-10 16:55:11.514372714 +0000 UTC m=+55.603701742 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/634f2940-fb9b-4a74-ae85-4fba5ee47d8d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qzmtz" (UID: "634f2940-fb9b-4a74-ae85-4fba5ee47d8d") : secret "insights-runtime-extractor-tls" not found Jul 10 16:55:08.118852 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:08.118814 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84-service-ca-bundle\") pod \"router-default-bb9cb67d8-ddrbp\" (UID: \"3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84\") " pod="openshift-ingress/router-default-bb9cb67d8-ddrbp" Jul 10 16:55:08.118852 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:08.118854 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c0ba50ec-0b78-46ce-9b5c-e644af63989d-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-fxc54\" (UID: \"c0ba50ec-0b78-46ce-9b5c-e644af63989d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fxc54" Jul 10 16:55:08.119179 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:08.118875 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/04e15cd0-bcac-4375-97d5-b135b8ef9997-registry-tls\") pod \"image-registry-657d95d548-bj62l\" (UID: \"04e15cd0-bcac-4375-97d5-b135b8ef9997\") " pod="openshift-image-registry/image-registry-657d95d548-bj62l" Jul 10 16:55:08.119179 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:08.118901 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/40988796-3d52-4414-89dc-53c322b4d514-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-zfbfz\" (UID: \"40988796-3d52-4414-89dc-53c322b4d514\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zfbfz" Jul 10 16:55:08.119179 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:55:08.118980 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 10 16:55:08.119179 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:55:08.118985 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 16:55:08.119179 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:55:08.119026 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 10 16:55:08.119179 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:55:08.119030 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-657d95d548-bj62l: secret "image-registry-tls" not found Jul 10 16:55:08.119179 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:55:08.118992 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84-service-ca-bundle podName:3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84 nodeName:}" failed. No retries permitted until 2026-07-10 16:55:24.11897305 +0000 UTC m=+68.208302102 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84-service-ca-bundle") pod "router-default-bb9cb67d8-ddrbp" (UID: "3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84") : configmap references non-existent config key: service-ca.crt Jul 10 16:55:08.119179 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:55:08.119141 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0ba50ec-0b78-46ce-9b5c-e644af63989d-samples-operator-tls podName:c0ba50ec-0b78-46ce-9b5c-e644af63989d nodeName:}" failed. No retries permitted until 2026-07-10 16:55:24.119107775 +0000 UTC m=+68.208436804 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c0ba50ec-0b78-46ce-9b5c-e644af63989d-samples-operator-tls") pod "cluster-samples-operator-65db858894-fxc54" (UID: "c0ba50ec-0b78-46ce-9b5c-e644af63989d") : secret "samples-operator-tls" not found Jul 10 16:55:08.119179 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:55:08.119170 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40988796-3d52-4414-89dc-53c322b4d514-cluster-monitoring-operator-tls podName:40988796-3d52-4414-89dc-53c322b4d514 nodeName:}" failed. No retries permitted until 2026-07-10 16:55:24.119163443 +0000 UTC m=+68.208492474 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/40988796-3d52-4414-89dc-53c322b4d514-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-zfbfz" (UID: "40988796-3d52-4414-89dc-53c322b4d514") : secret "cluster-monitoring-operator-tls" not found Jul 10 16:55:08.119179 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:55:08.119179 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/04e15cd0-bcac-4375-97d5-b135b8ef9997-registry-tls podName:04e15cd0-bcac-4375-97d5-b135b8ef9997 nodeName:}" failed. No retries permitted until 2026-07-10 16:55:24.119174455 +0000 UTC m=+68.208503482 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/04e15cd0-bcac-4375-97d5-b135b8ef9997-registry-tls") pod "image-registry-657d95d548-bj62l" (UID: "04e15cd0-bcac-4375-97d5-b135b8ef9997") : secret "image-registry-tls" not found Jul 10 16:55:08.119488 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:08.119223 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84-metrics-certs\") pod \"router-default-bb9cb67d8-ddrbp\" (UID: \"3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84\") " pod="openshift-ingress/router-default-bb9cb67d8-ddrbp" Jul 10 16:55:08.119488 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:55:08.119320 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 10 16:55:08.119488 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:55:08.119354 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84-metrics-certs podName:3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84 nodeName:}" failed. No retries permitted until 2026-07-10 16:55:24.119339476 +0000 UTC m=+68.208668504 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84-metrics-certs") pod "router-default-bb9cb67d8-ddrbp" (UID: "3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84") : secret "router-metrics-certs-default" not found Jul 10 16:55:08.219716 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:08.219678 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/55d7dbb1-c8db-40fc-b416-7b781aae9aab-metrics-tls\") pod \"dns-default-g5nt8\" (UID: \"55d7dbb1-c8db-40fc-b416-7b781aae9aab\") " pod="openshift-dns/dns-default-g5nt8" Jul 10 16:55:08.219956 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:55:08.219793 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 16:55:08.219956 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:55:08.219855 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55d7dbb1-c8db-40fc-b416-7b781aae9aab-metrics-tls podName:55d7dbb1-c8db-40fc-b416-7b781aae9aab nodeName:}" failed. No retries permitted until 2026-07-10 16:55:24.219842023 +0000 UTC m=+68.309171050 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/55d7dbb1-c8db-40fc-b416-7b781aae9aab-metrics-tls") pod "dns-default-g5nt8" (UID: "55d7dbb1-c8db-40fc-b416-7b781aae9aab") : secret "dns-default-metrics-tls" not found Jul 10 16:55:11.551607 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:11.551565 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/634f2940-fb9b-4a74-ae85-4fba5ee47d8d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qzmtz\" (UID: \"634f2940-fb9b-4a74-ae85-4fba5ee47d8d\") " pod="openshift-insights/insights-runtime-extractor-qzmtz" Jul 10 16:55:11.551991 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:11.551621 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/634f2940-fb9b-4a74-ae85-4fba5ee47d8d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qzmtz\" (UID: \"634f2940-fb9b-4a74-ae85-4fba5ee47d8d\") " pod="openshift-insights/insights-runtime-extractor-qzmtz" Jul 10 16:55:11.551991 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:55:11.551724 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 16:55:11.551991 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:55:11.551791 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/634f2940-fb9b-4a74-ae85-4fba5ee47d8d-insights-runtime-extractor-tls podName:634f2940-fb9b-4a74-ae85-4fba5ee47d8d nodeName:}" failed. No retries permitted until 2026-07-10 16:55:19.551775437 +0000 UTC m=+63.641104465 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/634f2940-fb9b-4a74-ae85-4fba5ee47d8d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qzmtz" (UID: "634f2940-fb9b-4a74-ae85-4fba5ee47d8d") : secret "insights-runtime-extractor-tls" not found Jul 10 16:55:11.552122 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:11.552085 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/634f2940-fb9b-4a74-ae85-4fba5ee47d8d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qzmtz\" (UID: \"634f2940-fb9b-4a74-ae85-4fba5ee47d8d\") " pod="openshift-insights/insights-runtime-extractor-qzmtz" Jul 10 16:55:12.643816 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:12.643776 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-5j4ws" Jul 10 16:55:12.644266 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:12.643839 2578 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-5j4ws" Jul 10 16:55:12.644266 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:12.644208 2578 scope.go:117] "RemoveContainer" containerID="48fb817c00618ec6a1745d7c284428d34b11e85cd73a109f800874f572416ffd" Jul 10 16:55:12.839779 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:12.839753 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-5j4ws_27742f15-b602-4554-84dc-ff9d46a9adda/console-operator/2.log" Jul 10 16:55:12.840164 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:12.840148 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-5j4ws_27742f15-b602-4554-84dc-ff9d46a9adda/console-operator/1.log" Jul 10 16:55:12.840226 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:12.840185 2578 generic.go:356] "Generic (PLEG): container finished" podID="27742f15-b602-4554-84dc-ff9d46a9adda" containerID="3f705ea7a7116fe87cc6aa88a8d46f318d52a64fa2a3bc9518490663adc4208e" exitCode=255 Jul 10 16:55:12.840260 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:12.840232 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-5j4ws" event={"ID":"27742f15-b602-4554-84dc-ff9d46a9adda","Type":"ContainerDied","Data":"3f705ea7a7116fe87cc6aa88a8d46f318d52a64fa2a3bc9518490663adc4208e"} Jul 10 16:55:12.840300 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:12.840289 2578 scope.go:117] "RemoveContainer" containerID="48fb817c00618ec6a1745d7c284428d34b11e85cd73a109f800874f572416ffd" Jul 10 16:55:12.840636 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:12.840612 2578 scope.go:117] "RemoveContainer" containerID="3f705ea7a7116fe87cc6aa88a8d46f318d52a64fa2a3bc9518490663adc4208e" Jul 10 16:55:12.840808 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:55:12.840785 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7866497494-5j4ws_openshift-console-operator(27742f15-b602-4554-84dc-ff9d46a9adda)\"" pod="openshift-console-operator/console-operator-7866497494-5j4ws" podUID="27742f15-b602-4554-84dc-ff9d46a9adda" Jul 10 16:55:13.844498 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:13.844472 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-5j4ws_27742f15-b602-4554-84dc-ff9d46a9adda/console-operator/2.log" Jul 10 16:55:18.728343 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:18.728314 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gqgkg" Jul 10 16:55:19.624679 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:19.624640 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/634f2940-fb9b-4a74-ae85-4fba5ee47d8d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qzmtz\" (UID: \"634f2940-fb9b-4a74-ae85-4fba5ee47d8d\") " pod="openshift-insights/insights-runtime-extractor-qzmtz" Jul 10 16:55:19.627009 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:19.626988 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/634f2940-fb9b-4a74-ae85-4fba5ee47d8d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qzmtz\" (UID: \"634f2940-fb9b-4a74-ae85-4fba5ee47d8d\") " pod="openshift-insights/insights-runtime-extractor-qzmtz" Jul 10 16:55:19.920330 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:19.920245 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vsnsq\"" Jul 10 16:55:19.928425 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:19.928391 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qzmtz" Jul 10 16:55:20.077121 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:20.077091 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qzmtz"] Jul 10 16:55:20.079946 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:55:20.079920 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod634f2940_fb9b_4a74_ae85_4fba5ee47d8d.slice/crio-c3478ed056829bc510a1eeead91f7904cc53bbb75b71195fb1d55be960feb017 WatchSource:0}: Error finding container c3478ed056829bc510a1eeead91f7904cc53bbb75b71195fb1d55be960feb017: Status 404 returned error can't find the container with id c3478ed056829bc510a1eeead91f7904cc53bbb75b71195fb1d55be960feb017 Jul 10 16:55:20.870417 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:20.870380 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qzmtz" event={"ID":"634f2940-fb9b-4a74-ae85-4fba5ee47d8d","Type":"ContainerStarted","Data":"c3478ed056829bc510a1eeead91f7904cc53bbb75b71195fb1d55be960feb017"} Jul 10 16:55:21.140537 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:21.140453 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a0b23658-26fb-454a-a42e-88432bf12b0a-original-pull-secret\") pod \"global-pull-secret-syncer-dgk2w\" (UID: \"a0b23658-26fb-454a-a42e-88432bf12b0a\") " pod="kube-system/global-pull-secret-syncer-dgk2w" Jul 10 16:55:21.140537 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:21.140518 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d43ce12f-a8fd-4a94-bb94-09e6562e1250-metrics-certs\") pod \"network-metrics-daemon-jr8jk\" (UID: \"d43ce12f-a8fd-4a94-bb94-09e6562e1250\") " pod="openshift-multus/network-metrics-daemon-jr8jk" Jul 10 16:55:21.143151 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:21.143127 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 10 16:55:21.144079 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:21.144043 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 10 16:55:21.153538 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:21.153514 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d43ce12f-a8fd-4a94-bb94-09e6562e1250-metrics-certs\") pod \"network-metrics-daemon-jr8jk\" (UID: \"d43ce12f-a8fd-4a94-bb94-09e6562e1250\") " pod="openshift-multus/network-metrics-daemon-jr8jk" Jul 10 16:55:21.153646 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:21.153564 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a0b23658-26fb-454a-a42e-88432bf12b0a-original-pull-secret\") pod \"global-pull-secret-syncer-dgk2w\" (UID: \"a0b23658-26fb-454a-a42e-88432bf12b0a\") " pod="kube-system/global-pull-secret-syncer-dgk2w" Jul 10 16:55:21.235301 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:21.235248 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dgk2w" Jul 10 16:55:21.241650 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:21.241618 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfk5q\" (UniqueName: \"kubernetes.io/projected/16033c9b-cf7b-4afc-abe6-df0ded50ff80-kube-api-access-sfk5q\") pod \"network-check-target-jvkw6\" (UID: \"16033c9b-cf7b-4afc-abe6-df0ded50ff80\") " pod="openshift-network-diagnostics/network-check-target-jvkw6" Jul 10 16:55:21.244119 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:21.243911 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qdmq6\"" Jul 10 16:55:21.244742 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:21.244723 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfk5q\" (UniqueName: \"kubernetes.io/projected/16033c9b-cf7b-4afc-abe6-df0ded50ff80-kube-api-access-sfk5q\") pod \"network-check-target-jvkw6\" (UID: \"16033c9b-cf7b-4afc-abe6-df0ded50ff80\") " pod="openshift-network-diagnostics/network-check-target-jvkw6" Jul 10 16:55:21.252387 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:21.252167 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jr8jk" Jul 10 16:55:21.354513 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:21.354313 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jn8d4\"" Jul 10 16:55:21.362493 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:21.362207 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jvkw6" Jul 10 16:55:21.386188 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:21.386150 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dgk2w"] Jul 10 16:55:21.390184 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:55:21.390137 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0b23658_26fb_454a_a42e_88432bf12b0a.slice/crio-5f1944e910fc03d8a191731ae524cd9a9090a7ad05eff4b731c2eb4ddd78036a WatchSource:0}: Error finding container 5f1944e910fc03d8a191731ae524cd9a9090a7ad05eff4b731c2eb4ddd78036a: Status 404 returned error can't find the container with id 5f1944e910fc03d8a191731ae524cd9a9090a7ad05eff4b731c2eb4ddd78036a Jul 10 16:55:21.404266 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:21.404198 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jr8jk"] Jul 10 16:55:21.408536 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:55:21.408506 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd43ce12f_a8fd_4a94_bb94_09e6562e1250.slice/crio-4c13d2825b9f595b5b7afd50a319f28f19b905bbda13e5fb4f2a50a2a776fa2e WatchSource:0}: Error finding container 4c13d2825b9f595b5b7afd50a319f28f19b905bbda13e5fb4f2a50a2a776fa2e: Status 404 returned error can't find the container with id 4c13d2825b9f595b5b7afd50a319f28f19b905bbda13e5fb4f2a50a2a776fa2e Jul 10 16:55:21.510822 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:21.510788 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jvkw6"] Jul 10 16:55:21.514403 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:55:21.514377 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16033c9b_cf7b_4afc_abe6_df0ded50ff80.slice/crio-4a2e768be1b460c927117a1bb462e6d221b3c9c4689b9deb4632f16e3e5d22a7 WatchSource:0}: Error finding container 4a2e768be1b460c927117a1bb462e6d221b3c9c4689b9deb4632f16e3e5d22a7: Status 404 returned error can't find the container with id 4a2e768be1b460c927117a1bb462e6d221b3c9c4689b9deb4632f16e3e5d22a7 Jul 10 16:55:21.875795 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:21.875761 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dgk2w" event={"ID":"a0b23658-26fb-454a-a42e-88432bf12b0a","Type":"ContainerStarted","Data":"5f1944e910fc03d8a191731ae524cd9a9090a7ad05eff4b731c2eb4ddd78036a"} Jul 10 16:55:21.877408 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:21.877383 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jvkw6" event={"ID":"16033c9b-cf7b-4afc-abe6-df0ded50ff80","Type":"ContainerStarted","Data":"fb54171ee6bc472eafafb3c7973606c6635bbb96c8d64c1973c9d3e60d75f15c"} Jul 10 16:55:21.877546 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:21.877425 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jvkw6" event={"ID":"16033c9b-cf7b-4afc-abe6-df0ded50ff80","Type":"ContainerStarted","Data":"4a2e768be1b460c927117a1bb462e6d221b3c9c4689b9deb4632f16e3e5d22a7"} Jul 10 16:55:21.877546 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:21.877508 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-jvkw6" Jul 10 16:55:21.878642 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:21.878620 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jr8jk" event={"ID":"d43ce12f-a8fd-4a94-bb94-09e6562e1250","Type":"ContainerStarted","Data":"4c13d2825b9f595b5b7afd50a319f28f19b905bbda13e5fb4f2a50a2a776fa2e"} Jul 10 16:55:21.896607 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:21.896551 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-jvkw6" podStartSLOduration=65.896536768 podStartE2EDuration="1m5.896536768s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:54:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:55:21.894975323 +0000 UTC m=+65.984304373" watchObservedRunningTime="2026-07-10 16:55:21.896536768 +0000 UTC m=+65.985865795" Jul 10 16:55:22.643881 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:22.643835 2578 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-5j4ws" Jul 10 16:55:22.644418 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:22.643909 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-5j4ws" Jul 10 16:55:22.644418 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:22.644343 2578 scope.go:117] "RemoveContainer" containerID="3f705ea7a7116fe87cc6aa88a8d46f318d52a64fa2a3bc9518490663adc4208e" Jul 10 16:55:22.644560 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:55:22.644512 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7866497494-5j4ws_openshift-console-operator(27742f15-b602-4554-84dc-ff9d46a9adda)\"" pod="openshift-console-operator/console-operator-7866497494-5j4ws" podUID="27742f15-b602-4554-84dc-ff9d46a9adda" Jul 10 16:55:24.172641 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:24.172598 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/40988796-3d52-4414-89dc-53c322b4d514-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-zfbfz\" (UID: \"40988796-3d52-4414-89dc-53c322b4d514\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zfbfz" Jul 10 16:55:24.173135 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:24.172677 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84-metrics-certs\") pod \"router-default-bb9cb67d8-ddrbp\" (UID: \"3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84\") " pod="openshift-ingress/router-default-bb9cb67d8-ddrbp" Jul 10 16:55:24.173135 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:24.172737 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84-service-ca-bundle\") pod \"router-default-bb9cb67d8-ddrbp\" (UID: \"3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84\") " pod="openshift-ingress/router-default-bb9cb67d8-ddrbp" Jul 10 16:55:24.173135 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:24.172872 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c0ba50ec-0b78-46ce-9b5c-e644af63989d-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-fxc54\" (UID: \"c0ba50ec-0b78-46ce-9b5c-e644af63989d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fxc54" Jul 10 16:55:24.173135 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:24.172926 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/04e15cd0-bcac-4375-97d5-b135b8ef9997-registry-tls\") pod \"image-registry-657d95d548-bj62l\" (UID: \"04e15cd0-bcac-4375-97d5-b135b8ef9997\") " pod="openshift-image-registry/image-registry-657d95d548-bj62l" Jul 10 16:55:24.173412 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:24.173391 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84-service-ca-bundle\") pod \"router-default-bb9cb67d8-ddrbp\" (UID: \"3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84\") " pod="openshift-ingress/router-default-bb9cb67d8-ddrbp" Jul 10 16:55:24.175766 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:24.175738 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c0ba50ec-0b78-46ce-9b5c-e644af63989d-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-fxc54\" (UID: \"c0ba50ec-0b78-46ce-9b5c-e644af63989d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fxc54" Jul 10 16:55:24.175965 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:24.175939 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/04e15cd0-bcac-4375-97d5-b135b8ef9997-registry-tls\") pod \"image-registry-657d95d548-bj62l\" (UID: \"04e15cd0-bcac-4375-97d5-b135b8ef9997\") " pod="openshift-image-registry/image-registry-657d95d548-bj62l" Jul 10 16:55:24.176141 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:24.176122 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/40988796-3d52-4414-89dc-53c322b4d514-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-zfbfz\" (UID: \"40988796-3d52-4414-89dc-53c322b4d514\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zfbfz" Jul 10 16:55:24.176577 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:24.176560 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84-metrics-certs\") pod \"router-default-bb9cb67d8-ddrbp\" (UID: \"3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84\") " pod="openshift-ingress/router-default-bb9cb67d8-ddrbp" Jul 10 16:55:24.274239 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:24.274199 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/55d7dbb1-c8db-40fc-b416-7b781aae9aab-metrics-tls\") pod \"dns-default-g5nt8\" (UID: \"55d7dbb1-c8db-40fc-b416-7b781aae9aab\") " pod="openshift-dns/dns-default-g5nt8" Jul 10 16:55:24.277203 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:24.277178 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/55d7dbb1-c8db-40fc-b416-7b781aae9aab-metrics-tls\") pod \"dns-default-g5nt8\" (UID: \"55d7dbb1-c8db-40fc-b416-7b781aae9aab\") " pod="openshift-dns/dns-default-g5nt8" Jul 10 16:55:24.393161 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:24.393123 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-mxq54\"" Jul 10 16:55:24.401513 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:24.401305 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zfbfz" Jul 10 16:55:24.403876 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:24.403856 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-phvw6\"" Jul 10 16:55:24.412133 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:24.411932 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-657d95d548-bj62l" Jul 10 16:55:24.414442 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:24.414414 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-j6txs\"" Jul 10 16:55:24.422098 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:24.422076 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fxc54" Jul 10 16:55:24.424837 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:24.424817 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-l7wpk\"" Jul 10 16:55:24.432959 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:24.432936 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-bb9cb67d8-ddrbp" Jul 10 16:55:24.460640 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:24.460612 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ccqfl\"" Jul 10 16:55:24.468837 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:24.468799 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-g5nt8" Jul 10 16:55:24.829225 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:24.829036 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fxc54"] Jul 10 16:55:24.872343 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:24.872267 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zfbfz"] Jul 10 16:55:24.882277 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:55:24.881903 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40988796_3d52_4414_89dc_53c322b4d514.slice/crio-ef664d50cfbb8bc2759de1e2474be061adbde42f22d5bdb24fa135efdcf1bf2e WatchSource:0}: Error finding container ef664d50cfbb8bc2759de1e2474be061adbde42f22d5bdb24fa135efdcf1bf2e: Status 404 returned error can't find the container with id ef664d50cfbb8bc2759de1e2474be061adbde42f22d5bdb24fa135efdcf1bf2e Jul 10 16:55:24.896888 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:24.896822 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-bb9cb67d8-ddrbp"] Jul 10 16:55:24.897270 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:24.897194 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zfbfz" event={"ID":"40988796-3d52-4414-89dc-53c322b4d514","Type":"ContainerStarted","Data":"ef664d50cfbb8bc2759de1e2474be061adbde42f22d5bdb24fa135efdcf1bf2e"} Jul 10 16:55:24.901980 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:55:24.901934 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c7fb6e2_bbb4_45a7_b717_7bd085ad9a84.slice/crio-e882e0413bc30862771ef722f7ab821e8dba47836a10ac388cd6231648ee359d WatchSource:0}: Error finding container e882e0413bc30862771ef722f7ab821e8dba47836a10ac388cd6231648ee359d: Status 404 returned error can't find the container with id e882e0413bc30862771ef722f7ab821e8dba47836a10ac388cd6231648ee359d Jul 10 16:55:24.902275 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:24.902206 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qzmtz" event={"ID":"634f2940-fb9b-4a74-ae85-4fba5ee47d8d","Type":"ContainerStarted","Data":"940cb31d8179dff77cf47b2559d342d4ea41b3dd91852efe7b6c6b614e892de7"} Jul 10 16:55:24.904391 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:24.904253 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jr8jk" event={"ID":"d43ce12f-a8fd-4a94-bb94-09e6562e1250","Type":"ContainerStarted","Data":"336c4142d933ad8cd8b107d811c7663667d1e66d78d08cfafa3b1afc198987ee"} Jul 10 16:55:24.920822 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:24.920776 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-657d95d548-bj62l"] Jul 10 16:55:24.926506 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:55:24.926476 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04e15cd0_bcac_4375_97d5_b135b8ef9997.slice/crio-2861230c20d842f0f83e0932c1d2fc55f926ded28384e4a753c110480436c51e WatchSource:0}: Error finding container 2861230c20d842f0f83e0932c1d2fc55f926ded28384e4a753c110480436c51e: Status 404 returned error can't find the container with id 2861230c20d842f0f83e0932c1d2fc55f926ded28384e4a753c110480436c51e Jul 10 16:55:25.126398 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:25.126342 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-g5nt8"] Jul 10 16:55:25.131600 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:55:25.131549 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55d7dbb1_c8db_40fc_b416_7b781aae9aab.slice/crio-02ed27967182690bd2ff41bd0d6d650c73bd3ea0297fc5c7926119bdb5575377 WatchSource:0}: Error finding container 02ed27967182690bd2ff41bd0d6d650c73bd3ea0297fc5c7926119bdb5575377: Status 404 returned error can't find the container with id 02ed27967182690bd2ff41bd0d6d650c73bd3ea0297fc5c7926119bdb5575377 Jul 10 16:55:25.910817 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:25.910772 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-g5nt8" event={"ID":"55d7dbb1-c8db-40fc-b416-7b781aae9aab","Type":"ContainerStarted","Data":"02ed27967182690bd2ff41bd0d6d650c73bd3ea0297fc5c7926119bdb5575377"} Jul 10 16:55:25.913757 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:25.912946 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-bb9cb67d8-ddrbp" event={"ID":"3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84","Type":"ContainerStarted","Data":"5dc67e02cff8ac133f37d3cfe1490f8e0b44d91c76c7ee3b99ba16e0c0eb65df"} Jul 10 16:55:25.913757 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:25.912994 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-bb9cb67d8-ddrbp" event={"ID":"3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84","Type":"ContainerStarted","Data":"e882e0413bc30862771ef722f7ab821e8dba47836a10ac388cd6231648ee359d"} Jul 10 16:55:25.916218 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:25.916144 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fxc54" event={"ID":"c0ba50ec-0b78-46ce-9b5c-e644af63989d","Type":"ContainerStarted","Data":"502bbb2ef4c1bd3591cd757513f5dc03ecc2f2bccfe9ac3d19c99222cbbdbd23"} Jul 10 16:55:25.919850 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:25.918417 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-657d95d548-bj62l" event={"ID":"04e15cd0-bcac-4375-97d5-b135b8ef9997","Type":"ContainerStarted","Data":"20772b6787cfaf262e719a98a0e1614bdb0ce0e6fa3a9ff6a5967b13931fb206"} Jul 10 16:55:25.919850 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:25.918454 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-657d95d548-bj62l" event={"ID":"04e15cd0-bcac-4375-97d5-b135b8ef9997","Type":"ContainerStarted","Data":"2861230c20d842f0f83e0932c1d2fc55f926ded28384e4a753c110480436c51e"} Jul 10 16:55:25.919850 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:25.918965 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-657d95d548-bj62l" Jul 10 16:55:25.923816 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:25.923749 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qzmtz" event={"ID":"634f2940-fb9b-4a74-ae85-4fba5ee47d8d","Type":"ContainerStarted","Data":"556708d8c2fafaab043223e628ecfaa0f933990949c1f95b4f7ea490460b8533"} Jul 10 16:55:25.926551 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:25.926512 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jr8jk" event={"ID":"d43ce12f-a8fd-4a94-bb94-09e6562e1250","Type":"ContainerStarted","Data":"285ad2cb29f14be56e0b80de6cb131130abc41a55f7df5b889b8b3dec7e11ed1"} Jul 10 16:55:25.938017 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:25.936900 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-bb9cb67d8-ddrbp" podStartSLOduration=45.936881693 podStartE2EDuration="45.936881693s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:54:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:55:25.935248403 +0000 UTC m=+70.024577443" watchObservedRunningTime="2026-07-10 16:55:25.936881693 +0000 UTC m=+70.026210742" Jul 10 16:55:25.973857 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:25.973462 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-657d95d548-bj62l" podStartSLOduration=68.973441837 podStartE2EDuration="1m8.973441837s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:54:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:55:25.956087757 +0000 UTC m=+70.045416808" watchObservedRunningTime="2026-07-10 16:55:25.973441837 +0000 UTC m=+70.062770886" Jul 10 16:55:26.433981 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:26.433951 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-bb9cb67d8-ddrbp" Jul 10 16:55:26.436637 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:26.436612 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-bb9cb67d8-ddrbp" Jul 10 16:55:26.461420 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:26.460346 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-jr8jk" podStartSLOduration=67.224734351 podStartE2EDuration="1m10.460324586s" totalImagesPullingTime="3.235590235s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:54:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:55:25.973833793 +0000 UTC m=+70.063162841" watchObservedRunningTime="2026-07-10 16:55:26.460324586 +0000 UTC m=+70.549653637" Jul 10 16:55:26.933472 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:26.933437 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-bb9cb67d8-ddrbp" Jul 10 16:55:26.934796 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:26.934773 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-bb9cb67d8-ddrbp" Jul 10 16:55:31.953042 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:31.953001 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fxc54" event={"ID":"c0ba50ec-0b78-46ce-9b5c-e644af63989d","Type":"ContainerStarted","Data":"ce0d288a9f1b9f80afec928c614dc66620219e243f77be9ca1f43918eb1d6e4d"} Jul 10 16:55:31.953497 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:31.953092 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fxc54" event={"ID":"c0ba50ec-0b78-46ce-9b5c-e644af63989d","Type":"ContainerStarted","Data":"c7ee84b153b2e442cdf2528740f13bd8aaf8e440396232833552715dfe55e83f"} Jul 10 16:55:31.954501 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:31.954474 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dgk2w" event={"ID":"a0b23658-26fb-454a-a42e-88432bf12b0a","Type":"ContainerStarted","Data":"d6e7c9630f9c0ec7b28364f12396a9c2913f9362feabc83aee01d1f02511fe71"} Jul 10 16:55:31.955814 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:31.955784 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zfbfz" event={"ID":"40988796-3d52-4414-89dc-53c322b4d514","Type":"ContainerStarted","Data":"b80fe9a77510cfd4d8d74f5be67990493141b402efa10c65a498d39b363c6240"} Jul 10 16:55:31.957812 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:31.957785 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qzmtz" event={"ID":"634f2940-fb9b-4a74-ae85-4fba5ee47d8d","Type":"ContainerStarted","Data":"b485ca8d27329c31d20ef012ff9d970cab15909b72a17cc79c7eb931d4b1e2cb"} Jul 10 16:55:31.961811 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:31.961791 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-g5nt8" event={"ID":"55d7dbb1-c8db-40fc-b416-7b781aae9aab","Type":"ContainerStarted","Data":"c31d86ce5049dfd6431cf5a8770639e72b7a1bd6120cba751a8610b6f55bd4eb"} Jul 10 16:55:31.961894 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:31.961828 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-g5nt8" event={"ID":"55d7dbb1-c8db-40fc-b416-7b781aae9aab","Type":"ContainerStarted","Data":"8060600bc5f724919a97d647922eefd5dc248e479fcf39cc85c8f9314b356970"} Jul 10 16:55:31.961942 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:31.961928 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-g5nt8" Jul 10 16:55:31.970780 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:31.970741 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-fxc54" podStartSLOduration=45.915970374 podStartE2EDuration="51.970729586s" totalImagesPullingTime="6.054759212s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:54:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:55:31.970424588 +0000 UTC m=+76.059753643" watchObservedRunningTime="2026-07-10 16:55:31.970729586 +0000 UTC m=+76.060058686" Jul 10 16:55:31.988282 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:31.988229 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-dgk2w" podStartSLOduration=65.413616805 podStartE2EDuration="1m14.988211577s" totalImagesPullingTime="9.574594772s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 16:54:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:55:31.987696963 +0000 UTC m=+76.077026012" watchObservedRunningTime="2026-07-10 16:55:31.988211577 +0000 UTC m=+76.077540626" Jul 10 16:55:32.004338 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:32.004283 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-zfbfz" podStartSLOduration=45.922483932 podStartE2EDuration="52.004267994s" totalImagesPullingTime="6.081784062s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:54:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:55:32.003824564 +0000 UTC m=+76.093153614" watchObservedRunningTime="2026-07-10 16:55:32.004267994 +0000 UTC m=+76.093597042" Jul 10 16:55:32.024952 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:32.024904 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-qzmtz" podStartSLOduration=18.35664354 podStartE2EDuration="29.024888831s" totalImagesPullingTime="10.668245291s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 16:55:03 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:55:32.023390452 +0000 UTC m=+76.112719500" watchObservedRunningTime="2026-07-10 16:55:32.024888831 +0000 UTC m=+76.114217893" Jul 10 16:55:32.046413 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:32.046362 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-g5nt8" podStartSLOduration=34.214968664 podStartE2EDuration="40.046342403s" totalImagesPullingTime="5.831373739s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:54:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:55:32.045948092 +0000 UTC m=+76.135277150" watchObservedRunningTime="2026-07-10 16:55:32.046342403 +0000 UTC m=+76.135671453" Jul 10 16:55:36.804386 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:36.804236 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-gnmbh"] Jul 10 16:55:36.806816 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:36.806783 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-gnmbh" Jul 10 16:55:36.810020 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:36.809999 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 10 16:55:36.811132 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:36.811114 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 10 16:55:36.811203 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:36.811113 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 10 16:55:36.811203 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:36.811118 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-tplnz\"" Jul 10 16:55:36.818778 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:36.818752 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-gnmbh"] Jul 10 16:55:36.977699 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:36.977661 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/04f0588a-4a06-4776-9ab6-8f60cdd61155-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-gnmbh\" (UID: \"04f0588a-4a06-4776-9ab6-8f60cdd61155\") " pod="openshift-monitoring/prometheus-operator-64785b6675-gnmbh" Jul 10 16:55:36.977699 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:36.977707 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/04f0588a-4a06-4776-9ab6-8f60cdd61155-metrics-client-ca\") pod \"prometheus-operator-64785b6675-gnmbh\" (UID: \"04f0588a-4a06-4776-9ab6-8f60cdd61155\") " pod="openshift-monitoring/prometheus-operator-64785b6675-gnmbh" Jul 10 16:55:36.977916 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:36.977763 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ss6bp\" (UniqueName: \"kubernetes.io/projected/04f0588a-4a06-4776-9ab6-8f60cdd61155-kube-api-access-ss6bp\") pod \"prometheus-operator-64785b6675-gnmbh\" (UID: \"04f0588a-4a06-4776-9ab6-8f60cdd61155\") " pod="openshift-monitoring/prometheus-operator-64785b6675-gnmbh" Jul 10 16:55:36.977916 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:36.977784 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/04f0588a-4a06-4776-9ab6-8f60cdd61155-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-gnmbh\" (UID: \"04f0588a-4a06-4776-9ab6-8f60cdd61155\") " pod="openshift-monitoring/prometheus-operator-64785b6675-gnmbh" Jul 10 16:55:37.079078 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:37.078968 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/04f0588a-4a06-4776-9ab6-8f60cdd61155-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-gnmbh\" (UID: \"04f0588a-4a06-4776-9ab6-8f60cdd61155\") " pod="openshift-monitoring/prometheus-operator-64785b6675-gnmbh" Jul 10 16:55:37.079078 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:37.079002 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/04f0588a-4a06-4776-9ab6-8f60cdd61155-metrics-client-ca\") pod \"prometheus-operator-64785b6675-gnmbh\" (UID: \"04f0588a-4a06-4776-9ab6-8f60cdd61155\") " pod="openshift-monitoring/prometheus-operator-64785b6675-gnmbh" Jul 10 16:55:37.079078 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:37.079031 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ss6bp\" (UniqueName: \"kubernetes.io/projected/04f0588a-4a06-4776-9ab6-8f60cdd61155-kube-api-access-ss6bp\") pod \"prometheus-operator-64785b6675-gnmbh\" (UID: \"04f0588a-4a06-4776-9ab6-8f60cdd61155\") " pod="openshift-monitoring/prometheus-operator-64785b6675-gnmbh" Jul 10 16:55:37.079356 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:37.079082 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/04f0588a-4a06-4776-9ab6-8f60cdd61155-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-gnmbh\" (UID: \"04f0588a-4a06-4776-9ab6-8f60cdd61155\") " pod="openshift-monitoring/prometheus-operator-64785b6675-gnmbh" Jul 10 16:55:37.079663 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:37.079635 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/04f0588a-4a06-4776-9ab6-8f60cdd61155-metrics-client-ca\") pod \"prometheus-operator-64785b6675-gnmbh\" (UID: \"04f0588a-4a06-4776-9ab6-8f60cdd61155\") " pod="openshift-monitoring/prometheus-operator-64785b6675-gnmbh" Jul 10 16:55:37.081464 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:37.081443 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/04f0588a-4a06-4776-9ab6-8f60cdd61155-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-gnmbh\" (UID: \"04f0588a-4a06-4776-9ab6-8f60cdd61155\") " pod="openshift-monitoring/prometheus-operator-64785b6675-gnmbh" Jul 10 16:55:37.081553 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:37.081476 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/04f0588a-4a06-4776-9ab6-8f60cdd61155-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-gnmbh\" (UID: \"04f0588a-4a06-4776-9ab6-8f60cdd61155\") " pod="openshift-monitoring/prometheus-operator-64785b6675-gnmbh" Jul 10 16:55:37.091482 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:37.091460 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ss6bp\" (UniqueName: \"kubernetes.io/projected/04f0588a-4a06-4776-9ab6-8f60cdd61155-kube-api-access-ss6bp\") pod \"prometheus-operator-64785b6675-gnmbh\" (UID: \"04f0588a-4a06-4776-9ab6-8f60cdd61155\") " pod="openshift-monitoring/prometheus-operator-64785b6675-gnmbh" Jul 10 16:55:37.117454 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:37.117424 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-gnmbh" Jul 10 16:55:37.250497 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:37.250388 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-gnmbh"] Jul 10 16:55:37.253008 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:55:37.252975 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04f0588a_4a06_4776_9ab6_8f60cdd61155.slice/crio-b095a16f100a65e509c09fcd58fd552b35d0d5254a7d5494f9f8971397053343 WatchSource:0}: Error finding container b095a16f100a65e509c09fcd58fd552b35d0d5254a7d5494f9f8971397053343: Status 404 returned error can't find the container with id b095a16f100a65e509c09fcd58fd552b35d0d5254a7d5494f9f8971397053343 Jul 10 16:55:37.420399 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:37.420317 2578 scope.go:117] "RemoveContainer" containerID="3f705ea7a7116fe87cc6aa88a8d46f318d52a64fa2a3bc9518490663adc4208e" Jul 10 16:55:37.986957 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:37.986923 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-5j4ws_27742f15-b602-4554-84dc-ff9d46a9adda/console-operator/2.log" Jul 10 16:55:37.987449 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:37.987082 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-5j4ws" event={"ID":"27742f15-b602-4554-84dc-ff9d46a9adda","Type":"ContainerStarted","Data":"b99913ebe55a9a04c87ead029be775b4fa097b84b1decc73526b5855b7eef1e8"} Jul 10 16:55:37.987778 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:37.987759 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-5j4ws" Jul 10 16:55:37.988927 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:37.988892 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-gnmbh" event={"ID":"04f0588a-4a06-4776-9ab6-8f60cdd61155","Type":"ContainerStarted","Data":"b095a16f100a65e509c09fcd58fd552b35d0d5254a7d5494f9f8971397053343"} Jul 10 16:55:38.009385 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:38.009332 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-5j4ws" podStartSLOduration=50.179027381 podStartE2EDuration="58.009318008s" totalImagesPullingTime="7.830290627s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:54:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:55:38.00798169 +0000 UTC m=+82.097310752" watchObservedRunningTime="2026-07-10 16:55:38.009318008 +0000 UTC m=+82.098647057" Jul 10 16:55:38.266062 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:38.265962 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-5j4ws" Jul 10 16:55:38.994562 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:38.994525 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-gnmbh" event={"ID":"04f0588a-4a06-4776-9ab6-8f60cdd61155","Type":"ContainerStarted","Data":"62736eaf5bedcbcee106aa4f8176eae12001d53b7282adeb7749732e5209d5cb"} Jul 10 16:55:38.994958 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:38.994580 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-gnmbh" event={"ID":"04f0588a-4a06-4776-9ab6-8f60cdd61155","Type":"ContainerStarted","Data":"547d85b9a34a64b638f16220d03f8d5e1c668449a282b8328f224e09e0e153f1"} Jul 10 16:55:39.015575 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:39.015520 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-gnmbh" podStartSLOduration=1.833910801 podStartE2EDuration="3.015503608s" totalImagesPullingTime="1.181592807s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:55:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:55:39.014330255 +0000 UTC m=+83.103659305" watchObservedRunningTime="2026-07-10 16:55:39.015503608 +0000 UTC m=+83.104832656" Jul 10 16:55:41.222290 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:41.222255 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-t6dps"] Jul 10 16:55:41.225244 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:41.225225 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-t6dps" Jul 10 16:55:41.227839 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:41.227816 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 10 16:55:41.227979 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:41.227871 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 10 16:55:41.228238 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:41.228218 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-kbv2t\"" Jul 10 16:55:41.228289 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:41.228243 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 10 16:55:41.310630 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:41.310594 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b67ee4fa-977b-42e9-b192-6ebcc6471bbe-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-t6dps\" (UID: \"b67ee4fa-977b-42e9-b192-6ebcc6471bbe\") " pod="openshift-monitoring/node-exporter-t6dps" Jul 10 16:55:41.310630 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:41.310635 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b67ee4fa-977b-42e9-b192-6ebcc6471bbe-node-exporter-accelerators-collector-config\") pod \"node-exporter-t6dps\" (UID: \"b67ee4fa-977b-42e9-b192-6ebcc6471bbe\") " pod="openshift-monitoring/node-exporter-t6dps" Jul 10 16:55:41.310851 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:41.310655 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b67ee4fa-977b-42e9-b192-6ebcc6471bbe-sys\") pod \"node-exporter-t6dps\" (UID: \"b67ee4fa-977b-42e9-b192-6ebcc6471bbe\") " pod="openshift-monitoring/node-exporter-t6dps" Jul 10 16:55:41.310851 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:41.310684 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hncnm\" (UniqueName: \"kubernetes.io/projected/b67ee4fa-977b-42e9-b192-6ebcc6471bbe-kube-api-access-hncnm\") pod \"node-exporter-t6dps\" (UID: \"b67ee4fa-977b-42e9-b192-6ebcc6471bbe\") " pod="openshift-monitoring/node-exporter-t6dps" Jul 10 16:55:41.310851 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:41.310727 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b67ee4fa-977b-42e9-b192-6ebcc6471bbe-node-exporter-textfile\") pod \"node-exporter-t6dps\" (UID: \"b67ee4fa-977b-42e9-b192-6ebcc6471bbe\") " pod="openshift-monitoring/node-exporter-t6dps" Jul 10 16:55:41.310851 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:41.310755 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b67ee4fa-977b-42e9-b192-6ebcc6471bbe-node-exporter-tls\") pod \"node-exporter-t6dps\" (UID: \"b67ee4fa-977b-42e9-b192-6ebcc6471bbe\") " pod="openshift-monitoring/node-exporter-t6dps" Jul 10 16:55:41.310851 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:41.310806 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b67ee4fa-977b-42e9-b192-6ebcc6471bbe-node-exporter-wtmp\") pod \"node-exporter-t6dps\" (UID: \"b67ee4fa-977b-42e9-b192-6ebcc6471bbe\") " pod="openshift-monitoring/node-exporter-t6dps" Jul 10 16:55:41.311031 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:41.310863 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b67ee4fa-977b-42e9-b192-6ebcc6471bbe-metrics-client-ca\") pod \"node-exporter-t6dps\" (UID: \"b67ee4fa-977b-42e9-b192-6ebcc6471bbe\") " pod="openshift-monitoring/node-exporter-t6dps" Jul 10 16:55:41.311031 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:41.310886 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b67ee4fa-977b-42e9-b192-6ebcc6471bbe-root\") pod \"node-exporter-t6dps\" (UID: \"b67ee4fa-977b-42e9-b192-6ebcc6471bbe\") " pod="openshift-monitoring/node-exporter-t6dps" Jul 10 16:55:41.411382 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:41.411346 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b67ee4fa-977b-42e9-b192-6ebcc6471bbe-node-exporter-accelerators-collector-config\") pod \"node-exporter-t6dps\" (UID: \"b67ee4fa-977b-42e9-b192-6ebcc6471bbe\") " pod="openshift-monitoring/node-exporter-t6dps" Jul 10 16:55:41.411382 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:41.411385 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b67ee4fa-977b-42e9-b192-6ebcc6471bbe-sys\") pod \"node-exporter-t6dps\" (UID: \"b67ee4fa-977b-42e9-b192-6ebcc6471bbe\") " pod="openshift-monitoring/node-exporter-t6dps" Jul 10 16:55:41.411623 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:41.411418 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hncnm\" (UniqueName: \"kubernetes.io/projected/b67ee4fa-977b-42e9-b192-6ebcc6471bbe-kube-api-access-hncnm\") pod \"node-exporter-t6dps\" (UID: \"b67ee4fa-977b-42e9-b192-6ebcc6471bbe\") " pod="openshift-monitoring/node-exporter-t6dps" Jul 10 16:55:41.411623 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:41.411440 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b67ee4fa-977b-42e9-b192-6ebcc6471bbe-node-exporter-textfile\") pod \"node-exporter-t6dps\" (UID: \"b67ee4fa-977b-42e9-b192-6ebcc6471bbe\") " pod="openshift-monitoring/node-exporter-t6dps" Jul 10 16:55:41.411623 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:41.411466 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b67ee4fa-977b-42e9-b192-6ebcc6471bbe-node-exporter-tls\") pod \"node-exporter-t6dps\" (UID: \"b67ee4fa-977b-42e9-b192-6ebcc6471bbe\") " pod="openshift-monitoring/node-exporter-t6dps" Jul 10 16:55:41.411623 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:41.411470 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b67ee4fa-977b-42e9-b192-6ebcc6471bbe-sys\") pod \"node-exporter-t6dps\" (UID: \"b67ee4fa-977b-42e9-b192-6ebcc6471bbe\") " pod="openshift-monitoring/node-exporter-t6dps" Jul 10 16:55:41.411623 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:41.411496 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b67ee4fa-977b-42e9-b192-6ebcc6471bbe-node-exporter-wtmp\") pod \"node-exporter-t6dps\" (UID: \"b67ee4fa-977b-42e9-b192-6ebcc6471bbe\") " pod="openshift-monitoring/node-exporter-t6dps" Jul 10 16:55:41.411623 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:41.411547 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b67ee4fa-977b-42e9-b192-6ebcc6471bbe-metrics-client-ca\") pod \"node-exporter-t6dps\" (UID: \"b67ee4fa-977b-42e9-b192-6ebcc6471bbe\") " pod="openshift-monitoring/node-exporter-t6dps" Jul 10 16:55:41.411623 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:41.411571 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b67ee4fa-977b-42e9-b192-6ebcc6471bbe-root\") pod \"node-exporter-t6dps\" (UID: \"b67ee4fa-977b-42e9-b192-6ebcc6471bbe\") " pod="openshift-monitoring/node-exporter-t6dps" Jul 10 16:55:41.411623 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:55:41.411597 2578 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 10 16:55:41.411623 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:41.411610 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b67ee4fa-977b-42e9-b192-6ebcc6471bbe-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-t6dps\" (UID: \"b67ee4fa-977b-42e9-b192-6ebcc6471bbe\") " pod="openshift-monitoring/node-exporter-t6dps" Jul 10 16:55:41.411952 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:55:41.411670 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b67ee4fa-977b-42e9-b192-6ebcc6471bbe-node-exporter-tls podName:b67ee4fa-977b-42e9-b192-6ebcc6471bbe nodeName:}" failed. No retries permitted until 2026-07-10 16:55:41.911649292 +0000 UTC m=+86.000978327 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/b67ee4fa-977b-42e9-b192-6ebcc6471bbe-node-exporter-tls") pod "node-exporter-t6dps" (UID: "b67ee4fa-977b-42e9-b192-6ebcc6471bbe") : secret "node-exporter-tls" not found Jul 10 16:55:41.411952 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:41.411691 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b67ee4fa-977b-42e9-b192-6ebcc6471bbe-root\") pod \"node-exporter-t6dps\" (UID: \"b67ee4fa-977b-42e9-b192-6ebcc6471bbe\") " pod="openshift-monitoring/node-exporter-t6dps" Jul 10 16:55:41.411952 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:41.411810 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b67ee4fa-977b-42e9-b192-6ebcc6471bbe-node-exporter-textfile\") pod \"node-exporter-t6dps\" (UID: \"b67ee4fa-977b-42e9-b192-6ebcc6471bbe\") " pod="openshift-monitoring/node-exporter-t6dps" Jul 10 16:55:41.411952 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:41.411839 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b67ee4fa-977b-42e9-b192-6ebcc6471bbe-node-exporter-wtmp\") pod \"node-exporter-t6dps\" (UID: \"b67ee4fa-977b-42e9-b192-6ebcc6471bbe\") " pod="openshift-monitoring/node-exporter-t6dps" Jul 10 16:55:41.411952 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:41.411942 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b67ee4fa-977b-42e9-b192-6ebcc6471bbe-node-exporter-accelerators-collector-config\") pod \"node-exporter-t6dps\" (UID: \"b67ee4fa-977b-42e9-b192-6ebcc6471bbe\") " pod="openshift-monitoring/node-exporter-t6dps" Jul 10 16:55:41.412139 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:41.412001 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b67ee4fa-977b-42e9-b192-6ebcc6471bbe-metrics-client-ca\") pod \"node-exporter-t6dps\" (UID: \"b67ee4fa-977b-42e9-b192-6ebcc6471bbe\") " pod="openshift-monitoring/node-exporter-t6dps" Jul 10 16:55:41.413957 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:41.413934 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b67ee4fa-977b-42e9-b192-6ebcc6471bbe-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-t6dps\" (UID: \"b67ee4fa-977b-42e9-b192-6ebcc6471bbe\") " pod="openshift-monitoring/node-exporter-t6dps" Jul 10 16:55:41.423240 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:41.423212 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hncnm\" (UniqueName: \"kubernetes.io/projected/b67ee4fa-977b-42e9-b192-6ebcc6471bbe-kube-api-access-hncnm\") pod \"node-exporter-t6dps\" (UID: \"b67ee4fa-977b-42e9-b192-6ebcc6471bbe\") " pod="openshift-monitoring/node-exporter-t6dps" Jul 10 16:55:41.916002 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:41.915963 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b67ee4fa-977b-42e9-b192-6ebcc6471bbe-node-exporter-tls\") pod \"node-exporter-t6dps\" (UID: \"b67ee4fa-977b-42e9-b192-6ebcc6471bbe\") " pod="openshift-monitoring/node-exporter-t6dps" Jul 10 16:55:41.918508 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:41.918481 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b67ee4fa-977b-42e9-b192-6ebcc6471bbe-node-exporter-tls\") pod \"node-exporter-t6dps\" (UID: \"b67ee4fa-977b-42e9-b192-6ebcc6471bbe\") " pod="openshift-monitoring/node-exporter-t6dps" Jul 10 16:55:41.968948 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:41.968915 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-g5nt8" Jul 10 16:55:42.133903 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:42.133873 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-t6dps" Jul 10 16:55:42.145451 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:55:42.145420 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb67ee4fa_977b_42e9_b192_6ebcc6471bbe.slice/crio-075483b26322eb4b8ab91484532de2e519927c47bdcc932fd1cb5236e934c31d WatchSource:0}: Error finding container 075483b26322eb4b8ab91484532de2e519927c47bdcc932fd1cb5236e934c31d: Status 404 returned error can't find the container with id 075483b26322eb4b8ab91484532de2e519927c47bdcc932fd1cb5236e934c31d Jul 10 16:55:42.258174 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:42.258092 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 10 16:55:42.262404 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:42.262376 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 10 16:55:42.268425 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:42.268403 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 10 16:55:42.268558 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:42.268409 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 10 16:55:42.268669 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:42.268409 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 10 16:55:42.268791 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:42.268467 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 10 16:55:42.268791 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:42.268787 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-bw2t6\"" Jul 10 16:55:42.268887 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:42.268853 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 10 16:55:42.268995 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:42.268980 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 10 16:55:42.269088 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:42.269072 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 10 16:55:42.269171 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:42.269159 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 10 16:55:42.269394 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:42.269376 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 10 16:55:42.279799 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:42.279777 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 10 16:55:42.420378 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:42.420334 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4f4e7564-e2a1-4b31-9240-57bedb05904a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4f4e7564-e2a1-4b31-9240-57bedb05904a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 16:55:42.420520 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:42.420390 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4f4e7564-e2a1-4b31-9240-57bedb05904a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4f4e7564-e2a1-4b31-9240-57bedb05904a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 16:55:42.420520 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:42.420421 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4f4e7564-e2a1-4b31-9240-57bedb05904a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4f4e7564-e2a1-4b31-9240-57bedb05904a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 16:55:42.420520 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:42.420476 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4f4e7564-e2a1-4b31-9240-57bedb05904a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4f4e7564-e2a1-4b31-9240-57bedb05904a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 16:55:42.420634 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:42.420515 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f4e7564-e2a1-4b31-9240-57bedb05904a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4f4e7564-e2a1-4b31-9240-57bedb05904a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 16:55:42.420634 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:42.420578 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4f4e7564-e2a1-4b31-9240-57bedb05904a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4f4e7564-e2a1-4b31-9240-57bedb05904a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 16:55:42.420693 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:42.420643 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55zpc\" (UniqueName: \"kubernetes.io/projected/4f4e7564-e2a1-4b31-9240-57bedb05904a-kube-api-access-55zpc\") pod \"alertmanager-main-0\" (UID: \"4f4e7564-e2a1-4b31-9240-57bedb05904a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 16:55:42.420693 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:42.420670 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4f4e7564-e2a1-4b31-9240-57bedb05904a-config-out\") pod \"alertmanager-main-0\" (UID: \"4f4e7564-e2a1-4b31-9240-57bedb05904a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 16:55:42.420765 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:42.420708 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4f4e7564-e2a1-4b31-9240-57bedb05904a-web-config\") pod \"alertmanager-main-0\" (UID: \"4f4e7564-e2a1-4b31-9240-57bedb05904a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 16:55:42.420765 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:42.420750 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4f4e7564-e2a1-4b31-9240-57bedb05904a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4f4e7564-e2a1-4b31-9240-57bedb05904a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 16:55:42.420825 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:42.420776 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f4e7564-e2a1-4b31-9240-57bedb05904a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4f4e7564-e2a1-4b31-9240-57bedb05904a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 16:55:42.420825 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:42.420814 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4f4e7564-e2a1-4b31-9240-57bedb05904a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4f4e7564-e2a1-4b31-9240-57bedb05904a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 16:55:42.420881 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:42.420857 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4f4e7564-e2a1-4b31-9240-57bedb05904a-config-volume\") pod \"alertmanager-main-0\" (UID: \"4f4e7564-e2a1-4b31-9240-57bedb05904a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 16:55:42.521682 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:42.521586 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4f4e7564-e2a1-4b31-9240-57bedb05904a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4f4e7564-e2a1-4b31-9240-57bedb05904a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 16:55:42.521682 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:42.521640 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4f4e7564-e2a1-4b31-9240-57bedb05904a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4f4e7564-e2a1-4b31-9240-57bedb05904a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 16:55:42.521682 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:42.521668 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4f4e7564-e2a1-4b31-9240-57bedb05904a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4f4e7564-e2a1-4b31-9240-57bedb05904a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 16:55:42.521950 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:42.521696 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4f4e7564-e2a1-4b31-9240-57bedb05904a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4f4e7564-e2a1-4b31-9240-57bedb05904a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 16:55:42.521950 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:42.521726 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f4e7564-e2a1-4b31-9240-57bedb05904a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4f4e7564-e2a1-4b31-9240-57bedb05904a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 16:55:42.521950 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:42.521754 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4f4e7564-e2a1-4b31-9240-57bedb05904a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4f4e7564-e2a1-4b31-9240-57bedb05904a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 16:55:42.521950 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:42.521784 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55zpc\" (UniqueName: \"kubernetes.io/projected/4f4e7564-e2a1-4b31-9240-57bedb05904a-kube-api-access-55zpc\") pod \"alertmanager-main-0\" (UID: \"4f4e7564-e2a1-4b31-9240-57bedb05904a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 16:55:42.521950 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:42.521814 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4f4e7564-e2a1-4b31-9240-57bedb05904a-config-out\") pod \"alertmanager-main-0\" (UID: \"4f4e7564-e2a1-4b31-9240-57bedb05904a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 16:55:42.521950 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:42.521849 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4f4e7564-e2a1-4b31-9240-57bedb05904a-web-config\") pod \"alertmanager-main-0\" (UID: \"4f4e7564-e2a1-4b31-9240-57bedb05904a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 16:55:42.521950 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:42.521880 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4f4e7564-e2a1-4b31-9240-57bedb05904a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4f4e7564-e2a1-4b31-9240-57bedb05904a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 16:55:42.521950 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:42.521908 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f4e7564-e2a1-4b31-9240-57bedb05904a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4f4e7564-e2a1-4b31-9240-57bedb05904a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 16:55:42.521950 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:42.521936 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4f4e7564-e2a1-4b31-9240-57bedb05904a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4f4e7564-e2a1-4b31-9240-57bedb05904a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 16:55:42.522417 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:42.521983 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4f4e7564-e2a1-4b31-9240-57bedb05904a-config-volume\") pod \"alertmanager-main-0\" (UID: \"4f4e7564-e2a1-4b31-9240-57bedb05904a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 16:55:42.522417 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:42.522089 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4f4e7564-e2a1-4b31-9240-57bedb05904a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4f4e7564-e2a1-4b31-9240-57bedb05904a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 16:55:42.522518 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:42.522504 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f4e7564-e2a1-4b31-9240-57bedb05904a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4f4e7564-e2a1-4b31-9240-57bedb05904a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 16:55:42.522994 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:55:42.522973 2578 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 10 16:55:42.523382 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:55:42.523366 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f4e7564-e2a1-4b31-9240-57bedb05904a-secret-alertmanager-main-tls podName:4f4e7564-e2a1-4b31-9240-57bedb05904a nodeName:}" failed. No retries permitted until 2026-07-10 16:55:43.023300964 +0000 UTC m=+87.112629993 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/4f4e7564-e2a1-4b31-9240-57bedb05904a-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "4f4e7564-e2a1-4b31-9240-57bedb05904a") : secret "alertmanager-main-tls" not found Jul 10 16:55:42.524308 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:42.524286 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f4e7564-e2a1-4b31-9240-57bedb05904a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4f4e7564-e2a1-4b31-9240-57bedb05904a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 16:55:42.524900 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:42.524871 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4f4e7564-e2a1-4b31-9240-57bedb05904a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4f4e7564-e2a1-4b31-9240-57bedb05904a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 16:55:42.525458 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:42.525432 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4f4e7564-e2a1-4b31-9240-57bedb05904a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4f4e7564-e2a1-4b31-9240-57bedb05904a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 16:55:42.525567 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:42.525539 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4f4e7564-e2a1-4b31-9240-57bedb05904a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4f4e7564-e2a1-4b31-9240-57bedb05904a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 16:55:42.525991 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:42.525967 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4f4e7564-e2a1-4b31-9240-57bedb05904a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4f4e7564-e2a1-4b31-9240-57bedb05904a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 16:55:42.526229 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:42.526203 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4f4e7564-e2a1-4b31-9240-57bedb05904a-config-out\") pod \"alertmanager-main-0\" (UID: \"4f4e7564-e2a1-4b31-9240-57bedb05904a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 16:55:42.526534 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:42.526514 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4f4e7564-e2a1-4b31-9240-57bedb05904a-config-volume\") pod \"alertmanager-main-0\" (UID: \"4f4e7564-e2a1-4b31-9240-57bedb05904a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 16:55:42.526864 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:42.526845 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4f4e7564-e2a1-4b31-9240-57bedb05904a-web-config\") pod \"alertmanager-main-0\" (UID: \"4f4e7564-e2a1-4b31-9240-57bedb05904a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 16:55:42.527001 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:42.526985 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4f4e7564-e2a1-4b31-9240-57bedb05904a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4f4e7564-e2a1-4b31-9240-57bedb05904a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 16:55:42.538296 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:42.538091 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-55zpc\" (UniqueName: \"kubernetes.io/projected/4f4e7564-e2a1-4b31-9240-57bedb05904a-kube-api-access-55zpc\") pod \"alertmanager-main-0\" (UID: \"4f4e7564-e2a1-4b31-9240-57bedb05904a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 16:55:43.012178 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:43.012135 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t6dps" event={"ID":"b67ee4fa-977b-42e9-b192-6ebcc6471bbe","Type":"ContainerStarted","Data":"075483b26322eb4b8ab91484532de2e519927c47bdcc932fd1cb5236e934c31d"} Jul 10 16:55:43.027512 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:43.027474 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4f4e7564-e2a1-4b31-9240-57bedb05904a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4f4e7564-e2a1-4b31-9240-57bedb05904a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 16:55:43.030381 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:43.030349 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4f4e7564-e2a1-4b31-9240-57bedb05904a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4f4e7564-e2a1-4b31-9240-57bedb05904a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 10 16:55:43.172899 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:43.172866 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 10 16:55:43.318525 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:43.318498 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 10 16:55:43.347941 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:55:43.347913 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f4e7564_e2a1_4b31_9240_57bedb05904a.slice/crio-3c4810a6a0d707f8299ef6e9386c11e711f87ab8f73ecbcd727b865efd5f91a8 WatchSource:0}: Error finding container 3c4810a6a0d707f8299ef6e9386c11e711f87ab8f73ecbcd727b865efd5f91a8: Status 404 returned error can't find the container with id 3c4810a6a0d707f8299ef6e9386c11e711f87ab8f73ecbcd727b865efd5f91a8 Jul 10 16:55:43.374388 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:43.374360 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-569fc6bfdb-d5n28"] Jul 10 16:55:43.376804 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:43.376783 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-569fc6bfdb-d5n28" Jul 10 16:55:43.379989 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:43.379525 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 10 16:55:43.379989 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:43.379582 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 10 16:55:43.379989 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:43.379822 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 10 16:55:43.379989 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:43.379847 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 10 16:55:43.379989 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:43.379874 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-6jlpk\"" Jul 10 16:55:43.380309 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:43.380082 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 10 16:55:43.380309 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:43.380109 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 10 16:55:43.380309 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:43.380242 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 10 16:55:43.387514 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:43.387492 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-569fc6bfdb-d5n28"] Jul 10 16:55:43.532435 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:43.532408 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/16ef858a-cf52-4c85-a4f7-fe382ade2581-oauth-serving-cert\") pod \"console-569fc6bfdb-d5n28\" (UID: \"16ef858a-cf52-4c85-a4f7-fe382ade2581\") " pod="openshift-console/console-569fc6bfdb-d5n28" Jul 10 16:55:43.532543 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:43.532460 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/16ef858a-cf52-4c85-a4f7-fe382ade2581-console-oauth-config\") pod \"console-569fc6bfdb-d5n28\" (UID: \"16ef858a-cf52-4c85-a4f7-fe382ade2581\") " pod="openshift-console/console-569fc6bfdb-d5n28" Jul 10 16:55:43.532543 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:43.532503 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/16ef858a-cf52-4c85-a4f7-fe382ade2581-console-config\") pod \"console-569fc6bfdb-d5n28\" (UID: \"16ef858a-cf52-4c85-a4f7-fe382ade2581\") " pod="openshift-console/console-569fc6bfdb-d5n28" Jul 10 16:55:43.532543 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:43.532529 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hmb8\" (UniqueName: \"kubernetes.io/projected/16ef858a-cf52-4c85-a4f7-fe382ade2581-kube-api-access-4hmb8\") pod \"console-569fc6bfdb-d5n28\" (UID: \"16ef858a-cf52-4c85-a4f7-fe382ade2581\") " pod="openshift-console/console-569fc6bfdb-d5n28" Jul 10 16:55:43.532638 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:43.532586 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/16ef858a-cf52-4c85-a4f7-fe382ade2581-console-serving-cert\") pod \"console-569fc6bfdb-d5n28\" (UID: \"16ef858a-cf52-4c85-a4f7-fe382ade2581\") " pod="openshift-console/console-569fc6bfdb-d5n28" Jul 10 16:55:43.532638 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:43.532625 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/16ef858a-cf52-4c85-a4f7-fe382ade2581-service-ca\") pod \"console-569fc6bfdb-d5n28\" (UID: \"16ef858a-cf52-4c85-a4f7-fe382ade2581\") " pod="openshift-console/console-569fc6bfdb-d5n28" Jul 10 16:55:43.633692 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:43.633604 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/16ef858a-cf52-4c85-a4f7-fe382ade2581-oauth-serving-cert\") pod \"console-569fc6bfdb-d5n28\" (UID: \"16ef858a-cf52-4c85-a4f7-fe382ade2581\") " pod="openshift-console/console-569fc6bfdb-d5n28" Jul 10 16:55:43.633692 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:43.633647 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/16ef858a-cf52-4c85-a4f7-fe382ade2581-console-oauth-config\") pod \"console-569fc6bfdb-d5n28\" (UID: \"16ef858a-cf52-4c85-a4f7-fe382ade2581\") " pod="openshift-console/console-569fc6bfdb-d5n28" Jul 10 16:55:43.633692 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:43.633673 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/16ef858a-cf52-4c85-a4f7-fe382ade2581-console-config\") pod \"console-569fc6bfdb-d5n28\" (UID: \"16ef858a-cf52-4c85-a4f7-fe382ade2581\") " pod="openshift-console/console-569fc6bfdb-d5n28" Jul 10 16:55:43.633692 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:43.633691 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hmb8\" (UniqueName: \"kubernetes.io/projected/16ef858a-cf52-4c85-a4f7-fe382ade2581-kube-api-access-4hmb8\") pod \"console-569fc6bfdb-d5n28\" (UID: \"16ef858a-cf52-4c85-a4f7-fe382ade2581\") " pod="openshift-console/console-569fc6bfdb-d5n28" Jul 10 16:55:43.634004 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:43.633825 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/16ef858a-cf52-4c85-a4f7-fe382ade2581-console-serving-cert\") pod \"console-569fc6bfdb-d5n28\" (UID: \"16ef858a-cf52-4c85-a4f7-fe382ade2581\") " pod="openshift-console/console-569fc6bfdb-d5n28" Jul 10 16:55:43.634004 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:43.633868 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/16ef858a-cf52-4c85-a4f7-fe382ade2581-service-ca\") pod \"console-569fc6bfdb-d5n28\" (UID: \"16ef858a-cf52-4c85-a4f7-fe382ade2581\") " pod="openshift-console/console-569fc6bfdb-d5n28" Jul 10 16:55:43.634417 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:43.634392 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/16ef858a-cf52-4c85-a4f7-fe382ade2581-console-config\") pod \"console-569fc6bfdb-d5n28\" (UID: \"16ef858a-cf52-4c85-a4f7-fe382ade2581\") " pod="openshift-console/console-569fc6bfdb-d5n28" Jul 10 16:55:43.634702 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:43.634511 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/16ef858a-cf52-4c85-a4f7-fe382ade2581-service-ca\") pod \"console-569fc6bfdb-d5n28\" (UID: \"16ef858a-cf52-4c85-a4f7-fe382ade2581\") " pod="openshift-console/console-569fc6bfdb-d5n28" Jul 10 16:55:43.634702 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:43.634511 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/16ef858a-cf52-4c85-a4f7-fe382ade2581-oauth-serving-cert\") pod \"console-569fc6bfdb-d5n28\" (UID: \"16ef858a-cf52-4c85-a4f7-fe382ade2581\") " pod="openshift-console/console-569fc6bfdb-d5n28" Jul 10 16:55:43.636165 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:43.636138 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/16ef858a-cf52-4c85-a4f7-fe382ade2581-console-oauth-config\") pod \"console-569fc6bfdb-d5n28\" (UID: \"16ef858a-cf52-4c85-a4f7-fe382ade2581\") " pod="openshift-console/console-569fc6bfdb-d5n28" Jul 10 16:55:43.636348 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:43.636332 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/16ef858a-cf52-4c85-a4f7-fe382ade2581-console-serving-cert\") pod \"console-569fc6bfdb-d5n28\" (UID: \"16ef858a-cf52-4c85-a4f7-fe382ade2581\") " pod="openshift-console/console-569fc6bfdb-d5n28" Jul 10 16:55:43.642400 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:43.642377 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hmb8\" (UniqueName: \"kubernetes.io/projected/16ef858a-cf52-4c85-a4f7-fe382ade2581-kube-api-access-4hmb8\") pod \"console-569fc6bfdb-d5n28\" (UID: \"16ef858a-cf52-4c85-a4f7-fe382ade2581\") " pod="openshift-console/console-569fc6bfdb-d5n28" Jul 10 16:55:43.687251 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:43.687207 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-569fc6bfdb-d5n28" Jul 10 16:55:43.835191 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:43.835163 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-569fc6bfdb-d5n28"] Jul 10 16:55:43.838367 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:55:43.838337 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16ef858a_cf52_4c85_a4f7_fe382ade2581.slice/crio-41637b1ed17bcc33742b5a7354488b68582dc85f19ef29716cd6b231b19a882a WatchSource:0}: Error finding container 41637b1ed17bcc33742b5a7354488b68582dc85f19ef29716cd6b231b19a882a: Status 404 returned error can't find the container with id 41637b1ed17bcc33742b5a7354488b68582dc85f19ef29716cd6b231b19a882a Jul 10 16:55:44.018676 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:44.018595 2578 generic.go:356] "Generic (PLEG): container finished" podID="b67ee4fa-977b-42e9-b192-6ebcc6471bbe" containerID="fbf3d08d6a81167f7bf9996c777febb135902e175a592c010045c6ad1a89f62c" exitCode=0 Jul 10 16:55:44.018838 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:44.018674 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t6dps" event={"ID":"b67ee4fa-977b-42e9-b192-6ebcc6471bbe","Type":"ContainerDied","Data":"fbf3d08d6a81167f7bf9996c777febb135902e175a592c010045c6ad1a89f62c"} Jul 10 16:55:44.020186 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:44.020163 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-569fc6bfdb-d5n28" event={"ID":"16ef858a-cf52-4c85-a4f7-fe382ade2581","Type":"ContainerStarted","Data":"41637b1ed17bcc33742b5a7354488b68582dc85f19ef29716cd6b231b19a882a"} Jul 10 16:55:44.021507 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:44.021480 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4f4e7564-e2a1-4b31-9240-57bedb05904a","Type":"ContainerStarted","Data":"3c4810a6a0d707f8299ef6e9386c11e711f87ab8f73ecbcd727b865efd5f91a8"} Jul 10 16:55:44.421299 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:44.421167 2578 patch_prober.go:28] interesting pod/image-registry-657d95d548-bj62l container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 10 16:55:44.421299 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:44.421233 2578 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-657d95d548-bj62l" podUID="04e15cd0-bcac-4375-97d5-b135b8ef9997" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 10 16:55:45.027973 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:45.027928 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t6dps" event={"ID":"b67ee4fa-977b-42e9-b192-6ebcc6471bbe","Type":"ContainerStarted","Data":"917bbfe1f37a7a74ff221e0e9597fd8bf57d9eeeeb4b53f94b0f7f1e87d94e3d"} Jul 10 16:55:45.028162 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:45.028008 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t6dps" event={"ID":"b67ee4fa-977b-42e9-b192-6ebcc6471bbe","Type":"ContainerStarted","Data":"11ac7c61fe51891379a419f060937863b110419baaf15ddacf22e2dd3ed26819"} Jul 10 16:55:45.029696 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:45.029667 2578 generic.go:356] "Generic (PLEG): container finished" podID="4f4e7564-e2a1-4b31-9240-57bedb05904a" containerID="1ed5e5171841686131fc4d7f266aa8280713e851c0c2aee44e857eb233c31b4d" exitCode=0 Jul 10 16:55:45.029836 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:45.029749 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4f4e7564-e2a1-4b31-9240-57bedb05904a","Type":"ContainerDied","Data":"1ed5e5171841686131fc4d7f266aa8280713e851c0c2aee44e857eb233c31b4d"} Jul 10 16:55:45.051154 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:45.051097 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-t6dps" podStartSLOduration=2.9823811620000003 podStartE2EDuration="4.051080904s" totalImagesPullingTime="992.225787ms" totalInitContainerRuntime="76.473955ms" isStatefulPod=true podCreationTimestamp="2026-07-10 16:55:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:55:45.049314894 +0000 UTC m=+89.138643945" watchObservedRunningTime="2026-07-10 16:55:45.051080904 +0000 UTC m=+89.140409953" Jul 10 16:55:47.615407 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.615381 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 10 16:55:47.620762 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.618191 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.648755 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.648728 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 10 16:55:47.648920 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.648894 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 10 16:55:47.650024 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.650003 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-hvzgc\"" Jul 10 16:55:47.650153 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.650135 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 10 16:55:47.650478 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.650284 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 10 16:55:47.650478 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.650341 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 10 16:55:47.650478 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.650366 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 10 16:55:47.650478 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.650409 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 10 16:55:47.650478 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.650374 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-11j5b5j568pj0\"" Jul 10 16:55:47.650816 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.650676 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 10 16:55:47.650816 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.650808 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 10 16:55:47.650945 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.650866 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 10 16:55:47.651153 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.651122 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 10 16:55:47.651735 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.651590 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 10 16:55:47.652175 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.652026 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 10 16:55:47.654885 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.654859 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 10 16:55:47.677599 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.677578 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 10 16:55:47.702294 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.702250 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 10 16:55:47.770362 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.770334 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be8c418d-b1dc-44c2-9b59-7b8e673f822a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.770482 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.770380 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/be8c418d-b1dc-44c2-9b59-7b8e673f822a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.770482 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.770402 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/be8c418d-b1dc-44c2-9b59-7b8e673f822a-config-out\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.770482 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.770428 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/be8c418d-b1dc-44c2-9b59-7b8e673f822a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.770646 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.770499 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7pt4\" (UniqueName: \"kubernetes.io/projected/be8c418d-b1dc-44c2-9b59-7b8e673f822a-kube-api-access-r7pt4\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.770646 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.770563 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/be8c418d-b1dc-44c2-9b59-7b8e673f822a-config\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.770646 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.770607 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/be8c418d-b1dc-44c2-9b59-7b8e673f822a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.770646 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.770637 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/be8c418d-b1dc-44c2-9b59-7b8e673f822a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.770825 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.770719 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/be8c418d-b1dc-44c2-9b59-7b8e673f822a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.770825 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.770752 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/be8c418d-b1dc-44c2-9b59-7b8e673f822a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.770825 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.770790 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/be8c418d-b1dc-44c2-9b59-7b8e673f822a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.770825 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.770815 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/be8c418d-b1dc-44c2-9b59-7b8e673f822a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.771040 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.770844 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/be8c418d-b1dc-44c2-9b59-7b8e673f822a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.771040 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.770866 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/be8c418d-b1dc-44c2-9b59-7b8e673f822a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.771040 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.770896 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/be8c418d-b1dc-44c2-9b59-7b8e673f822a-web-config\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.771040 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.770947 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/be8c418d-b1dc-44c2-9b59-7b8e673f822a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.771040 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.770977 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/be8c418d-b1dc-44c2-9b59-7b8e673f822a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.771288 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.771036 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/be8c418d-b1dc-44c2-9b59-7b8e673f822a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.771288 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.771133 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be8c418d-b1dc-44c2-9b59-7b8e673f822a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.771288 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.771157 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be8c418d-b1dc-44c2-9b59-7b8e673f822a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.871636 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.871599 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/be8c418d-b1dc-44c2-9b59-7b8e673f822a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.871636 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.871638 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/be8c418d-b1dc-44c2-9b59-7b8e673f822a-config-out\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.871860 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.871670 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/be8c418d-b1dc-44c2-9b59-7b8e673f822a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.871860 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.871686 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7pt4\" (UniqueName: \"kubernetes.io/projected/be8c418d-b1dc-44c2-9b59-7b8e673f822a-kube-api-access-r7pt4\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.871860 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.871711 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/be8c418d-b1dc-44c2-9b59-7b8e673f822a-config\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.871860 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.871739 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/be8c418d-b1dc-44c2-9b59-7b8e673f822a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.871860 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.871756 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/be8c418d-b1dc-44c2-9b59-7b8e673f822a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.871860 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.871784 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/be8c418d-b1dc-44c2-9b59-7b8e673f822a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.871860 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.871808 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/be8c418d-b1dc-44c2-9b59-7b8e673f822a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.871860 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.871843 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/be8c418d-b1dc-44c2-9b59-7b8e673f822a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.872231 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.871867 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/be8c418d-b1dc-44c2-9b59-7b8e673f822a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.872231 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.871905 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/be8c418d-b1dc-44c2-9b59-7b8e673f822a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.872231 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.871925 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/be8c418d-b1dc-44c2-9b59-7b8e673f822a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.872231 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.871946 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/be8c418d-b1dc-44c2-9b59-7b8e673f822a-web-config\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.872231 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.871974 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/be8c418d-b1dc-44c2-9b59-7b8e673f822a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.872231 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.871997 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/be8c418d-b1dc-44c2-9b59-7b8e673f822a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.872231 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.872026 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/be8c418d-b1dc-44c2-9b59-7b8e673f822a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.872231 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.872123 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be8c418d-b1dc-44c2-9b59-7b8e673f822a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.872231 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.872146 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be8c418d-b1dc-44c2-9b59-7b8e673f822a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.872231 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.872186 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be8c418d-b1dc-44c2-9b59-7b8e673f822a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.872721 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.872681 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/be8c418d-b1dc-44c2-9b59-7b8e673f822a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.873643 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.872862 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be8c418d-b1dc-44c2-9b59-7b8e673f822a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.873643 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.873359 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be8c418d-b1dc-44c2-9b59-7b8e673f822a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.877275 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.875708 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/be8c418d-b1dc-44c2-9b59-7b8e673f822a-config-out\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.877275 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.875743 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/be8c418d-b1dc-44c2-9b59-7b8e673f822a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.877275 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.876092 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/be8c418d-b1dc-44c2-9b59-7b8e673f822a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.877275 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.876361 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/be8c418d-b1dc-44c2-9b59-7b8e673f822a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.877275 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.876376 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/be8c418d-b1dc-44c2-9b59-7b8e673f822a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.877275 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.876583 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/be8c418d-b1dc-44c2-9b59-7b8e673f822a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.877275 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.876635 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/be8c418d-b1dc-44c2-9b59-7b8e673f822a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.877275 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.877166 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be8c418d-b1dc-44c2-9b59-7b8e673f822a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.877275 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.877094 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/be8c418d-b1dc-44c2-9b59-7b8e673f822a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.878658 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.878422 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/be8c418d-b1dc-44c2-9b59-7b8e673f822a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.883745 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.878963 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/be8c418d-b1dc-44c2-9b59-7b8e673f822a-config\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.883745 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.883257 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/be8c418d-b1dc-44c2-9b59-7b8e673f822a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.884608 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.884582 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/be8c418d-b1dc-44c2-9b59-7b8e673f822a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.885025 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.885001 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/be8c418d-b1dc-44c2-9b59-7b8e673f822a-web-config\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.886953 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.886927 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/be8c418d-b1dc-44c2-9b59-7b8e673f822a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.887084 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.887023 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/be8c418d-b1dc-44c2-9b59-7b8e673f822a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.887302 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.887282 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7pt4\" (UniqueName: \"kubernetes.io/projected/be8c418d-b1dc-44c2-9b59-7b8e673f822a-kube-api-access-r7pt4\") pod \"prometheus-k8s-0\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.932177 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.932132 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:55:47.940334 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:47.940310 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-657d95d548-bj62l" Jul 10 16:55:48.054741 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:48.054696 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-569fc6bfdb-d5n28" event={"ID":"16ef858a-cf52-4c85-a4f7-fe382ade2581","Type":"ContainerStarted","Data":"889d9bce47aaf49fc33c8375512ec6a39a13200522e04447a2808fb1c0a3abae"} Jul 10 16:55:48.063717 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:48.063682 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4f4e7564-e2a1-4b31-9240-57bedb05904a","Type":"ContainerStarted","Data":"9f0a097ed8893130017380f5ad31b01976ac178beacd74d93465278447063f35"} Jul 10 16:55:48.063872 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:48.063767 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4f4e7564-e2a1-4b31-9240-57bedb05904a","Type":"ContainerStarted","Data":"b554e8d66828632dfb462be68f01ce9be916a5947902df361dd2b7ad1f1d66e6"} Jul 10 16:55:48.063872 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:48.063805 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4f4e7564-e2a1-4b31-9240-57bedb05904a","Type":"ContainerStarted","Data":"7bd7406962280714cb85d5ce55f59ace5ca7ef3a5c519a550bd0a99e63e99bd8"} Jul 10 16:55:48.063872 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:48.063843 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4f4e7564-e2a1-4b31-9240-57bedb05904a","Type":"ContainerStarted","Data":"84d693208254d4f9bf74b08ee7a2391f7ba7a770c96ae3e7482cd34f97168f95"} Jul 10 16:55:48.064007 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:48.063898 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4f4e7564-e2a1-4b31-9240-57bedb05904a","Type":"ContainerStarted","Data":"3cec9b37591085582cc600b7d66e6648f5c5b273daee1b523773cfcc75841688"} Jul 10 16:55:48.082578 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:48.082519 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-569fc6bfdb-d5n28" podStartSLOduration=1.47224454 podStartE2EDuration="5.082500952s" totalImagesPullingTime="3.610256412s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:55:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:55:48.080785964 +0000 UTC m=+92.170115015" watchObservedRunningTime="2026-07-10 16:55:48.082500952 +0000 UTC m=+92.171829999" Jul 10 16:55:48.103394 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:48.103368 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 10 16:55:48.105201 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:55:48.105170 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe8c418d_b1dc_44c2_9b59_7b8e673f822a.slice/crio-2bb97f432c93db754b378f119eaf699597b9744bcfc84e425ae442de93da43a3 WatchSource:0}: Error finding container 2bb97f432c93db754b378f119eaf699597b9744bcfc84e425ae442de93da43a3: Status 404 returned error can't find the container with id 2bb97f432c93db754b378f119eaf699597b9744bcfc84e425ae442de93da43a3 Jul 10 16:55:49.068506 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:49.068469 2578 generic.go:356] "Generic (PLEG): container finished" podID="be8c418d-b1dc-44c2-9b59-7b8e673f822a" containerID="9bff15e61ab2f20af2353f41e781d84e88adfef68e00f9f1dc2497bdda5d2375" exitCode=0 Jul 10 16:55:49.068887 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:49.068556 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"be8c418d-b1dc-44c2-9b59-7b8e673f822a","Type":"ContainerDied","Data":"9bff15e61ab2f20af2353f41e781d84e88adfef68e00f9f1dc2497bdda5d2375"} Jul 10 16:55:49.068887 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:49.068638 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"be8c418d-b1dc-44c2-9b59-7b8e673f822a","Type":"ContainerStarted","Data":"2bb97f432c93db754b378f119eaf699597b9744bcfc84e425ae442de93da43a3"} Jul 10 16:55:49.072097 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:49.072066 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4f4e7564-e2a1-4b31-9240-57bedb05904a","Type":"ContainerStarted","Data":"f99d386e8d6d7310399a49cbfa3d81e642ca39a5a36074be5f2eabfa815054eb"} Jul 10 16:55:49.127872 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:49.127785 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.148068741 podStartE2EDuration="7.127765828s" totalImagesPullingTime="3.960091391s" totalInitContainerRuntime="19.605696ms" isStatefulPod=false podCreationTimestamp="2026-07-10 16:55:42 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:55:49.127336326 +0000 UTC m=+93.216665376" watchObservedRunningTime="2026-07-10 16:55:49.127765828 +0000 UTC m=+93.217094878" Jul 10 16:55:52.885204 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:52.885166 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-jvkw6" Jul 10 16:55:53.092944 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:53.092891 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"be8c418d-b1dc-44c2-9b59-7b8e673f822a","Type":"ContainerStarted","Data":"f69bc78775e591cb96b746738dfcb7cb6398ed6fe8c7bfab84f8e9b9d6a2d5a5"} Jul 10 16:55:53.092944 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:53.092977 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"be8c418d-b1dc-44c2-9b59-7b8e673f822a","Type":"ContainerStarted","Data":"221094880b4b9763e20c54092552d25b98b0234b31420459d45590ef599e99d4"} Jul 10 16:55:53.687842 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:53.687802 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-569fc6bfdb-d5n28" Jul 10 16:55:53.688078 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:53.687876 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-569fc6bfdb-d5n28" Jul 10 16:55:53.692981 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:53.692956 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-569fc6bfdb-d5n28" Jul 10 16:55:54.100852 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:54.100825 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"be8c418d-b1dc-44c2-9b59-7b8e673f822a","Type":"ContainerStarted","Data":"45c7105720d4b5f4bffd225882fcb7dfe3ba576af42454033210e77fb3a69c43"} Jul 10 16:55:54.101190 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:54.100904 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"be8c418d-b1dc-44c2-9b59-7b8e673f822a","Type":"ContainerStarted","Data":"44a361bf537e5f93a017ccd402d7c49ad4bdb3fd8fc9e482c2957e0304d752d3"} Jul 10 16:55:54.101190 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:54.100952 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"be8c418d-b1dc-44c2-9b59-7b8e673f822a","Type":"ContainerStarted","Data":"331e92f64ca5f409187e8e4d696bde59498ef92036cffdda86ad1bb78722f063"} Jul 10 16:55:54.104945 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:54.104916 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-569fc6bfdb-d5n28" Jul 10 16:55:54.896462 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:54.896428 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-657d95d548-bj62l"] Jul 10 16:55:55.108891 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:55.108855 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"be8c418d-b1dc-44c2-9b59-7b8e673f822a","Type":"ContainerStarted","Data":"b13e0e6b0e2d1600d8f4331cec0e338a82bf916af0b5310bbb53801bbc7aefcd"} Jul 10 16:55:55.137193 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:55.137135 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.4722739479999998 podStartE2EDuration="8.137113957s" totalImagesPullingTime="4.641692089s" totalInitContainerRuntime="23.14792ms" isStatefulPod=false podCreationTimestamp="2026-07-10 16:55:47 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:55:55.134759399 +0000 UTC m=+99.224088484" watchObservedRunningTime="2026-07-10 16:55:55.137113957 +0000 UTC m=+99.226443008" Jul 10 16:55:57.933024 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:55:57.932987 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:56:03.257912 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:03.257871 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-569fc6bfdb-d5n28"] Jul 10 16:56:19.917509 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:19.917457 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-657d95d548-bj62l" podUID="04e15cd0-bcac-4375-97d5-b135b8ef9997" containerName="registry" containerID="cri-o://20772b6787cfaf262e719a98a0e1614bdb0ce0e6fa3a9ff6a5967b13931fb206" gracePeriod=30 Jul 10 16:56:20.173473 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:20.173409 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-657d95d548-bj62l" Jul 10 16:56:20.211289 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:20.211256 2578 generic.go:356] "Generic (PLEG): container finished" podID="04e15cd0-bcac-4375-97d5-b135b8ef9997" containerID="20772b6787cfaf262e719a98a0e1614bdb0ce0e6fa3a9ff6a5967b13931fb206" exitCode=0 Jul 10 16:56:20.211477 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:20.211296 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-657d95d548-bj62l" Jul 10 16:56:20.211477 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:20.211310 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-657d95d548-bj62l" event={"ID":"04e15cd0-bcac-4375-97d5-b135b8ef9997","Type":"ContainerDied","Data":"20772b6787cfaf262e719a98a0e1614bdb0ce0e6fa3a9ff6a5967b13931fb206"} Jul 10 16:56:20.211477 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:20.211366 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-657d95d548-bj62l" event={"ID":"04e15cd0-bcac-4375-97d5-b135b8ef9997","Type":"ContainerDied","Data":"2861230c20d842f0f83e0932c1d2fc55f926ded28384e4a753c110480436c51e"} Jul 10 16:56:20.211477 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:20.211393 2578 scope.go:117] "RemoveContainer" containerID="20772b6787cfaf262e719a98a0e1614bdb0ce0e6fa3a9ff6a5967b13931fb206" Jul 10 16:56:20.220101 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:20.220076 2578 scope.go:117] "RemoveContainer" containerID="20772b6787cfaf262e719a98a0e1614bdb0ce0e6fa3a9ff6a5967b13931fb206" Jul 10 16:56:20.220390 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:56:20.220367 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20772b6787cfaf262e719a98a0e1614bdb0ce0e6fa3a9ff6a5967b13931fb206\": container with ID starting with 20772b6787cfaf262e719a98a0e1614bdb0ce0e6fa3a9ff6a5967b13931fb206 not found: ID does not exist" containerID="20772b6787cfaf262e719a98a0e1614bdb0ce0e6fa3a9ff6a5967b13931fb206" Jul 10 16:56:20.220478 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:20.220399 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20772b6787cfaf262e719a98a0e1614bdb0ce0e6fa3a9ff6a5967b13931fb206"} err="failed to get container status \"20772b6787cfaf262e719a98a0e1614bdb0ce0e6fa3a9ff6a5967b13931fb206\": rpc error: code = NotFound desc = could not find container \"20772b6787cfaf262e719a98a0e1614bdb0ce0e6fa3a9ff6a5967b13931fb206\": container with ID starting with 20772b6787cfaf262e719a98a0e1614bdb0ce0e6fa3a9ff6a5967b13931fb206 not found: ID does not exist" Jul 10 16:56:20.253450 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:20.253415 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/04e15cd0-bcac-4375-97d5-b135b8ef9997-registry-certificates\") pod \"04e15cd0-bcac-4375-97d5-b135b8ef9997\" (UID: \"04e15cd0-bcac-4375-97d5-b135b8ef9997\") " Jul 10 16:56:20.253450 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:20.253458 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/04e15cd0-bcac-4375-97d5-b135b8ef9997-trusted-ca\") pod \"04e15cd0-bcac-4375-97d5-b135b8ef9997\" (UID: \"04e15cd0-bcac-4375-97d5-b135b8ef9997\") " Jul 10 16:56:20.253695 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:20.253484 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/04e15cd0-bcac-4375-97d5-b135b8ef9997-ca-trust-extracted\") pod \"04e15cd0-bcac-4375-97d5-b135b8ef9997\" (UID: \"04e15cd0-bcac-4375-97d5-b135b8ef9997\") " Jul 10 16:56:20.253695 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:20.253503 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/04e15cd0-bcac-4375-97d5-b135b8ef9997-registry-tls\") pod \"04e15cd0-bcac-4375-97d5-b135b8ef9997\" (UID: \"04e15cd0-bcac-4375-97d5-b135b8ef9997\") " Jul 10 16:56:20.253695 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:20.253542 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/04e15cd0-bcac-4375-97d5-b135b8ef9997-image-registry-private-configuration\") pod \"04e15cd0-bcac-4375-97d5-b135b8ef9997\" (UID: \"04e15cd0-bcac-4375-97d5-b135b8ef9997\") " Jul 10 16:56:20.253849 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:20.253709 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q6xwk\" (UniqueName: \"kubernetes.io/projected/04e15cd0-bcac-4375-97d5-b135b8ef9997-kube-api-access-q6xwk\") pod \"04e15cd0-bcac-4375-97d5-b135b8ef9997\" (UID: \"04e15cd0-bcac-4375-97d5-b135b8ef9997\") " Jul 10 16:56:20.253849 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:20.253749 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/04e15cd0-bcac-4375-97d5-b135b8ef9997-installation-pull-secrets\") pod \"04e15cd0-bcac-4375-97d5-b135b8ef9997\" (UID: \"04e15cd0-bcac-4375-97d5-b135b8ef9997\") " Jul 10 16:56:20.253849 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:20.253781 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/04e15cd0-bcac-4375-97d5-b135b8ef9997-bound-sa-token\") pod \"04e15cd0-bcac-4375-97d5-b135b8ef9997\" (UID: \"04e15cd0-bcac-4375-97d5-b135b8ef9997\") " Jul 10 16:56:20.254005 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:20.253856 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04e15cd0-bcac-4375-97d5-b135b8ef9997-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "04e15cd0-bcac-4375-97d5-b135b8ef9997" (UID: "04e15cd0-bcac-4375-97d5-b135b8ef9997"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 16:56:20.254005 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:20.253889 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04e15cd0-bcac-4375-97d5-b135b8ef9997-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "04e15cd0-bcac-4375-97d5-b135b8ef9997" (UID: "04e15cd0-bcac-4375-97d5-b135b8ef9997"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 16:56:20.254183 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:20.254156 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/04e15cd0-bcac-4375-97d5-b135b8ef9997-trusted-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 16:56:20.254241 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:20.254186 2578 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/04e15cd0-bcac-4375-97d5-b135b8ef9997-registry-certificates\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 16:56:20.256453 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:20.256422 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04e15cd0-bcac-4375-97d5-b135b8ef9997-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "04e15cd0-bcac-4375-97d5-b135b8ef9997" (UID: "04e15cd0-bcac-4375-97d5-b135b8ef9997"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 16:56:20.256668 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:20.256616 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04e15cd0-bcac-4375-97d5-b135b8ef9997-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "04e15cd0-bcac-4375-97d5-b135b8ef9997" (UID: "04e15cd0-bcac-4375-97d5-b135b8ef9997"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 16:56:20.256668 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:20.256636 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04e15cd0-bcac-4375-97d5-b135b8ef9997-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "04e15cd0-bcac-4375-97d5-b135b8ef9997" (UID: "04e15cd0-bcac-4375-97d5-b135b8ef9997"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 16:56:20.256790 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:20.256719 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04e15cd0-bcac-4375-97d5-b135b8ef9997-kube-api-access-q6xwk" (OuterVolumeSpecName: "kube-api-access-q6xwk") pod "04e15cd0-bcac-4375-97d5-b135b8ef9997" (UID: "04e15cd0-bcac-4375-97d5-b135b8ef9997"). InnerVolumeSpecName "kube-api-access-q6xwk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 16:56:20.256790 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:20.256776 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04e15cd0-bcac-4375-97d5-b135b8ef9997-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "04e15cd0-bcac-4375-97d5-b135b8ef9997" (UID: "04e15cd0-bcac-4375-97d5-b135b8ef9997"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 16:56:20.262234 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:20.262204 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04e15cd0-bcac-4375-97d5-b135b8ef9997-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "04e15cd0-bcac-4375-97d5-b135b8ef9997" (UID: "04e15cd0-bcac-4375-97d5-b135b8ef9997"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:56:20.355638 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:20.355582 2578 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/04e15cd0-bcac-4375-97d5-b135b8ef9997-image-registry-private-configuration\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 16:56:20.355638 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:20.355628 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q6xwk\" (UniqueName: \"kubernetes.io/projected/04e15cd0-bcac-4375-97d5-b135b8ef9997-kube-api-access-q6xwk\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 16:56:20.355638 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:20.355638 2578 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/04e15cd0-bcac-4375-97d5-b135b8ef9997-installation-pull-secrets\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 16:56:20.355638 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:20.355647 2578 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/04e15cd0-bcac-4375-97d5-b135b8ef9997-bound-sa-token\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 16:56:20.355638 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:20.355655 2578 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/04e15cd0-bcac-4375-97d5-b135b8ef9997-ca-trust-extracted\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 16:56:20.355937 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:20.355665 2578 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/04e15cd0-bcac-4375-97d5-b135b8ef9997-registry-tls\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 16:56:20.533924 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:20.533891 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-657d95d548-bj62l"] Jul 10 16:56:20.538205 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:20.538175 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-657d95d548-bj62l"] Jul 10 16:56:22.425033 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:22.424994 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04e15cd0-bcac-4375-97d5-b135b8ef9997" path="/var/lib/kubelet/pods/04e15cd0-bcac-4375-97d5-b135b8ef9997/volumes" Jul 10 16:56:28.245025 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:28.244988 2578 generic.go:356] "Generic (PLEG): container finished" podID="73c4ea4d-fc8b-4fca-b4bd-2fc9102d4347" containerID="3e2bdf4d4a69716d1609be8ee0a8ed17c97319d1cccfe84673313904ece865d4" exitCode=0 Jul 10 16:56:28.245515 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:28.245086 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-8r74k" event={"ID":"73c4ea4d-fc8b-4fca-b4bd-2fc9102d4347","Type":"ContainerDied","Data":"3e2bdf4d4a69716d1609be8ee0a8ed17c97319d1cccfe84673313904ece865d4"} Jul 10 16:56:28.245515 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:28.245444 2578 scope.go:117] "RemoveContainer" containerID="3e2bdf4d4a69716d1609be8ee0a8ed17c97319d1cccfe84673313904ece865d4" Jul 10 16:56:28.246769 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:28.246744 2578 generic.go:356] "Generic (PLEG): container finished" podID="ef2eb691-dbbf-4cfc-9fc1-bba4cd71895b" containerID="c57d2934a4aa2732d024f901cb72d3b8e2e516e6e7718ad5793df227bb80818b" exitCode=0 Jul 10 16:56:28.246841 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:28.246815 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-d5bgt" event={"ID":"ef2eb691-dbbf-4cfc-9fc1-bba4cd71895b","Type":"ContainerDied","Data":"c57d2934a4aa2732d024f901cb72d3b8e2e516e6e7718ad5793df227bb80818b"} Jul 10 16:56:28.247149 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:28.247134 2578 scope.go:117] "RemoveContainer" containerID="c57d2934a4aa2732d024f901cb72d3b8e2e516e6e7718ad5793df227bb80818b" Jul 10 16:56:28.280087 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:28.280024 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-569fc6bfdb-d5n28" podUID="16ef858a-cf52-4c85-a4f7-fe382ade2581" containerName="console" containerID="cri-o://889d9bce47aaf49fc33c8375512ec6a39a13200522e04447a2808fb1c0a3abae" gracePeriod=15 Jul 10 16:56:28.538938 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:28.538912 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-569fc6bfdb-d5n28" Jul 10 16:56:28.624240 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:28.624201 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/16ef858a-cf52-4c85-a4f7-fe382ade2581-console-oauth-config\") pod \"16ef858a-cf52-4c85-a4f7-fe382ade2581\" (UID: \"16ef858a-cf52-4c85-a4f7-fe382ade2581\") " Jul 10 16:56:28.624430 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:28.624267 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/16ef858a-cf52-4c85-a4f7-fe382ade2581-oauth-serving-cert\") pod \"16ef858a-cf52-4c85-a4f7-fe382ade2581\" (UID: \"16ef858a-cf52-4c85-a4f7-fe382ade2581\") " Jul 10 16:56:28.624430 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:28.624294 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/16ef858a-cf52-4c85-a4f7-fe382ade2581-console-serving-cert\") pod \"16ef858a-cf52-4c85-a4f7-fe382ade2581\" (UID: \"16ef858a-cf52-4c85-a4f7-fe382ade2581\") " Jul 10 16:56:28.624430 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:28.624353 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4hmb8\" (UniqueName: \"kubernetes.io/projected/16ef858a-cf52-4c85-a4f7-fe382ade2581-kube-api-access-4hmb8\") pod \"16ef858a-cf52-4c85-a4f7-fe382ade2581\" (UID: \"16ef858a-cf52-4c85-a4f7-fe382ade2581\") " Jul 10 16:56:28.624430 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:28.624387 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/16ef858a-cf52-4c85-a4f7-fe382ade2581-service-ca\") pod \"16ef858a-cf52-4c85-a4f7-fe382ade2581\" (UID: \"16ef858a-cf52-4c85-a4f7-fe382ade2581\") " Jul 10 16:56:28.624430 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:28.624429 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/16ef858a-cf52-4c85-a4f7-fe382ade2581-console-config\") pod \"16ef858a-cf52-4c85-a4f7-fe382ade2581\" (UID: \"16ef858a-cf52-4c85-a4f7-fe382ade2581\") " Jul 10 16:56:28.624863 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:28.624755 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16ef858a-cf52-4c85-a4f7-fe382ade2581-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "16ef858a-cf52-4c85-a4f7-fe382ade2581" (UID: "16ef858a-cf52-4c85-a4f7-fe382ade2581"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 16:56:28.624987 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:28.624910 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16ef858a-cf52-4c85-a4f7-fe382ade2581-service-ca" (OuterVolumeSpecName: "service-ca") pod "16ef858a-cf52-4c85-a4f7-fe382ade2581" (UID: "16ef858a-cf52-4c85-a4f7-fe382ade2581"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 16:56:28.625143 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:28.625124 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16ef858a-cf52-4c85-a4f7-fe382ade2581-console-config" (OuterVolumeSpecName: "console-config") pod "16ef858a-cf52-4c85-a4f7-fe382ade2581" (UID: "16ef858a-cf52-4c85-a4f7-fe382ade2581"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 16:56:28.626869 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:28.626841 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16ef858a-cf52-4c85-a4f7-fe382ade2581-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "16ef858a-cf52-4c85-a4f7-fe382ade2581" (UID: "16ef858a-cf52-4c85-a4f7-fe382ade2581"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 16:56:28.626978 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:28.626865 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16ef858a-cf52-4c85-a4f7-fe382ade2581-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "16ef858a-cf52-4c85-a4f7-fe382ade2581" (UID: "16ef858a-cf52-4c85-a4f7-fe382ade2581"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 16:56:28.627181 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:28.627163 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16ef858a-cf52-4c85-a4f7-fe382ade2581-kube-api-access-4hmb8" (OuterVolumeSpecName: "kube-api-access-4hmb8") pod "16ef858a-cf52-4c85-a4f7-fe382ade2581" (UID: "16ef858a-cf52-4c85-a4f7-fe382ade2581"). InnerVolumeSpecName "kube-api-access-4hmb8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 16:56:28.725884 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:28.725774 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/16ef858a-cf52-4c85-a4f7-fe382ade2581-console-oauth-config\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 16:56:28.725884 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:28.725812 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/16ef858a-cf52-4c85-a4f7-fe382ade2581-oauth-serving-cert\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 16:56:28.725884 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:28.725825 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/16ef858a-cf52-4c85-a4f7-fe382ade2581-console-serving-cert\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 16:56:28.725884 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:28.725834 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4hmb8\" (UniqueName: \"kubernetes.io/projected/16ef858a-cf52-4c85-a4f7-fe382ade2581-kube-api-access-4hmb8\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 16:56:28.725884 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:28.725843 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/16ef858a-cf52-4c85-a4f7-fe382ade2581-service-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 16:56:28.725884 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:28.725851 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/16ef858a-cf52-4c85-a4f7-fe382ade2581-console-config\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 16:56:29.257311 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:29.257276 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-8r74k" event={"ID":"73c4ea4d-fc8b-4fca-b4bd-2fc9102d4347","Type":"ContainerStarted","Data":"bf29f3bd85aac466d4aa424a2a22ed319bda058509a51b887ae7c3d57a08746e"} Jul 10 16:56:29.258720 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:29.258694 2578 generic.go:356] "Generic (PLEG): container finished" podID="16ef858a-cf52-4c85-a4f7-fe382ade2581" containerID="889d9bce47aaf49fc33c8375512ec6a39a13200522e04447a2808fb1c0a3abae" exitCode=0 Jul 10 16:56:29.258860 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:29.258729 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-569fc6bfdb-d5n28" Jul 10 16:56:29.258860 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:29.258778 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-569fc6bfdb-d5n28" event={"ID":"16ef858a-cf52-4c85-a4f7-fe382ade2581","Type":"ContainerDied","Data":"889d9bce47aaf49fc33c8375512ec6a39a13200522e04447a2808fb1c0a3abae"} Jul 10 16:56:29.258860 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:29.258833 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-569fc6bfdb-d5n28" event={"ID":"16ef858a-cf52-4c85-a4f7-fe382ade2581","Type":"ContainerDied","Data":"41637b1ed17bcc33742b5a7354488b68582dc85f19ef29716cd6b231b19a882a"} Jul 10 16:56:29.258860 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:29.258860 2578 scope.go:117] "RemoveContainer" containerID="889d9bce47aaf49fc33c8375512ec6a39a13200522e04447a2808fb1c0a3abae" Jul 10 16:56:29.264541 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:29.262415 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-d5bgt" event={"ID":"ef2eb691-dbbf-4cfc-9fc1-bba4cd71895b","Type":"ContainerStarted","Data":"6992f7f8b158270923f3a668da31189c3a60c5b8effb5fd3e41b5c8491125c3a"} Jul 10 16:56:29.277199 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:29.277161 2578 scope.go:117] "RemoveContainer" containerID="889d9bce47aaf49fc33c8375512ec6a39a13200522e04447a2808fb1c0a3abae" Jul 10 16:56:29.277618 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:56:29.277591 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"889d9bce47aaf49fc33c8375512ec6a39a13200522e04447a2808fb1c0a3abae\": container with ID starting with 889d9bce47aaf49fc33c8375512ec6a39a13200522e04447a2808fb1c0a3abae not found: ID does not exist" containerID="889d9bce47aaf49fc33c8375512ec6a39a13200522e04447a2808fb1c0a3abae" Jul 10 16:56:29.277692 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:29.277635 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"889d9bce47aaf49fc33c8375512ec6a39a13200522e04447a2808fb1c0a3abae"} err="failed to get container status \"889d9bce47aaf49fc33c8375512ec6a39a13200522e04447a2808fb1c0a3abae\": rpc error: code = NotFound desc = could not find container \"889d9bce47aaf49fc33c8375512ec6a39a13200522e04447a2808fb1c0a3abae\": container with ID starting with 889d9bce47aaf49fc33c8375512ec6a39a13200522e04447a2808fb1c0a3abae not found: ID does not exist" Jul 10 16:56:29.292016 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:29.291978 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-569fc6bfdb-d5n28"] Jul 10 16:56:29.298623 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:29.298590 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-569fc6bfdb-d5n28"] Jul 10 16:56:30.424834 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:30.424792 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16ef858a-cf52-4c85-a4f7-fe382ade2581" path="/var/lib/kubelet/pods/16ef858a-cf52-4c85-a4f7-fe382ade2581/volumes" Jul 10 16:56:33.280516 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:33.280479 2578 generic.go:356] "Generic (PLEG): container finished" podID="99905b79-72f4-478d-9c3a-05967f561ee2" containerID="8908e12948680a50a9b9964d60704482611fdecf428dfe6a760b1e2497f78404" exitCode=0 Jul 10 16:56:33.280897 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:33.280561 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-gqqc7" event={"ID":"99905b79-72f4-478d-9c3a-05967f561ee2","Type":"ContainerDied","Data":"8908e12948680a50a9b9964d60704482611fdecf428dfe6a760b1e2497f78404"} Jul 10 16:56:33.280897 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:33.280893 2578 scope.go:117] "RemoveContainer" containerID="8908e12948680a50a9b9964d60704482611fdecf428dfe6a760b1e2497f78404" Jul 10 16:56:34.288000 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:34.287963 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-gqqc7" event={"ID":"99905b79-72f4-478d-9c3a-05967f561ee2","Type":"ContainerStarted","Data":"70a5625c00f609a2ab2a338ac5e7c9f75f4fd3fc04c93ea748ad7bef94a106f8"} Jul 10 16:56:47.932902 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:47.932862 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:56:47.949890 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:47.949859 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:56:48.366583 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:56:48.366555 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:01.597157 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:01.597114 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 10 16:57:01.597652 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:01.597476 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4f4e7564-e2a1-4b31-9240-57bedb05904a" containerName="alertmanager" containerID="cri-o://3cec9b37591085582cc600b7d66e6648f5c5b273daee1b523773cfcc75841688" gracePeriod=120 Jul 10 16:57:01.597652 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:01.597529 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4f4e7564-e2a1-4b31-9240-57bedb05904a" containerName="kube-rbac-proxy-metric" containerID="cri-o://9f0a097ed8893130017380f5ad31b01976ac178beacd74d93465278447063f35" gracePeriod=120 Jul 10 16:57:01.597652 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:01.597596 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4f4e7564-e2a1-4b31-9240-57bedb05904a" containerName="config-reloader" containerID="cri-o://84d693208254d4f9bf74b08ee7a2391f7ba7a770c96ae3e7482cd34f97168f95" gracePeriod=120 Jul 10 16:57:01.597825 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:01.597625 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4f4e7564-e2a1-4b31-9240-57bedb05904a" containerName="kube-rbac-proxy" containerID="cri-o://b554e8d66828632dfb462be68f01ce9be916a5947902df361dd2b7ad1f1d66e6" gracePeriod=120 Jul 10 16:57:01.597825 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:01.597558 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4f4e7564-e2a1-4b31-9240-57bedb05904a" containerName="kube-rbac-proxy-web" containerID="cri-o://7bd7406962280714cb85d5ce55f59ace5ca7ef3a5c519a550bd0a99e63e99bd8" gracePeriod=120 Jul 10 16:57:01.597825 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:01.597625 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4f4e7564-e2a1-4b31-9240-57bedb05904a" containerName="prom-label-proxy" containerID="cri-o://f99d386e8d6d7310399a49cbfa3d81e642ca39a5a36074be5f2eabfa815054eb" gracePeriod=120 Jul 10 16:57:02.413478 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:02.413441 2578 generic.go:356] "Generic (PLEG): container finished" podID="4f4e7564-e2a1-4b31-9240-57bedb05904a" containerID="f99d386e8d6d7310399a49cbfa3d81e642ca39a5a36074be5f2eabfa815054eb" exitCode=0 Jul 10 16:57:02.413478 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:02.413466 2578 generic.go:356] "Generic (PLEG): container finished" podID="4f4e7564-e2a1-4b31-9240-57bedb05904a" containerID="9f0a097ed8893130017380f5ad31b01976ac178beacd74d93465278447063f35" exitCode=0 Jul 10 16:57:02.413478 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:02.413472 2578 generic.go:356] "Generic (PLEG): container finished" podID="4f4e7564-e2a1-4b31-9240-57bedb05904a" containerID="b554e8d66828632dfb462be68f01ce9be916a5947902df361dd2b7ad1f1d66e6" exitCode=0 Jul 10 16:57:02.413478 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:02.413477 2578 generic.go:356] "Generic (PLEG): container finished" podID="4f4e7564-e2a1-4b31-9240-57bedb05904a" containerID="84d693208254d4f9bf74b08ee7a2391f7ba7a770c96ae3e7482cd34f97168f95" exitCode=0 Jul 10 16:57:02.413478 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:02.413482 2578 generic.go:356] "Generic (PLEG): container finished" podID="4f4e7564-e2a1-4b31-9240-57bedb05904a" containerID="3cec9b37591085582cc600b7d66e6648f5c5b273daee1b523773cfcc75841688" exitCode=0 Jul 10 16:57:02.413835 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:02.413512 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4f4e7564-e2a1-4b31-9240-57bedb05904a","Type":"ContainerDied","Data":"f99d386e8d6d7310399a49cbfa3d81e642ca39a5a36074be5f2eabfa815054eb"} Jul 10 16:57:02.413835 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:02.413594 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4f4e7564-e2a1-4b31-9240-57bedb05904a","Type":"ContainerDied","Data":"9f0a097ed8893130017380f5ad31b01976ac178beacd74d93465278447063f35"} Jul 10 16:57:02.413835 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:02.413630 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4f4e7564-e2a1-4b31-9240-57bedb05904a","Type":"ContainerDied","Data":"b554e8d66828632dfb462be68f01ce9be916a5947902df361dd2b7ad1f1d66e6"} Jul 10 16:57:02.413835 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:02.413667 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4f4e7564-e2a1-4b31-9240-57bedb05904a","Type":"ContainerDied","Data":"84d693208254d4f9bf74b08ee7a2391f7ba7a770c96ae3e7482cd34f97168f95"} Jul 10 16:57:02.413835 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:02.413703 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4f4e7564-e2a1-4b31-9240-57bedb05904a","Type":"ContainerDied","Data":"3cec9b37591085582cc600b7d66e6648f5c5b273daee1b523773cfcc75841688"} Jul 10 16:57:02.865225 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:02.865194 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 10 16:57:02.943153 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:02.942976 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4f4e7564-e2a1-4b31-9240-57bedb05904a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"4f4e7564-e2a1-4b31-9240-57bedb05904a\" (UID: \"4f4e7564-e2a1-4b31-9240-57bedb05904a\") " Jul 10 16:57:02.943153 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:02.943019 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f4e7564-e2a1-4b31-9240-57bedb05904a-alertmanager-trusted-ca-bundle\") pod \"4f4e7564-e2a1-4b31-9240-57bedb05904a\" (UID: \"4f4e7564-e2a1-4b31-9240-57bedb05904a\") " Jul 10 16:57:02.943153 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:02.943063 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4f4e7564-e2a1-4b31-9240-57bedb05904a-cluster-tls-config\") pod \"4f4e7564-e2a1-4b31-9240-57bedb05904a\" (UID: \"4f4e7564-e2a1-4b31-9240-57bedb05904a\") " Jul 10 16:57:02.943153 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:02.943095 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4f4e7564-e2a1-4b31-9240-57bedb05904a-config-out\") pod \"4f4e7564-e2a1-4b31-9240-57bedb05904a\" (UID: \"4f4e7564-e2a1-4b31-9240-57bedb05904a\") " Jul 10 16:57:02.943534 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:02.943503 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f4e7564-e2a1-4b31-9240-57bedb05904a-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "4f4e7564-e2a1-4b31-9240-57bedb05904a" (UID: "4f4e7564-e2a1-4b31-9240-57bedb05904a"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 16:57:02.943594 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:02.943554 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4f4e7564-e2a1-4b31-9240-57bedb05904a-secret-alertmanager-kube-rbac-proxy-web\") pod \"4f4e7564-e2a1-4b31-9240-57bedb05904a\" (UID: \"4f4e7564-e2a1-4b31-9240-57bedb05904a\") " Jul 10 16:57:02.943653 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:02.943608 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f4e7564-e2a1-4b31-9240-57bedb05904a-metrics-client-ca\") pod \"4f4e7564-e2a1-4b31-9240-57bedb05904a\" (UID: \"4f4e7564-e2a1-4b31-9240-57bedb05904a\") " Jul 10 16:57:02.943653 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:02.943644 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-55zpc\" (UniqueName: \"kubernetes.io/projected/4f4e7564-e2a1-4b31-9240-57bedb05904a-kube-api-access-55zpc\") pod \"4f4e7564-e2a1-4b31-9240-57bedb05904a\" (UID: \"4f4e7564-e2a1-4b31-9240-57bedb05904a\") " Jul 10 16:57:02.943755 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:02.943681 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4f4e7564-e2a1-4b31-9240-57bedb05904a-config-volume\") pod \"4f4e7564-e2a1-4b31-9240-57bedb05904a\" (UID: \"4f4e7564-e2a1-4b31-9240-57bedb05904a\") " Jul 10 16:57:02.943755 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:02.943713 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4f4e7564-e2a1-4b31-9240-57bedb05904a-secret-alertmanager-kube-rbac-proxy\") pod \"4f4e7564-e2a1-4b31-9240-57bedb05904a\" (UID: \"4f4e7564-e2a1-4b31-9240-57bedb05904a\") " Jul 10 16:57:02.943755 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:02.943750 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4f4e7564-e2a1-4b31-9240-57bedb05904a-web-config\") pod \"4f4e7564-e2a1-4b31-9240-57bedb05904a\" (UID: \"4f4e7564-e2a1-4b31-9240-57bedb05904a\") " Jul 10 16:57:02.943910 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:02.943807 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4f4e7564-e2a1-4b31-9240-57bedb05904a-tls-assets\") pod \"4f4e7564-e2a1-4b31-9240-57bedb05904a\" (UID: \"4f4e7564-e2a1-4b31-9240-57bedb05904a\") " Jul 10 16:57:02.943910 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:02.943857 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4f4e7564-e2a1-4b31-9240-57bedb05904a-alertmanager-main-db\") pod \"4f4e7564-e2a1-4b31-9240-57bedb05904a\" (UID: \"4f4e7564-e2a1-4b31-9240-57bedb05904a\") " Jul 10 16:57:02.943910 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:02.943882 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4f4e7564-e2a1-4b31-9240-57bedb05904a-secret-alertmanager-main-tls\") pod \"4f4e7564-e2a1-4b31-9240-57bedb05904a\" (UID: \"4f4e7564-e2a1-4b31-9240-57bedb05904a\") " Jul 10 16:57:02.944666 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:02.944353 2578 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f4e7564-e2a1-4b31-9240-57bedb05904a-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 16:57:02.945487 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:02.945457 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f4e7564-e2a1-4b31-9240-57bedb05904a-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "4f4e7564-e2a1-4b31-9240-57bedb05904a" (UID: "4f4e7564-e2a1-4b31-9240-57bedb05904a"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:57:02.945742 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:02.945713 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f4e7564-e2a1-4b31-9240-57bedb05904a-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "4f4e7564-e2a1-4b31-9240-57bedb05904a" (UID: "4f4e7564-e2a1-4b31-9240-57bedb05904a"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 16:57:02.947743 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:02.947701 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f4e7564-e2a1-4b31-9240-57bedb05904a-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "4f4e7564-e2a1-4b31-9240-57bedb05904a" (UID: "4f4e7564-e2a1-4b31-9240-57bedb05904a"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 16:57:02.948187 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:02.948147 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f4e7564-e2a1-4b31-9240-57bedb05904a-config-out" (OuterVolumeSpecName: "config-out") pod "4f4e7564-e2a1-4b31-9240-57bedb05904a" (UID: "4f4e7564-e2a1-4b31-9240-57bedb05904a"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:57:02.948448 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:02.948408 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f4e7564-e2a1-4b31-9240-57bedb05904a-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "4f4e7564-e2a1-4b31-9240-57bedb05904a" (UID: "4f4e7564-e2a1-4b31-9240-57bedb05904a"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 16:57:02.948659 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:02.948589 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f4e7564-e2a1-4b31-9240-57bedb05904a-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "4f4e7564-e2a1-4b31-9240-57bedb05904a" (UID: "4f4e7564-e2a1-4b31-9240-57bedb05904a"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 16:57:02.948756 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:02.948684 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f4e7564-e2a1-4b31-9240-57bedb05904a-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "4f4e7564-e2a1-4b31-9240-57bedb05904a" (UID: "4f4e7564-e2a1-4b31-9240-57bedb05904a"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 16:57:02.949067 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:02.949026 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f4e7564-e2a1-4b31-9240-57bedb05904a-kube-api-access-55zpc" (OuterVolumeSpecName: "kube-api-access-55zpc") pod "4f4e7564-e2a1-4b31-9240-57bedb05904a" (UID: "4f4e7564-e2a1-4b31-9240-57bedb05904a"). InnerVolumeSpecName "kube-api-access-55zpc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 16:57:02.949713 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:02.949678 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f4e7564-e2a1-4b31-9240-57bedb05904a-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "4f4e7564-e2a1-4b31-9240-57bedb05904a" (UID: "4f4e7564-e2a1-4b31-9240-57bedb05904a"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 16:57:02.950313 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:02.950287 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f4e7564-e2a1-4b31-9240-57bedb05904a-config-volume" (OuterVolumeSpecName: "config-volume") pod "4f4e7564-e2a1-4b31-9240-57bedb05904a" (UID: "4f4e7564-e2a1-4b31-9240-57bedb05904a"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 16:57:02.953069 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:02.952991 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f4e7564-e2a1-4b31-9240-57bedb05904a-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "4f4e7564-e2a1-4b31-9240-57bedb05904a" (UID: "4f4e7564-e2a1-4b31-9240-57bedb05904a"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 16:57:02.962569 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:02.962530 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f4e7564-e2a1-4b31-9240-57bedb05904a-web-config" (OuterVolumeSpecName: "web-config") pod "4f4e7564-e2a1-4b31-9240-57bedb05904a" (UID: "4f4e7564-e2a1-4b31-9240-57bedb05904a"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 16:57:03.045806 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:03.045759 2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4f4e7564-e2a1-4b31-9240-57bedb05904a-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 16:57:03.045806 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:03.045800 2578 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4f4e7564-e2a1-4b31-9240-57bedb05904a-cluster-tls-config\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 16:57:03.045806 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:03.045812 2578 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4f4e7564-e2a1-4b31-9240-57bedb05904a-config-out\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 16:57:03.046041 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:03.045823 2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4f4e7564-e2a1-4b31-9240-57bedb05904a-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 16:57:03.046041 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:03.045832 2578 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f4e7564-e2a1-4b31-9240-57bedb05904a-metrics-client-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 16:57:03.046041 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:03.045842 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-55zpc\" (UniqueName: \"kubernetes.io/projected/4f4e7564-e2a1-4b31-9240-57bedb05904a-kube-api-access-55zpc\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 16:57:03.046041 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:03.045851 2578 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4f4e7564-e2a1-4b31-9240-57bedb05904a-config-volume\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 16:57:03.046041 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:03.045859 2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4f4e7564-e2a1-4b31-9240-57bedb05904a-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 16:57:03.046041 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:03.045869 2578 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4f4e7564-e2a1-4b31-9240-57bedb05904a-web-config\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 16:57:03.046041 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:03.045877 2578 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4f4e7564-e2a1-4b31-9240-57bedb05904a-tls-assets\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 16:57:03.046041 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:03.045885 2578 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4f4e7564-e2a1-4b31-9240-57bedb05904a-alertmanager-main-db\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 16:57:03.046041 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:03.045893 2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4f4e7564-e2a1-4b31-9240-57bedb05904a-secret-alertmanager-main-tls\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 16:57:03.421275 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:03.421240 2578 generic.go:356] "Generic (PLEG): container finished" podID="4f4e7564-e2a1-4b31-9240-57bedb05904a" containerID="7bd7406962280714cb85d5ce55f59ace5ca7ef3a5c519a550bd0a99e63e99bd8" exitCode=0 Jul 10 16:57:03.421480 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:03.421319 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4f4e7564-e2a1-4b31-9240-57bedb05904a","Type":"ContainerDied","Data":"7bd7406962280714cb85d5ce55f59ace5ca7ef3a5c519a550bd0a99e63e99bd8"} Jul 10 16:57:03.421480 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:03.421392 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4f4e7564-e2a1-4b31-9240-57bedb05904a","Type":"ContainerDied","Data":"3c4810a6a0d707f8299ef6e9386c11e711f87ab8f73ecbcd727b865efd5f91a8"} Jul 10 16:57:03.421480 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:03.421323 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 10 16:57:03.421480 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:03.421440 2578 scope.go:117] "RemoveContainer" containerID="f99d386e8d6d7310399a49cbfa3d81e642ca39a5a36074be5f2eabfa815054eb" Jul 10 16:57:03.431418 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:03.431381 2578 scope.go:117] "RemoveContainer" containerID="9f0a097ed8893130017380f5ad31b01976ac178beacd74d93465278447063f35" Jul 10 16:57:03.439988 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:03.439960 2578 scope.go:117] "RemoveContainer" containerID="b554e8d66828632dfb462be68f01ce9be916a5947902df361dd2b7ad1f1d66e6" Jul 10 16:57:03.453876 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:03.449267 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 10 16:57:03.453876 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:03.449666 2578 scope.go:117] "RemoveContainer" containerID="7bd7406962280714cb85d5ce55f59ace5ca7ef3a5c519a550bd0a99e63e99bd8" Jul 10 16:57:03.455211 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:03.455174 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 10 16:57:03.457181 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:57:03.457145 2578 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f4e7564_e2a1_4b31_9240_57bedb05904a.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f4e7564_e2a1_4b31_9240_57bedb05904a.slice/crio-3c4810a6a0d707f8299ef6e9386c11e711f87ab8f73ecbcd727b865efd5f91a8\": RecentStats: unable to find data in memory cache]" Jul 10 16:57:03.461593 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:03.461565 2578 scope.go:117] "RemoveContainer" containerID="84d693208254d4f9bf74b08ee7a2391f7ba7a770c96ae3e7482cd34f97168f95" Jul 10 16:57:03.470112 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:03.470086 2578 scope.go:117] "RemoveContainer" containerID="3cec9b37591085582cc600b7d66e6648f5c5b273daee1b523773cfcc75841688" Jul 10 16:57:03.478830 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:03.478806 2578 scope.go:117] "RemoveContainer" containerID="1ed5e5171841686131fc4d7f266aa8280713e851c0c2aee44e857eb233c31b4d" Jul 10 16:57:03.488384 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:03.488360 2578 scope.go:117] "RemoveContainer" containerID="f99d386e8d6d7310399a49cbfa3d81e642ca39a5a36074be5f2eabfa815054eb" Jul 10 16:57:03.489032 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:57:03.489009 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f99d386e8d6d7310399a49cbfa3d81e642ca39a5a36074be5f2eabfa815054eb\": container with ID starting with f99d386e8d6d7310399a49cbfa3d81e642ca39a5a36074be5f2eabfa815054eb not found: ID does not exist" containerID="f99d386e8d6d7310399a49cbfa3d81e642ca39a5a36074be5f2eabfa815054eb" Jul 10 16:57:03.489262 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:03.489220 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f99d386e8d6d7310399a49cbfa3d81e642ca39a5a36074be5f2eabfa815054eb"} err="failed to get container status \"f99d386e8d6d7310399a49cbfa3d81e642ca39a5a36074be5f2eabfa815054eb\": rpc error: code = NotFound desc = could not find container \"f99d386e8d6d7310399a49cbfa3d81e642ca39a5a36074be5f2eabfa815054eb\": container with ID starting with f99d386e8d6d7310399a49cbfa3d81e642ca39a5a36074be5f2eabfa815054eb not found: ID does not exist" Jul 10 16:57:03.489384 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:03.489372 2578 scope.go:117] "RemoveContainer" containerID="9f0a097ed8893130017380f5ad31b01976ac178beacd74d93465278447063f35" Jul 10 16:57:03.489934 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:57:03.489898 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f0a097ed8893130017380f5ad31b01976ac178beacd74d93465278447063f35\": container with ID starting with 9f0a097ed8893130017380f5ad31b01976ac178beacd74d93465278447063f35 not found: ID does not exist" containerID="9f0a097ed8893130017380f5ad31b01976ac178beacd74d93465278447063f35" Jul 10 16:57:03.489934 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:03.489928 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f0a097ed8893130017380f5ad31b01976ac178beacd74d93465278447063f35"} err="failed to get container status \"9f0a097ed8893130017380f5ad31b01976ac178beacd74d93465278447063f35\": rpc error: code = NotFound desc = could not find container \"9f0a097ed8893130017380f5ad31b01976ac178beacd74d93465278447063f35\": container with ID starting with 9f0a097ed8893130017380f5ad31b01976ac178beacd74d93465278447063f35 not found: ID does not exist" Jul 10 16:57:03.490137 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:03.489947 2578 scope.go:117] "RemoveContainer" containerID="b554e8d66828632dfb462be68f01ce9be916a5947902df361dd2b7ad1f1d66e6" Jul 10 16:57:03.490473 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:57:03.490445 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b554e8d66828632dfb462be68f01ce9be916a5947902df361dd2b7ad1f1d66e6\": container with ID starting with b554e8d66828632dfb462be68f01ce9be916a5947902df361dd2b7ad1f1d66e6 not found: ID does not exist" containerID="b554e8d66828632dfb462be68f01ce9be916a5947902df361dd2b7ad1f1d66e6" Jul 10 16:57:03.490577 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:03.490476 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b554e8d66828632dfb462be68f01ce9be916a5947902df361dd2b7ad1f1d66e6"} err="failed to get container status \"b554e8d66828632dfb462be68f01ce9be916a5947902df361dd2b7ad1f1d66e6\": rpc error: code = NotFound desc = could not find container \"b554e8d66828632dfb462be68f01ce9be916a5947902df361dd2b7ad1f1d66e6\": container with ID starting with b554e8d66828632dfb462be68f01ce9be916a5947902df361dd2b7ad1f1d66e6 not found: ID does not exist" Jul 10 16:57:03.490577 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:03.490491 2578 scope.go:117] "RemoveContainer" containerID="7bd7406962280714cb85d5ce55f59ace5ca7ef3a5c519a550bd0a99e63e99bd8" Jul 10 16:57:03.490824 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:57:03.490803 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7bd7406962280714cb85d5ce55f59ace5ca7ef3a5c519a550bd0a99e63e99bd8\": container with ID starting with 7bd7406962280714cb85d5ce55f59ace5ca7ef3a5c519a550bd0a99e63e99bd8 not found: ID does not exist" containerID="7bd7406962280714cb85d5ce55f59ace5ca7ef3a5c519a550bd0a99e63e99bd8" Jul 10 16:57:03.490892 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:03.490833 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7bd7406962280714cb85d5ce55f59ace5ca7ef3a5c519a550bd0a99e63e99bd8"} err="failed to get container status \"7bd7406962280714cb85d5ce55f59ace5ca7ef3a5c519a550bd0a99e63e99bd8\": rpc error: code = NotFound desc = could not find container \"7bd7406962280714cb85d5ce55f59ace5ca7ef3a5c519a550bd0a99e63e99bd8\": container with ID starting with 7bd7406962280714cb85d5ce55f59ace5ca7ef3a5c519a550bd0a99e63e99bd8 not found: ID does not exist" Jul 10 16:57:03.490892 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:03.490855 2578 scope.go:117] "RemoveContainer" containerID="84d693208254d4f9bf74b08ee7a2391f7ba7a770c96ae3e7482cd34f97168f95" Jul 10 16:57:03.491138 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:57:03.491120 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84d693208254d4f9bf74b08ee7a2391f7ba7a770c96ae3e7482cd34f97168f95\": container with ID starting with 84d693208254d4f9bf74b08ee7a2391f7ba7a770c96ae3e7482cd34f97168f95 not found: ID does not exist" containerID="84d693208254d4f9bf74b08ee7a2391f7ba7a770c96ae3e7482cd34f97168f95" Jul 10 16:57:03.491219 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:03.491144 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84d693208254d4f9bf74b08ee7a2391f7ba7a770c96ae3e7482cd34f97168f95"} err="failed to get container status \"84d693208254d4f9bf74b08ee7a2391f7ba7a770c96ae3e7482cd34f97168f95\": rpc error: code = NotFound desc = could not find container \"84d693208254d4f9bf74b08ee7a2391f7ba7a770c96ae3e7482cd34f97168f95\": container with ID starting with 84d693208254d4f9bf74b08ee7a2391f7ba7a770c96ae3e7482cd34f97168f95 not found: ID does not exist" Jul 10 16:57:03.491219 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:03.491164 2578 scope.go:117] "RemoveContainer" containerID="3cec9b37591085582cc600b7d66e6648f5c5b273daee1b523773cfcc75841688" Jul 10 16:57:03.491428 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:57:03.491410 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3cec9b37591085582cc600b7d66e6648f5c5b273daee1b523773cfcc75841688\": container with ID starting with 3cec9b37591085582cc600b7d66e6648f5c5b273daee1b523773cfcc75841688 not found: ID does not exist" containerID="3cec9b37591085582cc600b7d66e6648f5c5b273daee1b523773cfcc75841688" Jul 10 16:57:03.491476 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:03.491432 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cec9b37591085582cc600b7d66e6648f5c5b273daee1b523773cfcc75841688"} err="failed to get container status \"3cec9b37591085582cc600b7d66e6648f5c5b273daee1b523773cfcc75841688\": rpc error: code = NotFound desc = could not find container \"3cec9b37591085582cc600b7d66e6648f5c5b273daee1b523773cfcc75841688\": container with ID starting with 3cec9b37591085582cc600b7d66e6648f5c5b273daee1b523773cfcc75841688 not found: ID does not exist" Jul 10 16:57:03.491476 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:03.491445 2578 scope.go:117] "RemoveContainer" containerID="1ed5e5171841686131fc4d7f266aa8280713e851c0c2aee44e857eb233c31b4d" Jul 10 16:57:03.491700 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:57:03.491679 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ed5e5171841686131fc4d7f266aa8280713e851c0c2aee44e857eb233c31b4d\": container with ID starting with 1ed5e5171841686131fc4d7f266aa8280713e851c0c2aee44e857eb233c31b4d not found: ID does not exist" containerID="1ed5e5171841686131fc4d7f266aa8280713e851c0c2aee44e857eb233c31b4d" Jul 10 16:57:03.491747 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:03.491710 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ed5e5171841686131fc4d7f266aa8280713e851c0c2aee44e857eb233c31b4d"} err="failed to get container status \"1ed5e5171841686131fc4d7f266aa8280713e851c0c2aee44e857eb233c31b4d\": rpc error: code = NotFound desc = could not find container \"1ed5e5171841686131fc4d7f266aa8280713e851c0c2aee44e857eb233c31b4d\": container with ID starting with 1ed5e5171841686131fc4d7f266aa8280713e851c0c2aee44e857eb233c31b4d not found: ID does not exist" Jul 10 16:57:04.425287 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:04.425250 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f4e7564-e2a1-4b31-9240-57bedb05904a" path="/var/lib/kubelet/pods/4f4e7564-e2a1-4b31-9240-57bedb05904a/volumes" Jul 10 16:57:05.579432 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.579382 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-66d47b6679-hxmgv"] Jul 10 16:57:05.579856 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.579838 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16ef858a-cf52-4c85-a4f7-fe382ade2581" containerName="console" Jul 10 16:57:05.579895 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.579861 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="16ef858a-cf52-4c85-a4f7-fe382ade2581" containerName="console" Jul 10 16:57:05.579895 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.579878 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f4e7564-e2a1-4b31-9240-57bedb05904a" containerName="alertmanager" Jul 10 16:57:05.579895 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.579887 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f4e7564-e2a1-4b31-9240-57bedb05904a" containerName="alertmanager" Jul 10 16:57:05.579988 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.579901 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f4e7564-e2a1-4b31-9240-57bedb05904a" containerName="kube-rbac-proxy" Jul 10 16:57:05.579988 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.579911 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f4e7564-e2a1-4b31-9240-57bedb05904a" containerName="kube-rbac-proxy" Jul 10 16:57:05.579988 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.579927 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f4e7564-e2a1-4b31-9240-57bedb05904a" containerName="init-config-reloader" Jul 10 16:57:05.579988 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.579936 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f4e7564-e2a1-4b31-9240-57bedb05904a" containerName="init-config-reloader" Jul 10 16:57:05.579988 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.579950 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04e15cd0-bcac-4375-97d5-b135b8ef9997" containerName="registry" Jul 10 16:57:05.579988 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.579958 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="04e15cd0-bcac-4375-97d5-b135b8ef9997" containerName="registry" Jul 10 16:57:05.579988 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.579969 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f4e7564-e2a1-4b31-9240-57bedb05904a" containerName="config-reloader" Jul 10 16:57:05.579988 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.579977 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f4e7564-e2a1-4b31-9240-57bedb05904a" containerName="config-reloader" Jul 10 16:57:05.579988 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.579989 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f4e7564-e2a1-4b31-9240-57bedb05904a" containerName="prom-label-proxy" Jul 10 16:57:05.580273 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.579997 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f4e7564-e2a1-4b31-9240-57bedb05904a" containerName="prom-label-proxy" Jul 10 16:57:05.580273 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.580020 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f4e7564-e2a1-4b31-9240-57bedb05904a" containerName="kube-rbac-proxy-web" Jul 10 16:57:05.580273 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.580029 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f4e7564-e2a1-4b31-9240-57bedb05904a" containerName="kube-rbac-proxy-web" Jul 10 16:57:05.580273 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.580042 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f4e7564-e2a1-4b31-9240-57bedb05904a" containerName="kube-rbac-proxy-metric" Jul 10 16:57:05.580273 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.580072 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f4e7564-e2a1-4b31-9240-57bedb05904a" containerName="kube-rbac-proxy-metric" Jul 10 16:57:05.580273 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.580157 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f4e7564-e2a1-4b31-9240-57bedb05904a" containerName="config-reloader" Jul 10 16:57:05.580273 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.580170 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f4e7564-e2a1-4b31-9240-57bedb05904a" containerName="kube-rbac-proxy" Jul 10 16:57:05.580273 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.580182 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f4e7564-e2a1-4b31-9240-57bedb05904a" containerName="prom-label-proxy" Jul 10 16:57:05.580273 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.580193 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="16ef858a-cf52-4c85-a4f7-fe382ade2581" containerName="console" Jul 10 16:57:05.580273 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.580201 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f4e7564-e2a1-4b31-9240-57bedb05904a" containerName="alertmanager" Jul 10 16:57:05.580273 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.580208 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f4e7564-e2a1-4b31-9240-57bedb05904a" containerName="kube-rbac-proxy-web" Jul 10 16:57:05.580273 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.580216 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="04e15cd0-bcac-4375-97d5-b135b8ef9997" containerName="registry" Jul 10 16:57:05.580273 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.580223 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f4e7564-e2a1-4b31-9240-57bedb05904a" containerName="kube-rbac-proxy-metric" Jul 10 16:57:05.585832 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.585797 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-66d47b6679-hxmgv" Jul 10 16:57:05.588580 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.588540 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 10 16:57:05.588580 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.588568 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-kds9w\"" Jul 10 16:57:05.588950 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.588905 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 10 16:57:05.588950 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.588905 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 10 16:57:05.589161 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.588908 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 10 16:57:05.589161 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.588908 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 10 16:57:05.594192 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.593965 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 10 16:57:05.594804 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.594775 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-66d47b6679-hxmgv"] Jul 10 16:57:05.669010 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.668975 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/399a676f-ae4a-42e5-aa7f-ac8a0428131b-metrics-client-ca\") pod \"telemeter-client-66d47b6679-hxmgv\" (UID: \"399a676f-ae4a-42e5-aa7f-ac8a0428131b\") " pod="openshift-monitoring/telemeter-client-66d47b6679-hxmgv" Jul 10 16:57:05.669010 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.669017 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpq4g\" (UniqueName: \"kubernetes.io/projected/399a676f-ae4a-42e5-aa7f-ac8a0428131b-kube-api-access-dpq4g\") pod \"telemeter-client-66d47b6679-hxmgv\" (UID: \"399a676f-ae4a-42e5-aa7f-ac8a0428131b\") " pod="openshift-monitoring/telemeter-client-66d47b6679-hxmgv" Jul 10 16:57:05.669289 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.669042 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/399a676f-ae4a-42e5-aa7f-ac8a0428131b-telemeter-trusted-ca-bundle\") pod \"telemeter-client-66d47b6679-hxmgv\" (UID: \"399a676f-ae4a-42e5-aa7f-ac8a0428131b\") " pod="openshift-monitoring/telemeter-client-66d47b6679-hxmgv" Jul 10 16:57:05.669289 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.669099 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/399a676f-ae4a-42e5-aa7f-ac8a0428131b-federate-client-tls\") pod \"telemeter-client-66d47b6679-hxmgv\" (UID: \"399a676f-ae4a-42e5-aa7f-ac8a0428131b\") " pod="openshift-monitoring/telemeter-client-66d47b6679-hxmgv" Jul 10 16:57:05.669289 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.669181 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/399a676f-ae4a-42e5-aa7f-ac8a0428131b-telemeter-client-tls\") pod \"telemeter-client-66d47b6679-hxmgv\" (UID: \"399a676f-ae4a-42e5-aa7f-ac8a0428131b\") " pod="openshift-monitoring/telemeter-client-66d47b6679-hxmgv" Jul 10 16:57:05.669289 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.669214 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/399a676f-ae4a-42e5-aa7f-ac8a0428131b-secret-telemeter-client\") pod \"telemeter-client-66d47b6679-hxmgv\" (UID: \"399a676f-ae4a-42e5-aa7f-ac8a0428131b\") " pod="openshift-monitoring/telemeter-client-66d47b6679-hxmgv" Jul 10 16:57:05.669289 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.669238 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/399a676f-ae4a-42e5-aa7f-ac8a0428131b-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-66d47b6679-hxmgv\" (UID: \"399a676f-ae4a-42e5-aa7f-ac8a0428131b\") " pod="openshift-monitoring/telemeter-client-66d47b6679-hxmgv" Jul 10 16:57:05.669461 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.669293 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/399a676f-ae4a-42e5-aa7f-ac8a0428131b-serving-certs-ca-bundle\") pod \"telemeter-client-66d47b6679-hxmgv\" (UID: \"399a676f-ae4a-42e5-aa7f-ac8a0428131b\") " pod="openshift-monitoring/telemeter-client-66d47b6679-hxmgv" Jul 10 16:57:05.770254 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.770207 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/399a676f-ae4a-42e5-aa7f-ac8a0428131b-metrics-client-ca\") pod \"telemeter-client-66d47b6679-hxmgv\" (UID: \"399a676f-ae4a-42e5-aa7f-ac8a0428131b\") " pod="openshift-monitoring/telemeter-client-66d47b6679-hxmgv" Jul 10 16:57:05.770456 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.770274 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpq4g\" (UniqueName: \"kubernetes.io/projected/399a676f-ae4a-42e5-aa7f-ac8a0428131b-kube-api-access-dpq4g\") pod \"telemeter-client-66d47b6679-hxmgv\" (UID: \"399a676f-ae4a-42e5-aa7f-ac8a0428131b\") " pod="openshift-monitoring/telemeter-client-66d47b6679-hxmgv" Jul 10 16:57:05.770456 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.770307 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/399a676f-ae4a-42e5-aa7f-ac8a0428131b-telemeter-trusted-ca-bundle\") pod \"telemeter-client-66d47b6679-hxmgv\" (UID: \"399a676f-ae4a-42e5-aa7f-ac8a0428131b\") " pod="openshift-monitoring/telemeter-client-66d47b6679-hxmgv" Jul 10 16:57:05.770456 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.770341 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/399a676f-ae4a-42e5-aa7f-ac8a0428131b-federate-client-tls\") pod \"telemeter-client-66d47b6679-hxmgv\" (UID: \"399a676f-ae4a-42e5-aa7f-ac8a0428131b\") " pod="openshift-monitoring/telemeter-client-66d47b6679-hxmgv" Jul 10 16:57:05.770456 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.770396 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/399a676f-ae4a-42e5-aa7f-ac8a0428131b-telemeter-client-tls\") pod \"telemeter-client-66d47b6679-hxmgv\" (UID: \"399a676f-ae4a-42e5-aa7f-ac8a0428131b\") " pod="openshift-monitoring/telemeter-client-66d47b6679-hxmgv" Jul 10 16:57:05.770456 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.770426 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/399a676f-ae4a-42e5-aa7f-ac8a0428131b-secret-telemeter-client\") pod \"telemeter-client-66d47b6679-hxmgv\" (UID: \"399a676f-ae4a-42e5-aa7f-ac8a0428131b\") " pod="openshift-monitoring/telemeter-client-66d47b6679-hxmgv" Jul 10 16:57:05.770456 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.770454 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/399a676f-ae4a-42e5-aa7f-ac8a0428131b-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-66d47b6679-hxmgv\" (UID: \"399a676f-ae4a-42e5-aa7f-ac8a0428131b\") " pod="openshift-monitoring/telemeter-client-66d47b6679-hxmgv" Jul 10 16:57:05.770786 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.770505 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/399a676f-ae4a-42e5-aa7f-ac8a0428131b-serving-certs-ca-bundle\") pod \"telemeter-client-66d47b6679-hxmgv\" (UID: \"399a676f-ae4a-42e5-aa7f-ac8a0428131b\") " pod="openshift-monitoring/telemeter-client-66d47b6679-hxmgv" Jul 10 16:57:05.771170 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.771138 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/399a676f-ae4a-42e5-aa7f-ac8a0428131b-metrics-client-ca\") pod \"telemeter-client-66d47b6679-hxmgv\" (UID: \"399a676f-ae4a-42e5-aa7f-ac8a0428131b\") " pod="openshift-monitoring/telemeter-client-66d47b6679-hxmgv" Jul 10 16:57:05.771516 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.771263 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/399a676f-ae4a-42e5-aa7f-ac8a0428131b-serving-certs-ca-bundle\") pod \"telemeter-client-66d47b6679-hxmgv\" (UID: \"399a676f-ae4a-42e5-aa7f-ac8a0428131b\") " pod="openshift-monitoring/telemeter-client-66d47b6679-hxmgv" Jul 10 16:57:05.771634 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.771577 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/399a676f-ae4a-42e5-aa7f-ac8a0428131b-telemeter-trusted-ca-bundle\") pod \"telemeter-client-66d47b6679-hxmgv\" (UID: \"399a676f-ae4a-42e5-aa7f-ac8a0428131b\") " pod="openshift-monitoring/telemeter-client-66d47b6679-hxmgv" Jul 10 16:57:05.773485 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.773461 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/399a676f-ae4a-42e5-aa7f-ac8a0428131b-federate-client-tls\") pod \"telemeter-client-66d47b6679-hxmgv\" (UID: \"399a676f-ae4a-42e5-aa7f-ac8a0428131b\") " pod="openshift-monitoring/telemeter-client-66d47b6679-hxmgv" Jul 10 16:57:05.773794 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.773755 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/399a676f-ae4a-42e5-aa7f-ac8a0428131b-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-66d47b6679-hxmgv\" (UID: \"399a676f-ae4a-42e5-aa7f-ac8a0428131b\") " pod="openshift-monitoring/telemeter-client-66d47b6679-hxmgv" Jul 10 16:57:05.773902 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.773829 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/399a676f-ae4a-42e5-aa7f-ac8a0428131b-secret-telemeter-client\") pod \"telemeter-client-66d47b6679-hxmgv\" (UID: \"399a676f-ae4a-42e5-aa7f-ac8a0428131b\") " pod="openshift-monitoring/telemeter-client-66d47b6679-hxmgv" Jul 10 16:57:05.773902 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.773870 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/399a676f-ae4a-42e5-aa7f-ac8a0428131b-telemeter-client-tls\") pod \"telemeter-client-66d47b6679-hxmgv\" (UID: \"399a676f-ae4a-42e5-aa7f-ac8a0428131b\") " pod="openshift-monitoring/telemeter-client-66d47b6679-hxmgv" Jul 10 16:57:05.781097 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.781065 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpq4g\" (UniqueName: \"kubernetes.io/projected/399a676f-ae4a-42e5-aa7f-ac8a0428131b-kube-api-access-dpq4g\") pod \"telemeter-client-66d47b6679-hxmgv\" (UID: \"399a676f-ae4a-42e5-aa7f-ac8a0428131b\") " pod="openshift-monitoring/telemeter-client-66d47b6679-hxmgv" Jul 10 16:57:05.900194 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.900104 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-66d47b6679-hxmgv" Jul 10 16:57:05.954170 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.953885 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 10 16:57:05.954846 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.954810 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="be8c418d-b1dc-44c2-9b59-7b8e673f822a" containerName="prometheus" containerID="cri-o://221094880b4b9763e20c54092552d25b98b0234b31420459d45590ef599e99d4" gracePeriod=600 Jul 10 16:57:05.955132 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.955105 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="be8c418d-b1dc-44c2-9b59-7b8e673f822a" containerName="kube-rbac-proxy" containerID="cri-o://45c7105720d4b5f4bffd225882fcb7dfe3ba576af42454033210e77fb3a69c43" gracePeriod=600 Jul 10 16:57:05.955240 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.955217 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="be8c418d-b1dc-44c2-9b59-7b8e673f822a" containerName="kube-rbac-proxy-thanos" containerID="cri-o://b13e0e6b0e2d1600d8f4331cec0e338a82bf916af0b5310bbb53801bbc7aefcd" gracePeriod=600 Jul 10 16:57:05.955373 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.955293 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="be8c418d-b1dc-44c2-9b59-7b8e673f822a" containerName="thanos-sidecar" containerID="cri-o://331e92f64ca5f409187e8e4d696bde59498ef92036cffdda86ad1bb78722f063" gracePeriod=600 Jul 10 16:57:05.955373 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.955360 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="be8c418d-b1dc-44c2-9b59-7b8e673f822a" containerName="kube-rbac-proxy-web" containerID="cri-o://44a361bf537e5f93a017ccd402d7c49ad4bdb3fd8fc9e482c2957e0304d752d3" gracePeriod=600 Jul 10 16:57:05.955494 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:05.955440 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="be8c418d-b1dc-44c2-9b59-7b8e673f822a" containerName="config-reloader" containerID="cri-o://f69bc78775e591cb96b746738dfcb7cb6398ed6fe8c7bfab84f8e9b9d6a2d5a5" gracePeriod=600 Jul 10 16:57:06.079384 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.079358 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-66d47b6679-hxmgv"] Jul 10 16:57:06.090155 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:57:06.090106 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod399a676f_ae4a_42e5_aa7f_ac8a0428131b.slice/crio-92a62b71940e4c3d8b2b62e89964965e96610ed105a4a18e6187d8aaeee7829c WatchSource:0}: Error finding container 92a62b71940e4c3d8b2b62e89964965e96610ed105a4a18e6187d8aaeee7829c: Status 404 returned error can't find the container with id 92a62b71940e4c3d8b2b62e89964965e96610ed105a4a18e6187d8aaeee7829c Jul 10 16:57:06.227543 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.227513 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.376561 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.376524 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/be8c418d-b1dc-44c2-9b59-7b8e673f822a-prometheus-k8s-rulefiles-1\") pod \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " Jul 10 16:57:06.376561 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.376565 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/be8c418d-b1dc-44c2-9b59-7b8e673f822a-secret-grpc-tls\") pod \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " Jul 10 16:57:06.376796 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.376609 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/be8c418d-b1dc-44c2-9b59-7b8e673f822a-secret-prometheus-k8s-tls\") pod \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " Jul 10 16:57:06.376796 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.376631 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/be8c418d-b1dc-44c2-9b59-7b8e673f822a-prometheus-k8s-db\") pod \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " Jul 10 16:57:06.376796 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.376658 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/be8c418d-b1dc-44c2-9b59-7b8e673f822a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " Jul 10 16:57:06.376796 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.376686 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/be8c418d-b1dc-44c2-9b59-7b8e673f822a-configmap-metrics-client-ca\") pod \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " Jul 10 16:57:06.376796 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.376719 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be8c418d-b1dc-44c2-9b59-7b8e673f822a-prometheus-trusted-ca-bundle\") pod \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " Jul 10 16:57:06.376796 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.376747 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/be8c418d-b1dc-44c2-9b59-7b8e673f822a-thanos-prometheus-http-client-file\") pod \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " Jul 10 16:57:06.376796 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.376792 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/be8c418d-b1dc-44c2-9b59-7b8e673f822a-prometheus-k8s-rulefiles-2\") pod \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " Jul 10 16:57:06.377241 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.376816 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/be8c418d-b1dc-44c2-9b59-7b8e673f822a-secret-kube-rbac-proxy\") pod \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " Jul 10 16:57:06.377241 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.376869 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/be8c418d-b1dc-44c2-9b59-7b8e673f822a-tls-assets\") pod \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " Jul 10 16:57:06.377241 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.376898 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/be8c418d-b1dc-44c2-9b59-7b8e673f822a-web-config\") pod \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " Jul 10 16:57:06.377241 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.376919 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be8c418d-b1dc-44c2-9b59-7b8e673f822a-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "be8c418d-b1dc-44c2-9b59-7b8e673f822a" (UID: "be8c418d-b1dc-44c2-9b59-7b8e673f822a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 16:57:06.377241 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.376969 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/be8c418d-b1dc-44c2-9b59-7b8e673f822a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " Jul 10 16:57:06.377241 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.377009 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/be8c418d-b1dc-44c2-9b59-7b8e673f822a-config-out\") pod \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " Jul 10 16:57:06.377241 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.377043 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/be8c418d-b1dc-44c2-9b59-7b8e673f822a-secret-metrics-client-certs\") pod \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " Jul 10 16:57:06.377241 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.377111 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be8c418d-b1dc-44c2-9b59-7b8e673f822a-configmap-serving-certs-ca-bundle\") pod \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " Jul 10 16:57:06.377241 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.377108 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be8c418d-b1dc-44c2-9b59-7b8e673f822a-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "be8c418d-b1dc-44c2-9b59-7b8e673f822a" (UID: "be8c418d-b1dc-44c2-9b59-7b8e673f822a"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 16:57:06.377241 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.377153 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r7pt4\" (UniqueName: \"kubernetes.io/projected/be8c418d-b1dc-44c2-9b59-7b8e673f822a-kube-api-access-r7pt4\") pod \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " Jul 10 16:57:06.377241 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.377177 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/be8c418d-b1dc-44c2-9b59-7b8e673f822a-config\") pod \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " Jul 10 16:57:06.377241 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.377204 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/be8c418d-b1dc-44c2-9b59-7b8e673f822a-prometheus-k8s-rulefiles-0\") pod \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " Jul 10 16:57:06.377241 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.377226 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be8c418d-b1dc-44c2-9b59-7b8e673f822a-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "be8c418d-b1dc-44c2-9b59-7b8e673f822a" (UID: "be8c418d-b1dc-44c2-9b59-7b8e673f822a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 16:57:06.377241 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.377234 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be8c418d-b1dc-44c2-9b59-7b8e673f822a-configmap-kubelet-serving-ca-bundle\") pod \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\" (UID: \"be8c418d-b1dc-44c2-9b59-7b8e673f822a\") " Jul 10 16:57:06.378082 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.377696 2578 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/be8c418d-b1dc-44c2-9b59-7b8e673f822a-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 16:57:06.378082 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.377700 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be8c418d-b1dc-44c2-9b59-7b8e673f822a-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "be8c418d-b1dc-44c2-9b59-7b8e673f822a" (UID: "be8c418d-b1dc-44c2-9b59-7b8e673f822a"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 16:57:06.378082 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.377718 2578 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/be8c418d-b1dc-44c2-9b59-7b8e673f822a-configmap-metrics-client-ca\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 16:57:06.378082 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.377735 2578 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/be8c418d-b1dc-44c2-9b59-7b8e673f822a-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 16:57:06.378082 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.377903 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be8c418d-b1dc-44c2-9b59-7b8e673f822a-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "be8c418d-b1dc-44c2-9b59-7b8e673f822a" (UID: "be8c418d-b1dc-44c2-9b59-7b8e673f822a"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:57:06.378327 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.378259 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be8c418d-b1dc-44c2-9b59-7b8e673f822a-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "be8c418d-b1dc-44c2-9b59-7b8e673f822a" (UID: "be8c418d-b1dc-44c2-9b59-7b8e673f822a"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 16:57:06.378420 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.378381 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be8c418d-b1dc-44c2-9b59-7b8e673f822a-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "be8c418d-b1dc-44c2-9b59-7b8e673f822a" (UID: "be8c418d-b1dc-44c2-9b59-7b8e673f822a"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 16:57:06.379934 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.379643 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be8c418d-b1dc-44c2-9b59-7b8e673f822a-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "be8c418d-b1dc-44c2-9b59-7b8e673f822a" (UID: "be8c418d-b1dc-44c2-9b59-7b8e673f822a"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 16:57:06.379934 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.379720 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be8c418d-b1dc-44c2-9b59-7b8e673f822a-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "be8c418d-b1dc-44c2-9b59-7b8e673f822a" (UID: "be8c418d-b1dc-44c2-9b59-7b8e673f822a"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 16:57:06.379934 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.379746 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be8c418d-b1dc-44c2-9b59-7b8e673f822a-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "be8c418d-b1dc-44c2-9b59-7b8e673f822a" (UID: "be8c418d-b1dc-44c2-9b59-7b8e673f822a"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 16:57:06.380607 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.380569 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be8c418d-b1dc-44c2-9b59-7b8e673f822a-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "be8c418d-b1dc-44c2-9b59-7b8e673f822a" (UID: "be8c418d-b1dc-44c2-9b59-7b8e673f822a"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 16:57:06.381273 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.381238 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be8c418d-b1dc-44c2-9b59-7b8e673f822a-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "be8c418d-b1dc-44c2-9b59-7b8e673f822a" (UID: "be8c418d-b1dc-44c2-9b59-7b8e673f822a"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 16:57:06.381273 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.381252 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be8c418d-b1dc-44c2-9b59-7b8e673f822a-config-out" (OuterVolumeSpecName: "config-out") pod "be8c418d-b1dc-44c2-9b59-7b8e673f822a" (UID: "be8c418d-b1dc-44c2-9b59-7b8e673f822a"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:57:06.381937 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.381807 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be8c418d-b1dc-44c2-9b59-7b8e673f822a-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "be8c418d-b1dc-44c2-9b59-7b8e673f822a" (UID: "be8c418d-b1dc-44c2-9b59-7b8e673f822a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 16:57:06.381937 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.381845 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be8c418d-b1dc-44c2-9b59-7b8e673f822a-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "be8c418d-b1dc-44c2-9b59-7b8e673f822a" (UID: "be8c418d-b1dc-44c2-9b59-7b8e673f822a"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 16:57:06.382301 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.382277 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be8c418d-b1dc-44c2-9b59-7b8e673f822a-config" (OuterVolumeSpecName: "config") pod "be8c418d-b1dc-44c2-9b59-7b8e673f822a" (UID: "be8c418d-b1dc-44c2-9b59-7b8e673f822a"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 16:57:06.382428 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.382303 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be8c418d-b1dc-44c2-9b59-7b8e673f822a-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "be8c418d-b1dc-44c2-9b59-7b8e673f822a" (UID: "be8c418d-b1dc-44c2-9b59-7b8e673f822a"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 16:57:06.382696 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.382672 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be8c418d-b1dc-44c2-9b59-7b8e673f822a-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "be8c418d-b1dc-44c2-9b59-7b8e673f822a" (UID: "be8c418d-b1dc-44c2-9b59-7b8e673f822a"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 16:57:06.383252 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.383233 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be8c418d-b1dc-44c2-9b59-7b8e673f822a-kube-api-access-r7pt4" (OuterVolumeSpecName: "kube-api-access-r7pt4") pod "be8c418d-b1dc-44c2-9b59-7b8e673f822a" (UID: "be8c418d-b1dc-44c2-9b59-7b8e673f822a"). InnerVolumeSpecName "kube-api-access-r7pt4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 16:57:06.393960 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.393923 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be8c418d-b1dc-44c2-9b59-7b8e673f822a-web-config" (OuterVolumeSpecName: "web-config") pod "be8c418d-b1dc-44c2-9b59-7b8e673f822a" (UID: "be8c418d-b1dc-44c2-9b59-7b8e673f822a"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 16:57:06.435595 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.435558 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-66d47b6679-hxmgv" event={"ID":"399a676f-ae4a-42e5-aa7f-ac8a0428131b","Type":"ContainerStarted","Data":"92a62b71940e4c3d8b2b62e89964965e96610ed105a4a18e6187d8aaeee7829c"} Jul 10 16:57:06.439139 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.439113 2578 generic.go:356] "Generic (PLEG): container finished" podID="be8c418d-b1dc-44c2-9b59-7b8e673f822a" containerID="b13e0e6b0e2d1600d8f4331cec0e338a82bf916af0b5310bbb53801bbc7aefcd" exitCode=0 Jul 10 16:57:06.439139 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.439137 2578 generic.go:356] "Generic (PLEG): container finished" podID="be8c418d-b1dc-44c2-9b59-7b8e673f822a" containerID="45c7105720d4b5f4bffd225882fcb7dfe3ba576af42454033210e77fb3a69c43" exitCode=0 Jul 10 16:57:06.439139 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.439143 2578 generic.go:356] "Generic (PLEG): container finished" podID="be8c418d-b1dc-44c2-9b59-7b8e673f822a" containerID="44a361bf537e5f93a017ccd402d7c49ad4bdb3fd8fc9e482c2957e0304d752d3" exitCode=0 Jul 10 16:57:06.439313 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.439150 2578 generic.go:356] "Generic (PLEG): container finished" podID="be8c418d-b1dc-44c2-9b59-7b8e673f822a" containerID="331e92f64ca5f409187e8e4d696bde59498ef92036cffdda86ad1bb78722f063" exitCode=0 Jul 10 16:57:06.439313 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.439156 2578 generic.go:356] "Generic (PLEG): container finished" podID="be8c418d-b1dc-44c2-9b59-7b8e673f822a" containerID="f69bc78775e591cb96b746738dfcb7cb6398ed6fe8c7bfab84f8e9b9d6a2d5a5" exitCode=0 Jul 10 16:57:06.439313 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.439161 2578 generic.go:356] "Generic (PLEG): container finished" podID="be8c418d-b1dc-44c2-9b59-7b8e673f822a" containerID="221094880b4b9763e20c54092552d25b98b0234b31420459d45590ef599e99d4" exitCode=0 Jul 10 16:57:06.439313 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.439193 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"be8c418d-b1dc-44c2-9b59-7b8e673f822a","Type":"ContainerDied","Data":"b13e0e6b0e2d1600d8f4331cec0e338a82bf916af0b5310bbb53801bbc7aefcd"} Jul 10 16:57:06.439313 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.439254 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"be8c418d-b1dc-44c2-9b59-7b8e673f822a","Type":"ContainerDied","Data":"45c7105720d4b5f4bffd225882fcb7dfe3ba576af42454033210e77fb3a69c43"} Jul 10 16:57:06.439313 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.439200 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.439313 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.439281 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"be8c418d-b1dc-44c2-9b59-7b8e673f822a","Type":"ContainerDied","Data":"44a361bf537e5f93a017ccd402d7c49ad4bdb3fd8fc9e482c2957e0304d752d3"} Jul 10 16:57:06.439313 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.439307 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"be8c418d-b1dc-44c2-9b59-7b8e673f822a","Type":"ContainerDied","Data":"331e92f64ca5f409187e8e4d696bde59498ef92036cffdda86ad1bb78722f063"} Jul 10 16:57:06.439559 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.439333 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"be8c418d-b1dc-44c2-9b59-7b8e673f822a","Type":"ContainerDied","Data":"f69bc78775e591cb96b746738dfcb7cb6398ed6fe8c7bfab84f8e9b9d6a2d5a5"} Jul 10 16:57:06.439559 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.439356 2578 scope.go:117] "RemoveContainer" containerID="b13e0e6b0e2d1600d8f4331cec0e338a82bf916af0b5310bbb53801bbc7aefcd" Jul 10 16:57:06.439559 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.439361 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"be8c418d-b1dc-44c2-9b59-7b8e673f822a","Type":"ContainerDied","Data":"221094880b4b9763e20c54092552d25b98b0234b31420459d45590ef599e99d4"} Jul 10 16:57:06.439559 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.439388 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"be8c418d-b1dc-44c2-9b59-7b8e673f822a","Type":"ContainerDied","Data":"2bb97f432c93db754b378f119eaf699597b9744bcfc84e425ae442de93da43a3"} Jul 10 16:57:06.447745 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.447728 2578 scope.go:117] "RemoveContainer" containerID="45c7105720d4b5f4bffd225882fcb7dfe3ba576af42454033210e77fb3a69c43" Jul 10 16:57:06.455457 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.455435 2578 scope.go:117] "RemoveContainer" containerID="44a361bf537e5f93a017ccd402d7c49ad4bdb3fd8fc9e482c2957e0304d752d3" Jul 10 16:57:06.459343 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.459320 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 10 16:57:06.464652 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.464624 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 10 16:57:06.464991 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.464969 2578 scope.go:117] "RemoveContainer" containerID="331e92f64ca5f409187e8e4d696bde59498ef92036cffdda86ad1bb78722f063" Jul 10 16:57:06.473001 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.472980 2578 scope.go:117] "RemoveContainer" containerID="f69bc78775e591cb96b746738dfcb7cb6398ed6fe8c7bfab84f8e9b9d6a2d5a5" Jul 10 16:57:06.478513 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.478489 2578 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/be8c418d-b1dc-44c2-9b59-7b8e673f822a-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 16:57:06.478513 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.478514 2578 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/be8c418d-b1dc-44c2-9b59-7b8e673f822a-config-out\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 16:57:06.478681 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.478526 2578 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/be8c418d-b1dc-44c2-9b59-7b8e673f822a-secret-metrics-client-certs\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 16:57:06.478681 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.478535 2578 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be8c418d-b1dc-44c2-9b59-7b8e673f822a-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 16:57:06.478681 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.478545 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r7pt4\" (UniqueName: \"kubernetes.io/projected/be8c418d-b1dc-44c2-9b59-7b8e673f822a-kube-api-access-r7pt4\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 16:57:06.478681 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.478554 2578 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/be8c418d-b1dc-44c2-9b59-7b8e673f822a-config\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 16:57:06.478681 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.478562 2578 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/be8c418d-b1dc-44c2-9b59-7b8e673f822a-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 16:57:06.478681 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.478572 2578 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be8c418d-b1dc-44c2-9b59-7b8e673f822a-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 16:57:06.478681 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.478580 2578 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/be8c418d-b1dc-44c2-9b59-7b8e673f822a-secret-grpc-tls\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 16:57:06.478681 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.478589 2578 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/be8c418d-b1dc-44c2-9b59-7b8e673f822a-secret-prometheus-k8s-tls\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 16:57:06.478681 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.478596 2578 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/be8c418d-b1dc-44c2-9b59-7b8e673f822a-prometheus-k8s-db\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 16:57:06.478681 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.478605 2578 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/be8c418d-b1dc-44c2-9b59-7b8e673f822a-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 16:57:06.478681 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.478614 2578 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be8c418d-b1dc-44c2-9b59-7b8e673f822a-prometheus-trusted-ca-bundle\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 16:57:06.478681 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.478626 2578 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/be8c418d-b1dc-44c2-9b59-7b8e673f822a-thanos-prometheus-http-client-file\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 16:57:06.478681 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.478639 2578 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/be8c418d-b1dc-44c2-9b59-7b8e673f822a-secret-kube-rbac-proxy\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 16:57:06.478681 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.478649 2578 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/be8c418d-b1dc-44c2-9b59-7b8e673f822a-tls-assets\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 16:57:06.478681 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.478681 2578 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/be8c418d-b1dc-44c2-9b59-7b8e673f822a-web-config\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 16:57:06.481146 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.481120 2578 scope.go:117] "RemoveContainer" containerID="221094880b4b9763e20c54092552d25b98b0234b31420459d45590ef599e99d4" Jul 10 16:57:06.488584 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.488556 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 10 16:57:06.489009 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.488989 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be8c418d-b1dc-44c2-9b59-7b8e673f822a" containerName="init-config-reloader" Jul 10 16:57:06.489091 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.489013 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="be8c418d-b1dc-44c2-9b59-7b8e673f822a" containerName="init-config-reloader" Jul 10 16:57:06.489091 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.489024 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be8c418d-b1dc-44c2-9b59-7b8e673f822a" containerName="config-reloader" Jul 10 16:57:06.489091 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.489032 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="be8c418d-b1dc-44c2-9b59-7b8e673f822a" containerName="config-reloader" Jul 10 16:57:06.489091 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.489064 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be8c418d-b1dc-44c2-9b59-7b8e673f822a" containerName="thanos-sidecar" Jul 10 16:57:06.489091 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.489074 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="be8c418d-b1dc-44c2-9b59-7b8e673f822a" containerName="thanos-sidecar" Jul 10 16:57:06.489245 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.489094 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be8c418d-b1dc-44c2-9b59-7b8e673f822a" containerName="kube-rbac-proxy" Jul 10 16:57:06.489245 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.489101 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="be8c418d-b1dc-44c2-9b59-7b8e673f822a" containerName="kube-rbac-proxy" Jul 10 16:57:06.489245 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.489121 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be8c418d-b1dc-44c2-9b59-7b8e673f822a" containerName="kube-rbac-proxy-web" Jul 10 16:57:06.489245 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.489129 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="be8c418d-b1dc-44c2-9b59-7b8e673f822a" containerName="kube-rbac-proxy-web" Jul 10 16:57:06.489245 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.489141 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be8c418d-b1dc-44c2-9b59-7b8e673f822a" containerName="prometheus" Jul 10 16:57:06.489245 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.489149 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="be8c418d-b1dc-44c2-9b59-7b8e673f822a" containerName="prometheus" Jul 10 16:57:06.489245 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.489158 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be8c418d-b1dc-44c2-9b59-7b8e673f822a" containerName="kube-rbac-proxy-thanos" Jul 10 16:57:06.489245 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.489166 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="be8c418d-b1dc-44c2-9b59-7b8e673f822a" containerName="kube-rbac-proxy-thanos" Jul 10 16:57:06.489245 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.489242 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="be8c418d-b1dc-44c2-9b59-7b8e673f822a" containerName="thanos-sidecar" Jul 10 16:57:06.489517 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.489254 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="be8c418d-b1dc-44c2-9b59-7b8e673f822a" containerName="prometheus" Jul 10 16:57:06.489517 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.489264 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="be8c418d-b1dc-44c2-9b59-7b8e673f822a" containerName="kube-rbac-proxy-web" Jul 10 16:57:06.489517 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.489273 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="be8c418d-b1dc-44c2-9b59-7b8e673f822a" containerName="kube-rbac-proxy" Jul 10 16:57:06.489517 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.489281 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="be8c418d-b1dc-44c2-9b59-7b8e673f822a" containerName="kube-rbac-proxy-thanos" Jul 10 16:57:06.489517 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.489291 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="be8c418d-b1dc-44c2-9b59-7b8e673f822a" containerName="config-reloader" Jul 10 16:57:06.491509 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.491488 2578 scope.go:117] "RemoveContainer" containerID="9bff15e61ab2f20af2353f41e781d84e88adfef68e00f9f1dc2497bdda5d2375" Jul 10 16:57:06.495365 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.495338 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.497972 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.497953 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 10 16:57:06.498106 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.497987 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-hvzgc\"" Jul 10 16:57:06.498215 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.498201 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 10 16:57:06.499393 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.499359 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 10 16:57:06.499781 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.499760 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 10 16:57:06.500297 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.500264 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 10 16:57:06.500297 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.500278 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 10 16:57:06.500444 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.500389 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 10 16:57:06.500786 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.500766 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 10 16:57:06.500933 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.500836 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 10 16:57:06.501019 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.500885 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 10 16:57:06.501429 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.501406 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 10 16:57:06.501498 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.501485 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 10 16:57:06.501558 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.501511 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 10 16:57:06.501608 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.501599 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-11j5b5j568pj0\"" Jul 10 16:57:06.503421 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.503384 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 10 16:57:06.504034 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.504012 2578 scope.go:117] "RemoveContainer" containerID="b13e0e6b0e2d1600d8f4331cec0e338a82bf916af0b5310bbb53801bbc7aefcd" Jul 10 16:57:06.504695 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:57:06.504668 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b13e0e6b0e2d1600d8f4331cec0e338a82bf916af0b5310bbb53801bbc7aefcd\": container with ID starting with b13e0e6b0e2d1600d8f4331cec0e338a82bf916af0b5310bbb53801bbc7aefcd not found: ID does not exist" containerID="b13e0e6b0e2d1600d8f4331cec0e338a82bf916af0b5310bbb53801bbc7aefcd" Jul 10 16:57:06.504803 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.504709 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b13e0e6b0e2d1600d8f4331cec0e338a82bf916af0b5310bbb53801bbc7aefcd"} err="failed to get container status \"b13e0e6b0e2d1600d8f4331cec0e338a82bf916af0b5310bbb53801bbc7aefcd\": rpc error: code = NotFound desc = could not find container \"b13e0e6b0e2d1600d8f4331cec0e338a82bf916af0b5310bbb53801bbc7aefcd\": container with ID starting with b13e0e6b0e2d1600d8f4331cec0e338a82bf916af0b5310bbb53801bbc7aefcd not found: ID does not exist" Jul 10 16:57:06.504803 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.504737 2578 scope.go:117] "RemoveContainer" containerID="45c7105720d4b5f4bffd225882fcb7dfe3ba576af42454033210e77fb3a69c43" Jul 10 16:57:06.505168 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:57:06.505143 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45c7105720d4b5f4bffd225882fcb7dfe3ba576af42454033210e77fb3a69c43\": container with ID starting with 45c7105720d4b5f4bffd225882fcb7dfe3ba576af42454033210e77fb3a69c43 not found: ID does not exist" containerID="45c7105720d4b5f4bffd225882fcb7dfe3ba576af42454033210e77fb3a69c43" Jul 10 16:57:06.505262 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.505177 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45c7105720d4b5f4bffd225882fcb7dfe3ba576af42454033210e77fb3a69c43"} err="failed to get container status \"45c7105720d4b5f4bffd225882fcb7dfe3ba576af42454033210e77fb3a69c43\": rpc error: code = NotFound desc = could not find container \"45c7105720d4b5f4bffd225882fcb7dfe3ba576af42454033210e77fb3a69c43\": container with ID starting with 45c7105720d4b5f4bffd225882fcb7dfe3ba576af42454033210e77fb3a69c43 not found: ID does not exist" Jul 10 16:57:06.505262 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.505201 2578 scope.go:117] "RemoveContainer" containerID="44a361bf537e5f93a017ccd402d7c49ad4bdb3fd8fc9e482c2957e0304d752d3" Jul 10 16:57:06.505638 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.505617 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 10 16:57:06.505742 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:57:06.505641 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44a361bf537e5f93a017ccd402d7c49ad4bdb3fd8fc9e482c2957e0304d752d3\": container with ID starting with 44a361bf537e5f93a017ccd402d7c49ad4bdb3fd8fc9e482c2957e0304d752d3 not found: ID does not exist" containerID="44a361bf537e5f93a017ccd402d7c49ad4bdb3fd8fc9e482c2957e0304d752d3" Jul 10 16:57:06.505742 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.505687 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44a361bf537e5f93a017ccd402d7c49ad4bdb3fd8fc9e482c2957e0304d752d3"} err="failed to get container status \"44a361bf537e5f93a017ccd402d7c49ad4bdb3fd8fc9e482c2957e0304d752d3\": rpc error: code = NotFound desc = could not find container \"44a361bf537e5f93a017ccd402d7c49ad4bdb3fd8fc9e482c2957e0304d752d3\": container with ID starting with 44a361bf537e5f93a017ccd402d7c49ad4bdb3fd8fc9e482c2957e0304d752d3 not found: ID does not exist" Jul 10 16:57:06.505742 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.505714 2578 scope.go:117] "RemoveContainer" containerID="331e92f64ca5f409187e8e4d696bde59498ef92036cffdda86ad1bb78722f063" Jul 10 16:57:06.506061 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:57:06.506021 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"331e92f64ca5f409187e8e4d696bde59498ef92036cffdda86ad1bb78722f063\": container with ID starting with 331e92f64ca5f409187e8e4d696bde59498ef92036cffdda86ad1bb78722f063 not found: ID does not exist" containerID="331e92f64ca5f409187e8e4d696bde59498ef92036cffdda86ad1bb78722f063" Jul 10 16:57:06.506154 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.506077 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"331e92f64ca5f409187e8e4d696bde59498ef92036cffdda86ad1bb78722f063"} err="failed to get container status \"331e92f64ca5f409187e8e4d696bde59498ef92036cffdda86ad1bb78722f063\": rpc error: code = NotFound desc = could not find container \"331e92f64ca5f409187e8e4d696bde59498ef92036cffdda86ad1bb78722f063\": container with ID starting with 331e92f64ca5f409187e8e4d696bde59498ef92036cffdda86ad1bb78722f063 not found: ID does not exist" Jul 10 16:57:06.506154 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.506104 2578 scope.go:117] "RemoveContainer" containerID="f69bc78775e591cb96b746738dfcb7cb6398ed6fe8c7bfab84f8e9b9d6a2d5a5" Jul 10 16:57:06.506445 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:57:06.506420 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f69bc78775e591cb96b746738dfcb7cb6398ed6fe8c7bfab84f8e9b9d6a2d5a5\": container with ID starting with f69bc78775e591cb96b746738dfcb7cb6398ed6fe8c7bfab84f8e9b9d6a2d5a5 not found: ID does not exist" containerID="f69bc78775e591cb96b746738dfcb7cb6398ed6fe8c7bfab84f8e9b9d6a2d5a5" Jul 10 16:57:06.506586 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.506476 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f69bc78775e591cb96b746738dfcb7cb6398ed6fe8c7bfab84f8e9b9d6a2d5a5"} err="failed to get container status \"f69bc78775e591cb96b746738dfcb7cb6398ed6fe8c7bfab84f8e9b9d6a2d5a5\": rpc error: code = NotFound desc = could not find container \"f69bc78775e591cb96b746738dfcb7cb6398ed6fe8c7bfab84f8e9b9d6a2d5a5\": container with ID starting with f69bc78775e591cb96b746738dfcb7cb6398ed6fe8c7bfab84f8e9b9d6a2d5a5 not found: ID does not exist" Jul 10 16:57:06.506586 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.506499 2578 scope.go:117] "RemoveContainer" containerID="221094880b4b9763e20c54092552d25b98b0234b31420459d45590ef599e99d4" Jul 10 16:57:06.507032 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:57:06.507006 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"221094880b4b9763e20c54092552d25b98b0234b31420459d45590ef599e99d4\": container with ID starting with 221094880b4b9763e20c54092552d25b98b0234b31420459d45590ef599e99d4 not found: ID does not exist" containerID="221094880b4b9763e20c54092552d25b98b0234b31420459d45590ef599e99d4" Jul 10 16:57:06.507156 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.507094 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"221094880b4b9763e20c54092552d25b98b0234b31420459d45590ef599e99d4"} err="failed to get container status \"221094880b4b9763e20c54092552d25b98b0234b31420459d45590ef599e99d4\": rpc error: code = NotFound desc = could not find container \"221094880b4b9763e20c54092552d25b98b0234b31420459d45590ef599e99d4\": container with ID starting with 221094880b4b9763e20c54092552d25b98b0234b31420459d45590ef599e99d4 not found: ID does not exist" Jul 10 16:57:06.507156 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.507121 2578 scope.go:117] "RemoveContainer" containerID="9bff15e61ab2f20af2353f41e781d84e88adfef68e00f9f1dc2497bdda5d2375" Jul 10 16:57:06.507394 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.507373 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 10 16:57:06.507581 ip-10-0-136-7 kubenswrapper[2578]: E0710 16:57:06.507558 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9bff15e61ab2f20af2353f41e781d84e88adfef68e00f9f1dc2497bdda5d2375\": container with ID starting with 9bff15e61ab2f20af2353f41e781d84e88adfef68e00f9f1dc2497bdda5d2375 not found: ID does not exist" containerID="9bff15e61ab2f20af2353f41e781d84e88adfef68e00f9f1dc2497bdda5d2375" Jul 10 16:57:06.507651 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.507588 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bff15e61ab2f20af2353f41e781d84e88adfef68e00f9f1dc2497bdda5d2375"} err="failed to get container status \"9bff15e61ab2f20af2353f41e781d84e88adfef68e00f9f1dc2497bdda5d2375\": rpc error: code = NotFound desc = could not find container \"9bff15e61ab2f20af2353f41e781d84e88adfef68e00f9f1dc2497bdda5d2375\": container with ID starting with 9bff15e61ab2f20af2353f41e781d84e88adfef68e00f9f1dc2497bdda5d2375 not found: ID does not exist" Jul 10 16:57:06.507651 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.507609 2578 scope.go:117] "RemoveContainer" containerID="b13e0e6b0e2d1600d8f4331cec0e338a82bf916af0b5310bbb53801bbc7aefcd" Jul 10 16:57:06.507873 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.507849 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b13e0e6b0e2d1600d8f4331cec0e338a82bf916af0b5310bbb53801bbc7aefcd"} err="failed to get container status \"b13e0e6b0e2d1600d8f4331cec0e338a82bf916af0b5310bbb53801bbc7aefcd\": rpc error: code = NotFound desc = could not find container \"b13e0e6b0e2d1600d8f4331cec0e338a82bf916af0b5310bbb53801bbc7aefcd\": container with ID starting with b13e0e6b0e2d1600d8f4331cec0e338a82bf916af0b5310bbb53801bbc7aefcd not found: ID does not exist" Jul 10 16:57:06.507935 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.507874 2578 scope.go:117] "RemoveContainer" containerID="45c7105720d4b5f4bffd225882fcb7dfe3ba576af42454033210e77fb3a69c43" Jul 10 16:57:06.508172 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.508153 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45c7105720d4b5f4bffd225882fcb7dfe3ba576af42454033210e77fb3a69c43"} err="failed to get container status \"45c7105720d4b5f4bffd225882fcb7dfe3ba576af42454033210e77fb3a69c43\": rpc error: code = NotFound desc = could not find container \"45c7105720d4b5f4bffd225882fcb7dfe3ba576af42454033210e77fb3a69c43\": container with ID starting with 45c7105720d4b5f4bffd225882fcb7dfe3ba576af42454033210e77fb3a69c43 not found: ID does not exist" Jul 10 16:57:06.508172 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.508171 2578 scope.go:117] "RemoveContainer" containerID="44a361bf537e5f93a017ccd402d7c49ad4bdb3fd8fc9e482c2957e0304d752d3" Jul 10 16:57:06.508435 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.508411 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44a361bf537e5f93a017ccd402d7c49ad4bdb3fd8fc9e482c2957e0304d752d3"} err="failed to get container status \"44a361bf537e5f93a017ccd402d7c49ad4bdb3fd8fc9e482c2957e0304d752d3\": rpc error: code = NotFound desc = could not find container \"44a361bf537e5f93a017ccd402d7c49ad4bdb3fd8fc9e482c2957e0304d752d3\": container with ID starting with 44a361bf537e5f93a017ccd402d7c49ad4bdb3fd8fc9e482c2957e0304d752d3 not found: ID does not exist" Jul 10 16:57:06.508517 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.508437 2578 scope.go:117] "RemoveContainer" containerID="331e92f64ca5f409187e8e4d696bde59498ef92036cffdda86ad1bb78722f063" Jul 10 16:57:06.508758 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.508720 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"331e92f64ca5f409187e8e4d696bde59498ef92036cffdda86ad1bb78722f063"} err="failed to get container status \"331e92f64ca5f409187e8e4d696bde59498ef92036cffdda86ad1bb78722f063\": rpc error: code = NotFound desc = could not find container \"331e92f64ca5f409187e8e4d696bde59498ef92036cffdda86ad1bb78722f063\": container with ID starting with 331e92f64ca5f409187e8e4d696bde59498ef92036cffdda86ad1bb78722f063 not found: ID does not exist" Jul 10 16:57:06.508844 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.508760 2578 scope.go:117] "RemoveContainer" containerID="f69bc78775e591cb96b746738dfcb7cb6398ed6fe8c7bfab84f8e9b9d6a2d5a5" Jul 10 16:57:06.509576 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.509548 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f69bc78775e591cb96b746738dfcb7cb6398ed6fe8c7bfab84f8e9b9d6a2d5a5"} err="failed to get container status \"f69bc78775e591cb96b746738dfcb7cb6398ed6fe8c7bfab84f8e9b9d6a2d5a5\": rpc error: code = NotFound desc = could not find container \"f69bc78775e591cb96b746738dfcb7cb6398ed6fe8c7bfab84f8e9b9d6a2d5a5\": container with ID starting with f69bc78775e591cb96b746738dfcb7cb6398ed6fe8c7bfab84f8e9b9d6a2d5a5 not found: ID does not exist" Jul 10 16:57:06.509677 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.509576 2578 scope.go:117] "RemoveContainer" containerID="221094880b4b9763e20c54092552d25b98b0234b31420459d45590ef599e99d4" Jul 10 16:57:06.510164 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.510138 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"221094880b4b9763e20c54092552d25b98b0234b31420459d45590ef599e99d4"} err="failed to get container status \"221094880b4b9763e20c54092552d25b98b0234b31420459d45590ef599e99d4\": rpc error: code = NotFound desc = could not find container \"221094880b4b9763e20c54092552d25b98b0234b31420459d45590ef599e99d4\": container with ID starting with 221094880b4b9763e20c54092552d25b98b0234b31420459d45590ef599e99d4 not found: ID does not exist" Jul 10 16:57:06.510164 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.510157 2578 scope.go:117] "RemoveContainer" containerID="9bff15e61ab2f20af2353f41e781d84e88adfef68e00f9f1dc2497bdda5d2375" Jul 10 16:57:06.510393 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.510373 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bff15e61ab2f20af2353f41e781d84e88adfef68e00f9f1dc2497bdda5d2375"} err="failed to get container status \"9bff15e61ab2f20af2353f41e781d84e88adfef68e00f9f1dc2497bdda5d2375\": rpc error: code = NotFound desc = could not find container \"9bff15e61ab2f20af2353f41e781d84e88adfef68e00f9f1dc2497bdda5d2375\": container with ID starting with 9bff15e61ab2f20af2353f41e781d84e88adfef68e00f9f1dc2497bdda5d2375 not found: ID does not exist" Jul 10 16:57:06.510441 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.510395 2578 scope.go:117] "RemoveContainer" containerID="b13e0e6b0e2d1600d8f4331cec0e338a82bf916af0b5310bbb53801bbc7aefcd" Jul 10 16:57:06.510628 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.510603 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b13e0e6b0e2d1600d8f4331cec0e338a82bf916af0b5310bbb53801bbc7aefcd"} err="failed to get container status \"b13e0e6b0e2d1600d8f4331cec0e338a82bf916af0b5310bbb53801bbc7aefcd\": rpc error: code = NotFound desc = could not find container \"b13e0e6b0e2d1600d8f4331cec0e338a82bf916af0b5310bbb53801bbc7aefcd\": container with ID starting with b13e0e6b0e2d1600d8f4331cec0e338a82bf916af0b5310bbb53801bbc7aefcd not found: ID does not exist" Jul 10 16:57:06.510750 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.510629 2578 scope.go:117] "RemoveContainer" containerID="45c7105720d4b5f4bffd225882fcb7dfe3ba576af42454033210e77fb3a69c43" Jul 10 16:57:06.510877 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.510858 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45c7105720d4b5f4bffd225882fcb7dfe3ba576af42454033210e77fb3a69c43"} err="failed to get container status \"45c7105720d4b5f4bffd225882fcb7dfe3ba576af42454033210e77fb3a69c43\": rpc error: code = NotFound desc = could not find container \"45c7105720d4b5f4bffd225882fcb7dfe3ba576af42454033210e77fb3a69c43\": container with ID starting with 45c7105720d4b5f4bffd225882fcb7dfe3ba576af42454033210e77fb3a69c43 not found: ID does not exist" Jul 10 16:57:06.510925 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.510878 2578 scope.go:117] "RemoveContainer" containerID="44a361bf537e5f93a017ccd402d7c49ad4bdb3fd8fc9e482c2957e0304d752d3" Jul 10 16:57:06.511209 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.511183 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44a361bf537e5f93a017ccd402d7c49ad4bdb3fd8fc9e482c2957e0304d752d3"} err="failed to get container status \"44a361bf537e5f93a017ccd402d7c49ad4bdb3fd8fc9e482c2957e0304d752d3\": rpc error: code = NotFound desc = could not find container \"44a361bf537e5f93a017ccd402d7c49ad4bdb3fd8fc9e482c2957e0304d752d3\": container with ID starting with 44a361bf537e5f93a017ccd402d7c49ad4bdb3fd8fc9e482c2957e0304d752d3 not found: ID does not exist" Jul 10 16:57:06.511310 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.511210 2578 scope.go:117] "RemoveContainer" containerID="331e92f64ca5f409187e8e4d696bde59498ef92036cffdda86ad1bb78722f063" Jul 10 16:57:06.511468 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.511447 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"331e92f64ca5f409187e8e4d696bde59498ef92036cffdda86ad1bb78722f063"} err="failed to get container status \"331e92f64ca5f409187e8e4d696bde59498ef92036cffdda86ad1bb78722f063\": rpc error: code = NotFound desc = could not find container \"331e92f64ca5f409187e8e4d696bde59498ef92036cffdda86ad1bb78722f063\": container with ID starting with 331e92f64ca5f409187e8e4d696bde59498ef92036cffdda86ad1bb78722f063 not found: ID does not exist" Jul 10 16:57:06.511526 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.511469 2578 scope.go:117] "RemoveContainer" containerID="f69bc78775e591cb96b746738dfcb7cb6398ed6fe8c7bfab84f8e9b9d6a2d5a5" Jul 10 16:57:06.511714 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.511694 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f69bc78775e591cb96b746738dfcb7cb6398ed6fe8c7bfab84f8e9b9d6a2d5a5"} err="failed to get container status \"f69bc78775e591cb96b746738dfcb7cb6398ed6fe8c7bfab84f8e9b9d6a2d5a5\": rpc error: code = NotFound desc = could not find container \"f69bc78775e591cb96b746738dfcb7cb6398ed6fe8c7bfab84f8e9b9d6a2d5a5\": container with ID starting with f69bc78775e591cb96b746738dfcb7cb6398ed6fe8c7bfab84f8e9b9d6a2d5a5 not found: ID does not exist" Jul 10 16:57:06.511775 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.511714 2578 scope.go:117] "RemoveContainer" containerID="221094880b4b9763e20c54092552d25b98b0234b31420459d45590ef599e99d4" Jul 10 16:57:06.511947 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.511929 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"221094880b4b9763e20c54092552d25b98b0234b31420459d45590ef599e99d4"} err="failed to get container status \"221094880b4b9763e20c54092552d25b98b0234b31420459d45590ef599e99d4\": rpc error: code = NotFound desc = could not find container \"221094880b4b9763e20c54092552d25b98b0234b31420459d45590ef599e99d4\": container with ID starting with 221094880b4b9763e20c54092552d25b98b0234b31420459d45590ef599e99d4 not found: ID does not exist" Jul 10 16:57:06.511991 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.511946 2578 scope.go:117] "RemoveContainer" containerID="9bff15e61ab2f20af2353f41e781d84e88adfef68e00f9f1dc2497bdda5d2375" Jul 10 16:57:06.512204 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.512188 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bff15e61ab2f20af2353f41e781d84e88adfef68e00f9f1dc2497bdda5d2375"} err="failed to get container status \"9bff15e61ab2f20af2353f41e781d84e88adfef68e00f9f1dc2497bdda5d2375\": rpc error: code = NotFound desc = could not find container \"9bff15e61ab2f20af2353f41e781d84e88adfef68e00f9f1dc2497bdda5d2375\": container with ID starting with 9bff15e61ab2f20af2353f41e781d84e88adfef68e00f9f1dc2497bdda5d2375 not found: ID does not exist" Jul 10 16:57:06.512252 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.512205 2578 scope.go:117] "RemoveContainer" containerID="b13e0e6b0e2d1600d8f4331cec0e338a82bf916af0b5310bbb53801bbc7aefcd" Jul 10 16:57:06.512436 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.512417 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b13e0e6b0e2d1600d8f4331cec0e338a82bf916af0b5310bbb53801bbc7aefcd"} err="failed to get container status \"b13e0e6b0e2d1600d8f4331cec0e338a82bf916af0b5310bbb53801bbc7aefcd\": rpc error: code = NotFound desc = could not find container \"b13e0e6b0e2d1600d8f4331cec0e338a82bf916af0b5310bbb53801bbc7aefcd\": container with ID starting with b13e0e6b0e2d1600d8f4331cec0e338a82bf916af0b5310bbb53801bbc7aefcd not found: ID does not exist" Jul 10 16:57:06.512507 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.512439 2578 scope.go:117] "RemoveContainer" containerID="45c7105720d4b5f4bffd225882fcb7dfe3ba576af42454033210e77fb3a69c43" Jul 10 16:57:06.512688 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.512672 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45c7105720d4b5f4bffd225882fcb7dfe3ba576af42454033210e77fb3a69c43"} err="failed to get container status \"45c7105720d4b5f4bffd225882fcb7dfe3ba576af42454033210e77fb3a69c43\": rpc error: code = NotFound desc = could not find container \"45c7105720d4b5f4bffd225882fcb7dfe3ba576af42454033210e77fb3a69c43\": container with ID starting with 45c7105720d4b5f4bffd225882fcb7dfe3ba576af42454033210e77fb3a69c43 not found: ID does not exist" Jul 10 16:57:06.512749 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.512689 2578 scope.go:117] "RemoveContainer" containerID="44a361bf537e5f93a017ccd402d7c49ad4bdb3fd8fc9e482c2957e0304d752d3" Jul 10 16:57:06.512961 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.512942 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44a361bf537e5f93a017ccd402d7c49ad4bdb3fd8fc9e482c2957e0304d752d3"} err="failed to get container status \"44a361bf537e5f93a017ccd402d7c49ad4bdb3fd8fc9e482c2957e0304d752d3\": rpc error: code = NotFound desc = could not find container \"44a361bf537e5f93a017ccd402d7c49ad4bdb3fd8fc9e482c2957e0304d752d3\": container with ID starting with 44a361bf537e5f93a017ccd402d7c49ad4bdb3fd8fc9e482c2957e0304d752d3 not found: ID does not exist" Jul 10 16:57:06.513011 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.512963 2578 scope.go:117] "RemoveContainer" containerID="331e92f64ca5f409187e8e4d696bde59498ef92036cffdda86ad1bb78722f063" Jul 10 16:57:06.513209 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.513192 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"331e92f64ca5f409187e8e4d696bde59498ef92036cffdda86ad1bb78722f063"} err="failed to get container status \"331e92f64ca5f409187e8e4d696bde59498ef92036cffdda86ad1bb78722f063\": rpc error: code = NotFound desc = could not find container \"331e92f64ca5f409187e8e4d696bde59498ef92036cffdda86ad1bb78722f063\": container with ID starting with 331e92f64ca5f409187e8e4d696bde59498ef92036cffdda86ad1bb78722f063 not found: ID does not exist" Jul 10 16:57:06.513286 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.513211 2578 scope.go:117] "RemoveContainer" containerID="f69bc78775e591cb96b746738dfcb7cb6398ed6fe8c7bfab84f8e9b9d6a2d5a5" Jul 10 16:57:06.513472 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.513455 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f69bc78775e591cb96b746738dfcb7cb6398ed6fe8c7bfab84f8e9b9d6a2d5a5"} err="failed to get container status \"f69bc78775e591cb96b746738dfcb7cb6398ed6fe8c7bfab84f8e9b9d6a2d5a5\": rpc error: code = NotFound desc = could not find container \"f69bc78775e591cb96b746738dfcb7cb6398ed6fe8c7bfab84f8e9b9d6a2d5a5\": container with ID starting with f69bc78775e591cb96b746738dfcb7cb6398ed6fe8c7bfab84f8e9b9d6a2d5a5 not found: ID does not exist" Jul 10 16:57:06.513526 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.513472 2578 scope.go:117] "RemoveContainer" containerID="221094880b4b9763e20c54092552d25b98b0234b31420459d45590ef599e99d4" Jul 10 16:57:06.513678 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.513659 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"221094880b4b9763e20c54092552d25b98b0234b31420459d45590ef599e99d4"} err="failed to get container status \"221094880b4b9763e20c54092552d25b98b0234b31420459d45590ef599e99d4\": rpc error: code = NotFound desc = could not find container \"221094880b4b9763e20c54092552d25b98b0234b31420459d45590ef599e99d4\": container with ID starting with 221094880b4b9763e20c54092552d25b98b0234b31420459d45590ef599e99d4 not found: ID does not exist" Jul 10 16:57:06.513728 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.513677 2578 scope.go:117] "RemoveContainer" containerID="9bff15e61ab2f20af2353f41e781d84e88adfef68e00f9f1dc2497bdda5d2375" Jul 10 16:57:06.513921 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.513900 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bff15e61ab2f20af2353f41e781d84e88adfef68e00f9f1dc2497bdda5d2375"} err="failed to get container status \"9bff15e61ab2f20af2353f41e781d84e88adfef68e00f9f1dc2497bdda5d2375\": rpc error: code = NotFound desc = could not find container \"9bff15e61ab2f20af2353f41e781d84e88adfef68e00f9f1dc2497bdda5d2375\": container with ID starting with 9bff15e61ab2f20af2353f41e781d84e88adfef68e00f9f1dc2497bdda5d2375 not found: ID does not exist" Jul 10 16:57:06.513983 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.513925 2578 scope.go:117] "RemoveContainer" containerID="b13e0e6b0e2d1600d8f4331cec0e338a82bf916af0b5310bbb53801bbc7aefcd" Jul 10 16:57:06.514128 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.514113 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b13e0e6b0e2d1600d8f4331cec0e338a82bf916af0b5310bbb53801bbc7aefcd"} err="failed to get container status \"b13e0e6b0e2d1600d8f4331cec0e338a82bf916af0b5310bbb53801bbc7aefcd\": rpc error: code = NotFound desc = could not find container \"b13e0e6b0e2d1600d8f4331cec0e338a82bf916af0b5310bbb53801bbc7aefcd\": container with ID starting with b13e0e6b0e2d1600d8f4331cec0e338a82bf916af0b5310bbb53801bbc7aefcd not found: ID does not exist" Jul 10 16:57:06.514185 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.514130 2578 scope.go:117] "RemoveContainer" containerID="45c7105720d4b5f4bffd225882fcb7dfe3ba576af42454033210e77fb3a69c43" Jul 10 16:57:06.514360 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.514329 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45c7105720d4b5f4bffd225882fcb7dfe3ba576af42454033210e77fb3a69c43"} err="failed to get container status \"45c7105720d4b5f4bffd225882fcb7dfe3ba576af42454033210e77fb3a69c43\": rpc error: code = NotFound desc = could not find container \"45c7105720d4b5f4bffd225882fcb7dfe3ba576af42454033210e77fb3a69c43\": container with ID starting with 45c7105720d4b5f4bffd225882fcb7dfe3ba576af42454033210e77fb3a69c43 not found: ID does not exist" Jul 10 16:57:06.514437 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.514363 2578 scope.go:117] "RemoveContainer" containerID="44a361bf537e5f93a017ccd402d7c49ad4bdb3fd8fc9e482c2957e0304d752d3" Jul 10 16:57:06.514561 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.514545 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44a361bf537e5f93a017ccd402d7c49ad4bdb3fd8fc9e482c2957e0304d752d3"} err="failed to get container status \"44a361bf537e5f93a017ccd402d7c49ad4bdb3fd8fc9e482c2957e0304d752d3\": rpc error: code = NotFound desc = could not find container \"44a361bf537e5f93a017ccd402d7c49ad4bdb3fd8fc9e482c2957e0304d752d3\": container with ID starting with 44a361bf537e5f93a017ccd402d7c49ad4bdb3fd8fc9e482c2957e0304d752d3 not found: ID does not exist" Jul 10 16:57:06.514612 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.514560 2578 scope.go:117] "RemoveContainer" containerID="331e92f64ca5f409187e8e4d696bde59498ef92036cffdda86ad1bb78722f063" Jul 10 16:57:06.514735 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.514717 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"331e92f64ca5f409187e8e4d696bde59498ef92036cffdda86ad1bb78722f063"} err="failed to get container status \"331e92f64ca5f409187e8e4d696bde59498ef92036cffdda86ad1bb78722f063\": rpc error: code = NotFound desc = could not find container \"331e92f64ca5f409187e8e4d696bde59498ef92036cffdda86ad1bb78722f063\": container with ID starting with 331e92f64ca5f409187e8e4d696bde59498ef92036cffdda86ad1bb78722f063 not found: ID does not exist" Jul 10 16:57:06.514802 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.514738 2578 scope.go:117] "RemoveContainer" containerID="f69bc78775e591cb96b746738dfcb7cb6398ed6fe8c7bfab84f8e9b9d6a2d5a5" Jul 10 16:57:06.514955 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.514937 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f69bc78775e591cb96b746738dfcb7cb6398ed6fe8c7bfab84f8e9b9d6a2d5a5"} err="failed to get container status \"f69bc78775e591cb96b746738dfcb7cb6398ed6fe8c7bfab84f8e9b9d6a2d5a5\": rpc error: code = NotFound desc = could not find container \"f69bc78775e591cb96b746738dfcb7cb6398ed6fe8c7bfab84f8e9b9d6a2d5a5\": container with ID starting with f69bc78775e591cb96b746738dfcb7cb6398ed6fe8c7bfab84f8e9b9d6a2d5a5 not found: ID does not exist" Jul 10 16:57:06.514955 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.514955 2578 scope.go:117] "RemoveContainer" containerID="221094880b4b9763e20c54092552d25b98b0234b31420459d45590ef599e99d4" Jul 10 16:57:06.515197 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.515174 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"221094880b4b9763e20c54092552d25b98b0234b31420459d45590ef599e99d4"} err="failed to get container status \"221094880b4b9763e20c54092552d25b98b0234b31420459d45590ef599e99d4\": rpc error: code = NotFound desc = could not find container \"221094880b4b9763e20c54092552d25b98b0234b31420459d45590ef599e99d4\": container with ID starting with 221094880b4b9763e20c54092552d25b98b0234b31420459d45590ef599e99d4 not found: ID does not exist" Jul 10 16:57:06.515237 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.515202 2578 scope.go:117] "RemoveContainer" containerID="9bff15e61ab2f20af2353f41e781d84e88adfef68e00f9f1dc2497bdda5d2375" Jul 10 16:57:06.515469 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.515451 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bff15e61ab2f20af2353f41e781d84e88adfef68e00f9f1dc2497bdda5d2375"} err="failed to get container status \"9bff15e61ab2f20af2353f41e781d84e88adfef68e00f9f1dc2497bdda5d2375\": rpc error: code = NotFound desc = could not find container \"9bff15e61ab2f20af2353f41e781d84e88adfef68e00f9f1dc2497bdda5d2375\": container with ID starting with 9bff15e61ab2f20af2353f41e781d84e88adfef68e00f9f1dc2497bdda5d2375 not found: ID does not exist" Jul 10 16:57:06.515531 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.515472 2578 scope.go:117] "RemoveContainer" containerID="b13e0e6b0e2d1600d8f4331cec0e338a82bf916af0b5310bbb53801bbc7aefcd" Jul 10 16:57:06.515703 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.515685 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b13e0e6b0e2d1600d8f4331cec0e338a82bf916af0b5310bbb53801bbc7aefcd"} err="failed to get container status \"b13e0e6b0e2d1600d8f4331cec0e338a82bf916af0b5310bbb53801bbc7aefcd\": rpc error: code = NotFound desc = could not find container \"b13e0e6b0e2d1600d8f4331cec0e338a82bf916af0b5310bbb53801bbc7aefcd\": container with ID starting with b13e0e6b0e2d1600d8f4331cec0e338a82bf916af0b5310bbb53801bbc7aefcd not found: ID does not exist" Jul 10 16:57:06.515751 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.515705 2578 scope.go:117] "RemoveContainer" containerID="45c7105720d4b5f4bffd225882fcb7dfe3ba576af42454033210e77fb3a69c43" Jul 10 16:57:06.515941 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.515924 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45c7105720d4b5f4bffd225882fcb7dfe3ba576af42454033210e77fb3a69c43"} err="failed to get container status \"45c7105720d4b5f4bffd225882fcb7dfe3ba576af42454033210e77fb3a69c43\": rpc error: code = NotFound desc = could not find container \"45c7105720d4b5f4bffd225882fcb7dfe3ba576af42454033210e77fb3a69c43\": container with ID starting with 45c7105720d4b5f4bffd225882fcb7dfe3ba576af42454033210e77fb3a69c43 not found: ID does not exist" Jul 10 16:57:06.515980 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.515942 2578 scope.go:117] "RemoveContainer" containerID="44a361bf537e5f93a017ccd402d7c49ad4bdb3fd8fc9e482c2957e0304d752d3" Jul 10 16:57:06.516226 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.516199 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44a361bf537e5f93a017ccd402d7c49ad4bdb3fd8fc9e482c2957e0304d752d3"} err="failed to get container status \"44a361bf537e5f93a017ccd402d7c49ad4bdb3fd8fc9e482c2957e0304d752d3\": rpc error: code = NotFound desc = could not find container \"44a361bf537e5f93a017ccd402d7c49ad4bdb3fd8fc9e482c2957e0304d752d3\": container with ID starting with 44a361bf537e5f93a017ccd402d7c49ad4bdb3fd8fc9e482c2957e0304d752d3 not found: ID does not exist" Jul 10 16:57:06.516271 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.516226 2578 scope.go:117] "RemoveContainer" containerID="331e92f64ca5f409187e8e4d696bde59498ef92036cffdda86ad1bb78722f063" Jul 10 16:57:06.516475 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.516459 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"331e92f64ca5f409187e8e4d696bde59498ef92036cffdda86ad1bb78722f063"} err="failed to get container status \"331e92f64ca5f409187e8e4d696bde59498ef92036cffdda86ad1bb78722f063\": rpc error: code = NotFound desc = could not find container \"331e92f64ca5f409187e8e4d696bde59498ef92036cffdda86ad1bb78722f063\": container with ID starting with 331e92f64ca5f409187e8e4d696bde59498ef92036cffdda86ad1bb78722f063 not found: ID does not exist" Jul 10 16:57:06.516518 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.516476 2578 scope.go:117] "RemoveContainer" containerID="f69bc78775e591cb96b746738dfcb7cb6398ed6fe8c7bfab84f8e9b9d6a2d5a5" Jul 10 16:57:06.516688 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.516669 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f69bc78775e591cb96b746738dfcb7cb6398ed6fe8c7bfab84f8e9b9d6a2d5a5"} err="failed to get container status \"f69bc78775e591cb96b746738dfcb7cb6398ed6fe8c7bfab84f8e9b9d6a2d5a5\": rpc error: code = NotFound desc = could not find container \"f69bc78775e591cb96b746738dfcb7cb6398ed6fe8c7bfab84f8e9b9d6a2d5a5\": container with ID starting with f69bc78775e591cb96b746738dfcb7cb6398ed6fe8c7bfab84f8e9b9d6a2d5a5 not found: ID does not exist" Jul 10 16:57:06.516755 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.516690 2578 scope.go:117] "RemoveContainer" containerID="221094880b4b9763e20c54092552d25b98b0234b31420459d45590ef599e99d4" Jul 10 16:57:06.516952 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.516934 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"221094880b4b9763e20c54092552d25b98b0234b31420459d45590ef599e99d4"} err="failed to get container status \"221094880b4b9763e20c54092552d25b98b0234b31420459d45590ef599e99d4\": rpc error: code = NotFound desc = could not find container \"221094880b4b9763e20c54092552d25b98b0234b31420459d45590ef599e99d4\": container with ID starting with 221094880b4b9763e20c54092552d25b98b0234b31420459d45590ef599e99d4 not found: ID does not exist" Jul 10 16:57:06.516952 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.516951 2578 scope.go:117] "RemoveContainer" containerID="9bff15e61ab2f20af2353f41e781d84e88adfef68e00f9f1dc2497bdda5d2375" Jul 10 16:57:06.517218 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.517199 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bff15e61ab2f20af2353f41e781d84e88adfef68e00f9f1dc2497bdda5d2375"} err="failed to get container status \"9bff15e61ab2f20af2353f41e781d84e88adfef68e00f9f1dc2497bdda5d2375\": rpc error: code = NotFound desc = could not find container \"9bff15e61ab2f20af2353f41e781d84e88adfef68e00f9f1dc2497bdda5d2375\": container with ID starting with 9bff15e61ab2f20af2353f41e781d84e88adfef68e00f9f1dc2497bdda5d2375 not found: ID does not exist" Jul 10 16:57:06.579817 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.579724 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.579817 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.579769 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.579817 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.579788 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.579817 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.579805 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.580357 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.579822 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78mn2\" (UniqueName: \"kubernetes.io/projected/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-kube-api-access-78mn2\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.580357 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.579842 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.580357 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.579863 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.580357 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.579880 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.580357 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.579941 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.580357 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.579991 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.580357 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.580031 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-web-config\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.580357 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.580089 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.580357 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.580115 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.580357 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.580134 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-config\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.580357 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.580153 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.580357 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.580177 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.580357 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.580217 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.580357 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.580238 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.580357 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.580257 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.580357 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.580281 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-config-out\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.680689 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.680641 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.680689 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.680691 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.680945 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.680717 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.680945 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.680746 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-web-config\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.680945 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.680769 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.680945 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.680800 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.680945 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.680823 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-config\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.680945 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.680849 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.680945 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.680868 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.680945 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.680908 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.680945 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.680942 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.681420 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.680975 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.681420 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.681020 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-config-out\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.681420 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.681092 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.681420 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.681123 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.681420 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.681152 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.681420 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.681180 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.681420 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.681204 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78mn2\" (UniqueName: \"kubernetes.io/projected/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-kube-api-access-78mn2\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.681420 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.681240 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.681420 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.681253 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.681420 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.681269 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.682104 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.681476 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.682104 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.681489 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.682104 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.681953 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.683107 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.683077 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.683672 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.683638 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.683893 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.683872 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.684215 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.684191 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-web-config\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.684306 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.684191 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.684373 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.684252 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-config-out\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.684966 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.684571 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.684966 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.684765 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.685790 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.685741 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.686533 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.686511 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-config\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.686660 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.686639 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.686869 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.686849 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.686906 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.686872 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.687277 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.687260 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.687483 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.687466 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.693998 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.693969 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78mn2\" (UniqueName: \"kubernetes.io/projected/82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99-kube-api-access-78mn2\") pod \"prometheus-k8s-0\" (UID: \"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.810742 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.810698 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:57:06.969437 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:06.969355 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 10 16:57:06.972403 ip-10-0-136-7 kubenswrapper[2578]: W0710 16:57:06.972362 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82c44fc5_e1fa_487c_a4ce_d2ec8ecf4c99.slice/crio-d24592642d5589d8684fc1b7db666959365422bad6b84e8f4acd2c69f6f63132 WatchSource:0}: Error finding container d24592642d5589d8684fc1b7db666959365422bad6b84e8f4acd2c69f6f63132: Status 404 returned error can't find the container with id d24592642d5589d8684fc1b7db666959365422bad6b84e8f4acd2c69f6f63132 Jul 10 16:57:07.446531 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:07.446443 2578 generic.go:356] "Generic (PLEG): container finished" podID="82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99" containerID="416f49de8b62011ed2bd93efb72afd67ecba59ee9ff08ec3398c50d09c9f8e4f" exitCode=0 Jul 10 16:57:07.446695 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:07.446539 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99","Type":"ContainerDied","Data":"416f49de8b62011ed2bd93efb72afd67ecba59ee9ff08ec3398c50d09c9f8e4f"} Jul 10 16:57:07.446695 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:07.446632 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99","Type":"ContainerStarted","Data":"d24592642d5589d8684fc1b7db666959365422bad6b84e8f4acd2c69f6f63132"} Jul 10 16:57:08.427214 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:08.427177 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be8c418d-b1dc-44c2-9b59-7b8e673f822a" path="/var/lib/kubelet/pods/be8c418d-b1dc-44c2-9b59-7b8e673f822a/volumes" Jul 10 16:57:08.454906 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:08.454879 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-66d47b6679-hxmgv" event={"ID":"399a676f-ae4a-42e5-aa7f-ac8a0428131b","Type":"ContainerStarted","Data":"d9de2e6f35f3a3bc90570cd8fb86a8854d6d19fd1771724b91576a45983ad456"} Jul 10 16:57:08.455071 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:08.454949 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-66d47b6679-hxmgv" event={"ID":"399a676f-ae4a-42e5-aa7f-ac8a0428131b","Type":"ContainerStarted","Data":"ffa5d87cbb1f572c50be9aa677e1845755650e2799c8cafad78bbf4f29e40288"} Jul 10 16:57:08.458671 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:08.458636 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99","Type":"ContainerStarted","Data":"4a9a488c77ddb3ba6f12cc3db32d094df6f472323e0ff02041701d8c2165468e"} Jul 10 16:57:08.458817 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:08.458729 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99","Type":"ContainerStarted","Data":"3d668a472d6adfea890fb2a09effc04faced8930c137f0b2c0aa2635176357b5"} Jul 10 16:57:08.458817 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:08.458775 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99","Type":"ContainerStarted","Data":"bd4dacf5dfcfaa6a99a3a5469c5e6b25b95ade8a87472fa455e6a0d36888c1ef"} Jul 10 16:57:08.458918 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:08.458819 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99","Type":"ContainerStarted","Data":"1b20c06913db4023ca78ecff9d2ceb2e881cdd6eabef4f8205b118fa35e6384b"} Jul 10 16:57:08.458918 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:08.458860 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99","Type":"ContainerStarted","Data":"c481f65174daf23e16ab164401490f974dfe826c9b4670d9afa4131a2406ee23"} Jul 10 16:57:08.458918 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:08.458897 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99","Type":"ContainerStarted","Data":"7344c66ab021b397d199372e2fc4b0a35688d1857fc0254ea2c06af35e06e416"} Jul 10 16:57:08.499622 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:08.499551 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.475603623 podStartE2EDuration="2.499526357s" totalImagesPullingTime="0s" totalInitContainerRuntime="23.922734ms" isStatefulPod=false podCreationTimestamp="2026-07-10 16:57:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:57:08.497336508 +0000 UTC m=+172.586665559" watchObservedRunningTime="2026-07-10 16:57:08.499526357 +0000 UTC m=+172.588855409" Jul 10 16:57:09.468099 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:09.468039 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-66d47b6679-hxmgv" event={"ID":"399a676f-ae4a-42e5-aa7f-ac8a0428131b","Type":"ContainerStarted","Data":"711168a98d1ae20d6d04bb31721ddbd2ae481f032248564fc8fab4815112f844"} Jul 10 16:57:09.491945 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:09.491880 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-66d47b6679-hxmgv" podStartSLOduration=2.331694143 podStartE2EDuration="4.491854347s" totalImagesPullingTime="2.160160204s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:57:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:57:09.490382402 +0000 UTC m=+173.579711452" watchObservedRunningTime="2026-07-10 16:57:09.491854347 +0000 UTC m=+173.581183398" Jul 10 16:57:11.811765 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:57:11.811718 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:58:06.810947 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:58:06.810909 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:58:06.827724 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:58:06.827693 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:58:07.723002 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:58:07.722968 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 10 16:59:16.292854 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:59:16.292821 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-5j4ws_27742f15-b602-4554-84dc-ff9d46a9adda/console-operator/2.log" Jul 10 16:59:16.295068 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:59:16.295029 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-5j4ws_27742f15-b602-4554-84dc-ff9d46a9adda/console-operator/2.log" Jul 10 16:59:16.300325 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:59:16.300298 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gqgkg_b9a53126-f278-4a9d-9735-5f73dd0c1964/ovn-acl-logging/0.log" Jul 10 16:59:16.302160 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:59:16.302134 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gqgkg_b9a53126-f278-4a9d-9735-5f73dd0c1964/ovn-acl-logging/0.log" Jul 10 16:59:16.303418 ip-10-0-136-7 kubenswrapper[2578]: I0710 16:59:16.303401 2578 kubelet.go:1648] "Image garbage collection succeeded" Jul 10 17:00:20.139342 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:20.139253 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-67764b8b78-tmqk6"] Jul 10 17:00:20.142230 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:20.142213 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-67764b8b78-tmqk6" Jul 10 17:00:20.144999 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:20.144970 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 10 17:00:20.145132 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:20.145022 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 10 17:00:20.145182 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:20.145155 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 10 17:00:20.145495 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:20.145481 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 10 17:00:20.145645 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:20.145630 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-gvgdz\"" Jul 10 17:00:20.162366 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:20.162339 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-67764b8b78-tmqk6"] Jul 10 17:00:20.260490 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:20.260451 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fcde3330-8614-47ac-9c0f-93c34a0cfc17-apiservice-cert\") pod \"opendatahub-operator-controller-manager-67764b8b78-tmqk6\" (UID: \"fcde3330-8614-47ac-9c0f-93c34a0cfc17\") " pod="opendatahub/opendatahub-operator-controller-manager-67764b8b78-tmqk6" Jul 10 17:00:20.260665 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:20.260542 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/fcde3330-8614-47ac-9c0f-93c34a0cfc17-manifests\") pod \"opendatahub-operator-controller-manager-67764b8b78-tmqk6\" (UID: \"fcde3330-8614-47ac-9c0f-93c34a0cfc17\") " pod="opendatahub/opendatahub-operator-controller-manager-67764b8b78-tmqk6" Jul 10 17:00:20.260665 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:20.260595 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fcde3330-8614-47ac-9c0f-93c34a0cfc17-tmp\") pod \"opendatahub-operator-controller-manager-67764b8b78-tmqk6\" (UID: \"fcde3330-8614-47ac-9c0f-93c34a0cfc17\") " pod="opendatahub/opendatahub-operator-controller-manager-67764b8b78-tmqk6" Jul 10 17:00:20.260665 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:20.260615 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fcde3330-8614-47ac-9c0f-93c34a0cfc17-webhook-cert\") pod \"opendatahub-operator-controller-manager-67764b8b78-tmqk6\" (UID: \"fcde3330-8614-47ac-9c0f-93c34a0cfc17\") " pod="opendatahub/opendatahub-operator-controller-manager-67764b8b78-tmqk6" Jul 10 17:00:20.260769 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:20.260666 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjwrp\" (UniqueName: \"kubernetes.io/projected/fcde3330-8614-47ac-9c0f-93c34a0cfc17-kube-api-access-pjwrp\") pod \"opendatahub-operator-controller-manager-67764b8b78-tmqk6\" (UID: \"fcde3330-8614-47ac-9c0f-93c34a0cfc17\") " pod="opendatahub/opendatahub-operator-controller-manager-67764b8b78-tmqk6" Jul 10 17:00:20.362063 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:20.362015 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjwrp\" (UniqueName: \"kubernetes.io/projected/fcde3330-8614-47ac-9c0f-93c34a0cfc17-kube-api-access-pjwrp\") pod \"opendatahub-operator-controller-manager-67764b8b78-tmqk6\" (UID: \"fcde3330-8614-47ac-9c0f-93c34a0cfc17\") " pod="opendatahub/opendatahub-operator-controller-manager-67764b8b78-tmqk6" Jul 10 17:00:20.362259 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:20.362118 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fcde3330-8614-47ac-9c0f-93c34a0cfc17-apiservice-cert\") pod \"opendatahub-operator-controller-manager-67764b8b78-tmqk6\" (UID: \"fcde3330-8614-47ac-9c0f-93c34a0cfc17\") " pod="opendatahub/opendatahub-operator-controller-manager-67764b8b78-tmqk6" Jul 10 17:00:20.362259 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:20.362171 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/fcde3330-8614-47ac-9c0f-93c34a0cfc17-manifests\") pod \"opendatahub-operator-controller-manager-67764b8b78-tmqk6\" (UID: \"fcde3330-8614-47ac-9c0f-93c34a0cfc17\") " pod="opendatahub/opendatahub-operator-controller-manager-67764b8b78-tmqk6" Jul 10 17:00:20.362259 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:20.362211 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fcde3330-8614-47ac-9c0f-93c34a0cfc17-tmp\") pod \"opendatahub-operator-controller-manager-67764b8b78-tmqk6\" (UID: \"fcde3330-8614-47ac-9c0f-93c34a0cfc17\") " pod="opendatahub/opendatahub-operator-controller-manager-67764b8b78-tmqk6" Jul 10 17:00:20.362259 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:20.362234 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fcde3330-8614-47ac-9c0f-93c34a0cfc17-webhook-cert\") pod \"opendatahub-operator-controller-manager-67764b8b78-tmqk6\" (UID: \"fcde3330-8614-47ac-9c0f-93c34a0cfc17\") " pod="opendatahub/opendatahub-operator-controller-manager-67764b8b78-tmqk6" Jul 10 17:00:20.362650 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:20.362614 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/fcde3330-8614-47ac-9c0f-93c34a0cfc17-manifests\") pod \"opendatahub-operator-controller-manager-67764b8b78-tmqk6\" (UID: \"fcde3330-8614-47ac-9c0f-93c34a0cfc17\") " pod="opendatahub/opendatahub-operator-controller-manager-67764b8b78-tmqk6" Jul 10 17:00:20.364531 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:20.364508 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fcde3330-8614-47ac-9c0f-93c34a0cfc17-tmp\") pod \"opendatahub-operator-controller-manager-67764b8b78-tmqk6\" (UID: \"fcde3330-8614-47ac-9c0f-93c34a0cfc17\") " pod="opendatahub/opendatahub-operator-controller-manager-67764b8b78-tmqk6" Jul 10 17:00:20.364739 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:20.364722 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fcde3330-8614-47ac-9c0f-93c34a0cfc17-apiservice-cert\") pod \"opendatahub-operator-controller-manager-67764b8b78-tmqk6\" (UID: \"fcde3330-8614-47ac-9c0f-93c34a0cfc17\") " pod="opendatahub/opendatahub-operator-controller-manager-67764b8b78-tmqk6" Jul 10 17:00:20.364808 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:20.364764 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fcde3330-8614-47ac-9c0f-93c34a0cfc17-webhook-cert\") pod \"opendatahub-operator-controller-manager-67764b8b78-tmqk6\" (UID: \"fcde3330-8614-47ac-9c0f-93c34a0cfc17\") " pod="opendatahub/opendatahub-operator-controller-manager-67764b8b78-tmqk6" Jul 10 17:00:20.370215 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:20.370194 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjwrp\" (UniqueName: \"kubernetes.io/projected/fcde3330-8614-47ac-9c0f-93c34a0cfc17-kube-api-access-pjwrp\") pod \"opendatahub-operator-controller-manager-67764b8b78-tmqk6\" (UID: \"fcde3330-8614-47ac-9c0f-93c34a0cfc17\") " pod="opendatahub/opendatahub-operator-controller-manager-67764b8b78-tmqk6" Jul 10 17:00:20.452453 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:20.452371 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-67764b8b78-tmqk6" Jul 10 17:00:20.592032 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:20.592009 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-67764b8b78-tmqk6"] Jul 10 17:00:20.594894 ip-10-0-136-7 kubenswrapper[2578]: W0710 17:00:20.594867 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcde3330_8614_47ac_9c0f_93c34a0cfc17.slice/crio-5bf3de04c42f7ffbe0ce427009ea080090fd8be750c08ff0409a41c117a3b884 WatchSource:0}: Error finding container 5bf3de04c42f7ffbe0ce427009ea080090fd8be750c08ff0409a41c117a3b884: Status 404 returned error can't find the container with id 5bf3de04c42f7ffbe0ce427009ea080090fd8be750c08ff0409a41c117a3b884 Jul 10 17:00:20.596682 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:20.596659 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 17:00:21.229039 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:21.228986 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67764b8b78-tmqk6" event={"ID":"fcde3330-8614-47ac-9c0f-93c34a0cfc17","Type":"ContainerStarted","Data":"5bf3de04c42f7ffbe0ce427009ea080090fd8be750c08ff0409a41c117a3b884"} Jul 10 17:00:24.244895 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:24.244785 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67764b8b78-tmqk6" event={"ID":"fcde3330-8614-47ac-9c0f-93c34a0cfc17","Type":"ContainerStarted","Data":"55ecd74f04f49260262414f12a6ef8aa8bf94272c9e5797c120ffd560a8251e7"} Jul 10 17:00:25.250258 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:25.250163 2578 generic.go:356] "Generic (PLEG): container finished" podID="fcde3330-8614-47ac-9c0f-93c34a0cfc17" containerID="55ecd74f04f49260262414f12a6ef8aa8bf94272c9e5797c120ffd560a8251e7" exitCode=0 Jul 10 17:00:25.250258 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:25.250228 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67764b8b78-tmqk6" event={"ID":"fcde3330-8614-47ac-9c0f-93c34a0cfc17","Type":"ContainerDied","Data":"55ecd74f04f49260262414f12a6ef8aa8bf94272c9e5797c120ffd560a8251e7"} Jul 10 17:00:26.256745 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:26.256709 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67764b8b78-tmqk6" event={"ID":"fcde3330-8614-47ac-9c0f-93c34a0cfc17","Type":"ContainerStarted","Data":"5254cbcba5f3ecf5f3b84789e26e91c91335bda7bc56a2f59979d4abe7cbd09b"} Jul 10 17:00:26.257199 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:26.256815 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-67764b8b78-tmqk6" Jul 10 17:00:26.281066 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:26.280998 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-67764b8b78-tmqk6" podStartSLOduration=1.740058977 podStartE2EDuration="6.280975686s" totalImagesPullingTime="3.52928935s" totalInitContainerRuntime="1.011627359s" isStatefulPod=false podCreationTimestamp="2026-07-10 17:00:20 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:00:26.279276817 +0000 UTC m=+370.368605868" watchObservedRunningTime="2026-07-10 17:00:26.280975686 +0000 UTC m=+370.370304738" Jul 10 17:00:37.263598 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:37.263567 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-67764b8b78-tmqk6" Jul 10 17:00:40.637282 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:40.637246 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-558b5454d6-rb5qx"] Jul 10 17:00:40.640098 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:40.640074 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-558b5454d6-rb5qx" Jul 10 17:00:40.643244 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:40.643211 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 10 17:00:40.643411 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:40.643391 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-mck87\"" Jul 10 17:00:40.644406 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:40.644385 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 10 17:00:40.658747 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:40.658719 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-558b5454d6-rb5qx"] Jul 10 17:00:40.749106 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:40.749042 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2a999b9a-8edf-4057-afbd-8761b62ae720-tmp\") pod \"kube-auth-proxy-558b5454d6-rb5qx\" (UID: \"2a999b9a-8edf-4057-afbd-8761b62ae720\") " pod="openshift-ingress/kube-auth-proxy-558b5454d6-rb5qx" Jul 10 17:00:40.749291 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:40.749115 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wp57\" (UniqueName: \"kubernetes.io/projected/2a999b9a-8edf-4057-afbd-8761b62ae720-kube-api-access-9wp57\") pod \"kube-auth-proxy-558b5454d6-rb5qx\" (UID: \"2a999b9a-8edf-4057-afbd-8761b62ae720\") " pod="openshift-ingress/kube-auth-proxy-558b5454d6-rb5qx" Jul 10 17:00:40.749291 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:40.749135 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2a999b9a-8edf-4057-afbd-8761b62ae720-tls-certs\") pod \"kube-auth-proxy-558b5454d6-rb5qx\" (UID: \"2a999b9a-8edf-4057-afbd-8761b62ae720\") " pod="openshift-ingress/kube-auth-proxy-558b5454d6-rb5qx" Jul 10 17:00:40.850000 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:40.849963 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2a999b9a-8edf-4057-afbd-8761b62ae720-tmp\") pod \"kube-auth-proxy-558b5454d6-rb5qx\" (UID: \"2a999b9a-8edf-4057-afbd-8761b62ae720\") " pod="openshift-ingress/kube-auth-proxy-558b5454d6-rb5qx" Jul 10 17:00:40.850000 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:40.850002 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wp57\" (UniqueName: \"kubernetes.io/projected/2a999b9a-8edf-4057-afbd-8761b62ae720-kube-api-access-9wp57\") pod \"kube-auth-proxy-558b5454d6-rb5qx\" (UID: \"2a999b9a-8edf-4057-afbd-8761b62ae720\") " pod="openshift-ingress/kube-auth-proxy-558b5454d6-rb5qx" Jul 10 17:00:40.850234 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:40.850022 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2a999b9a-8edf-4057-afbd-8761b62ae720-tls-certs\") pod \"kube-auth-proxy-558b5454d6-rb5qx\" (UID: \"2a999b9a-8edf-4057-afbd-8761b62ae720\") " pod="openshift-ingress/kube-auth-proxy-558b5454d6-rb5qx" Jul 10 17:00:40.852562 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:40.852528 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2a999b9a-8edf-4057-afbd-8761b62ae720-tmp\") pod \"kube-auth-proxy-558b5454d6-rb5qx\" (UID: \"2a999b9a-8edf-4057-afbd-8761b62ae720\") " pod="openshift-ingress/kube-auth-proxy-558b5454d6-rb5qx" Jul 10 17:00:40.852684 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:40.852640 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2a999b9a-8edf-4057-afbd-8761b62ae720-tls-certs\") pod \"kube-auth-proxy-558b5454d6-rb5qx\" (UID: \"2a999b9a-8edf-4057-afbd-8761b62ae720\") " pod="openshift-ingress/kube-auth-proxy-558b5454d6-rb5qx" Jul 10 17:00:40.873344 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:40.873301 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wp57\" (UniqueName: \"kubernetes.io/projected/2a999b9a-8edf-4057-afbd-8761b62ae720-kube-api-access-9wp57\") pod \"kube-auth-proxy-558b5454d6-rb5qx\" (UID: \"2a999b9a-8edf-4057-afbd-8761b62ae720\") " pod="openshift-ingress/kube-auth-proxy-558b5454d6-rb5qx" Jul 10 17:00:40.951249 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:40.951162 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-558b5454d6-rb5qx" Jul 10 17:00:41.109991 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:41.109951 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-558b5454d6-rb5qx"] Jul 10 17:00:41.112038 ip-10-0-136-7 kubenswrapper[2578]: W0710 17:00:41.111996 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a999b9a_8edf_4057_afbd_8761b62ae720.slice/crio-901d144092a5b29c98cca2b55c047e3deeb411041b033479c58048fdc8d5cc59 WatchSource:0}: Error finding container 901d144092a5b29c98cca2b55c047e3deeb411041b033479c58048fdc8d5cc59: Status 404 returned error can't find the container with id 901d144092a5b29c98cca2b55c047e3deeb411041b033479c58048fdc8d5cc59 Jul 10 17:00:41.321677 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:41.321642 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-558b5454d6-rb5qx" event={"ID":"2a999b9a-8edf-4057-afbd-8761b62ae720","Type":"ContainerStarted","Data":"901d144092a5b29c98cca2b55c047e3deeb411041b033479c58048fdc8d5cc59"} Jul 10 17:00:45.342608 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:45.342568 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-558b5454d6-rb5qx" event={"ID":"2a999b9a-8edf-4057-afbd-8761b62ae720","Type":"ContainerStarted","Data":"9ed1a88afd2b60eccbd25baddc385c8dd0109540d7e6b36a5fd7c1ae183090d3"} Jul 10 17:00:45.359957 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:45.359902 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-558b5454d6-rb5qx" podStartSLOduration=1.956727667 podStartE2EDuration="5.359881607s" totalImagesPullingTime="3.40315394s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 17:00:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:00:45.358207635 +0000 UTC m=+389.447536685" watchObservedRunningTime="2026-07-10 17:00:45.359881607 +0000 UTC m=+389.449210656" Jul 10 17:00:48.427861 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:48.427828 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2tm7x"] Jul 10 17:00:48.430066 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:48.430033 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2tm7x" Jul 10 17:00:48.432960 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:48.432918 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 10 17:00:48.433129 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:48.432918 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 10 17:00:48.433129 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:48.432932 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-5fql7\"" Jul 10 17:00:48.433129 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:48.433084 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 10 17:00:48.433450 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:48.433436 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 10 17:00:48.445891 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:48.445861 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2tm7x"] Jul 10 17:00:48.531691 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:48.531649 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/973dcb52-e499-429c-a73d-471e3cd87ffc-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-2tm7x\" (UID: \"973dcb52-e499-429c-a73d-471e3cd87ffc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2tm7x" Jul 10 17:00:48.531691 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:48.531691 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/973dcb52-e499-429c-a73d-471e3cd87ffc-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-2tm7x\" (UID: \"973dcb52-e499-429c-a73d-471e3cd87ffc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2tm7x" Jul 10 17:00:48.531910 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:48.531764 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/973dcb52-e499-429c-a73d-471e3cd87ffc-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-2tm7x\" (UID: \"973dcb52-e499-429c-a73d-471e3cd87ffc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2tm7x" Jul 10 17:00:48.531910 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:48.531807 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/973dcb52-e499-429c-a73d-471e3cd87ffc-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-2tm7x\" (UID: \"973dcb52-e499-429c-a73d-471e3cd87ffc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2tm7x" Jul 10 17:00:48.531910 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:48.531831 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/973dcb52-e499-429c-a73d-471e3cd87ffc-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-2tm7x\" (UID: \"973dcb52-e499-429c-a73d-471e3cd87ffc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2tm7x" Jul 10 17:00:48.531910 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:48.531850 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlzb2\" (UniqueName: \"kubernetes.io/projected/973dcb52-e499-429c-a73d-471e3cd87ffc-kube-api-access-wlzb2\") pod \"istiod-openshift-gateway-94bb8fbfd-2tm7x\" (UID: \"973dcb52-e499-429c-a73d-471e3cd87ffc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2tm7x" Jul 10 17:00:48.531910 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:48.531880 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/973dcb52-e499-429c-a73d-471e3cd87ffc-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-2tm7x\" (UID: \"973dcb52-e499-429c-a73d-471e3cd87ffc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2tm7x" Jul 10 17:00:48.632949 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:48.632908 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/973dcb52-e499-429c-a73d-471e3cd87ffc-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-2tm7x\" (UID: \"973dcb52-e499-429c-a73d-471e3cd87ffc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2tm7x" Jul 10 17:00:48.632949 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:48.632957 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/973dcb52-e499-429c-a73d-471e3cd87ffc-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-2tm7x\" (UID: \"973dcb52-e499-429c-a73d-471e3cd87ffc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2tm7x" Jul 10 17:00:48.633255 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:48.632998 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/973dcb52-e499-429c-a73d-471e3cd87ffc-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-2tm7x\" (UID: \"973dcb52-e499-429c-a73d-471e3cd87ffc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2tm7x" Jul 10 17:00:48.633255 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:48.633021 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/973dcb52-e499-429c-a73d-471e3cd87ffc-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-2tm7x\" (UID: \"973dcb52-e499-429c-a73d-471e3cd87ffc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2tm7x" Jul 10 17:00:48.633255 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:48.633035 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/973dcb52-e499-429c-a73d-471e3cd87ffc-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-2tm7x\" (UID: \"973dcb52-e499-429c-a73d-471e3cd87ffc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2tm7x" Jul 10 17:00:48.633255 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:48.633204 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlzb2\" (UniqueName: \"kubernetes.io/projected/973dcb52-e499-429c-a73d-471e3cd87ffc-kube-api-access-wlzb2\") pod \"istiod-openshift-gateway-94bb8fbfd-2tm7x\" (UID: \"973dcb52-e499-429c-a73d-471e3cd87ffc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2tm7x" Jul 10 17:00:48.633458 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:48.633309 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/973dcb52-e499-429c-a73d-471e3cd87ffc-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-2tm7x\" (UID: \"973dcb52-e499-429c-a73d-471e3cd87ffc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2tm7x" Jul 10 17:00:48.633803 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:48.633720 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/973dcb52-e499-429c-a73d-471e3cd87ffc-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-2tm7x\" (UID: \"973dcb52-e499-429c-a73d-471e3cd87ffc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2tm7x" Jul 10 17:00:48.635637 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:48.635605 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/973dcb52-e499-429c-a73d-471e3cd87ffc-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-2tm7x\" (UID: \"973dcb52-e499-429c-a73d-471e3cd87ffc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2tm7x" Jul 10 17:00:48.635754 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:48.635699 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/973dcb52-e499-429c-a73d-471e3cd87ffc-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-2tm7x\" (UID: \"973dcb52-e499-429c-a73d-471e3cd87ffc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2tm7x" Jul 10 17:00:48.636034 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:48.636013 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/973dcb52-e499-429c-a73d-471e3cd87ffc-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-2tm7x\" (UID: \"973dcb52-e499-429c-a73d-471e3cd87ffc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2tm7x" Jul 10 17:00:48.636115 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:48.636041 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/973dcb52-e499-429c-a73d-471e3cd87ffc-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-2tm7x\" (UID: \"973dcb52-e499-429c-a73d-471e3cd87ffc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2tm7x" Jul 10 17:00:48.659107 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:48.659076 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/973dcb52-e499-429c-a73d-471e3cd87ffc-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-2tm7x\" (UID: \"973dcb52-e499-429c-a73d-471e3cd87ffc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2tm7x" Jul 10 17:00:48.659265 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:48.659246 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlzb2\" (UniqueName: \"kubernetes.io/projected/973dcb52-e499-429c-a73d-471e3cd87ffc-kube-api-access-wlzb2\") pod \"istiod-openshift-gateway-94bb8fbfd-2tm7x\" (UID: \"973dcb52-e499-429c-a73d-471e3cd87ffc\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2tm7x" Jul 10 17:00:48.739475 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:48.739370 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2tm7x" Jul 10 17:00:48.904160 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:48.904061 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2tm7x"] Jul 10 17:00:48.906807 ip-10-0-136-7 kubenswrapper[2578]: W0710 17:00:48.906775 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod973dcb52_e499_429c_a73d_471e3cd87ffc.slice/crio-b4b5b8921aa1c95beaee45b935454c5ac68801301d6af36ea898f851f571cc6b WatchSource:0}: Error finding container b4b5b8921aa1c95beaee45b935454c5ac68801301d6af36ea898f851f571cc6b: Status 404 returned error can't find the container with id b4b5b8921aa1c95beaee45b935454c5ac68801301d6af36ea898f851f571cc6b Jul 10 17:00:49.363200 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:49.363166 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2tm7x" event={"ID":"973dcb52-e499-429c-a73d-471e3cd87ffc","Type":"ContainerStarted","Data":"b4b5b8921aa1c95beaee45b935454c5ac68801301d6af36ea898f851f571cc6b"} Jul 10 17:00:50.005256 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:50.005219 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7bcc59c859-67cq9"] Jul 10 17:00:50.007797 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:50.007771 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7bcc59c859-67cq9" Jul 10 17:00:50.012086 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:50.012033 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 10 17:00:50.013186 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:50.013157 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 10 17:00:50.013325 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:50.013211 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-2jxfb\"" Jul 10 17:00:50.013325 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:50.013249 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 10 17:00:50.013325 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:50.013160 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 10 17:00:50.013483 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:50.013160 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 10 17:00:50.028604 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:50.028570 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7bcc59c859-67cq9"] Jul 10 17:00:50.048698 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:50.048658 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5f682381-8753-4143-b130-4496b861e3c7-manager-config\") pod \"lws-controller-manager-7bcc59c859-67cq9\" (UID: \"5f682381-8753-4143-b130-4496b861e3c7\") " pod="openshift-lws-operator/lws-controller-manager-7bcc59c859-67cq9" Jul 10 17:00:50.048893 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:50.048710 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjvpw\" (UniqueName: \"kubernetes.io/projected/5f682381-8753-4143-b130-4496b861e3c7-kube-api-access-sjvpw\") pod \"lws-controller-manager-7bcc59c859-67cq9\" (UID: \"5f682381-8753-4143-b130-4496b861e3c7\") " pod="openshift-lws-operator/lws-controller-manager-7bcc59c859-67cq9" Jul 10 17:00:50.048893 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:50.048757 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5f682381-8753-4143-b130-4496b861e3c7-metrics-cert\") pod \"lws-controller-manager-7bcc59c859-67cq9\" (UID: \"5f682381-8753-4143-b130-4496b861e3c7\") " pod="openshift-lws-operator/lws-controller-manager-7bcc59c859-67cq9" Jul 10 17:00:50.048893 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:50.048829 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5f682381-8753-4143-b130-4496b861e3c7-cert\") pod \"lws-controller-manager-7bcc59c859-67cq9\" (UID: \"5f682381-8753-4143-b130-4496b861e3c7\") " pod="openshift-lws-operator/lws-controller-manager-7bcc59c859-67cq9" Jul 10 17:00:50.151074 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:50.150492 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5f682381-8753-4143-b130-4496b861e3c7-manager-config\") pod \"lws-controller-manager-7bcc59c859-67cq9\" (UID: \"5f682381-8753-4143-b130-4496b861e3c7\") " pod="openshift-lws-operator/lws-controller-manager-7bcc59c859-67cq9" Jul 10 17:00:50.151074 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:50.150560 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjvpw\" (UniqueName: \"kubernetes.io/projected/5f682381-8753-4143-b130-4496b861e3c7-kube-api-access-sjvpw\") pod \"lws-controller-manager-7bcc59c859-67cq9\" (UID: \"5f682381-8753-4143-b130-4496b861e3c7\") " pod="openshift-lws-operator/lws-controller-manager-7bcc59c859-67cq9" Jul 10 17:00:50.151074 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:50.150601 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5f682381-8753-4143-b130-4496b861e3c7-metrics-cert\") pod \"lws-controller-manager-7bcc59c859-67cq9\" (UID: \"5f682381-8753-4143-b130-4496b861e3c7\") " pod="openshift-lws-operator/lws-controller-manager-7bcc59c859-67cq9" Jul 10 17:00:50.151074 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:50.150636 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5f682381-8753-4143-b130-4496b861e3c7-cert\") pod \"lws-controller-manager-7bcc59c859-67cq9\" (UID: \"5f682381-8753-4143-b130-4496b861e3c7\") " pod="openshift-lws-operator/lws-controller-manager-7bcc59c859-67cq9" Jul 10 17:00:50.152304 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:50.152269 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5f682381-8753-4143-b130-4496b861e3c7-manager-config\") pod \"lws-controller-manager-7bcc59c859-67cq9\" (UID: \"5f682381-8753-4143-b130-4496b861e3c7\") " pod="openshift-lws-operator/lws-controller-manager-7bcc59c859-67cq9" Jul 10 17:00:50.160987 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:50.160918 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5f682381-8753-4143-b130-4496b861e3c7-cert\") pod \"lws-controller-manager-7bcc59c859-67cq9\" (UID: \"5f682381-8753-4143-b130-4496b861e3c7\") " pod="openshift-lws-operator/lws-controller-manager-7bcc59c859-67cq9" Jul 10 17:00:50.164386 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:50.164326 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5f682381-8753-4143-b130-4496b861e3c7-metrics-cert\") pod \"lws-controller-manager-7bcc59c859-67cq9\" (UID: \"5f682381-8753-4143-b130-4496b861e3c7\") " pod="openshift-lws-operator/lws-controller-manager-7bcc59c859-67cq9" Jul 10 17:00:50.170832 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:50.170765 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjvpw\" (UniqueName: \"kubernetes.io/projected/5f682381-8753-4143-b130-4496b861e3c7-kube-api-access-sjvpw\") pod \"lws-controller-manager-7bcc59c859-67cq9\" (UID: \"5f682381-8753-4143-b130-4496b861e3c7\") " pod="openshift-lws-operator/lws-controller-manager-7bcc59c859-67cq9" Jul 10 17:00:50.320875 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:50.320831 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7bcc59c859-67cq9" Jul 10 17:00:50.482749 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:50.482718 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7bcc59c859-67cq9"] Jul 10 17:00:51.305164 ip-10-0-136-7 kubenswrapper[2578]: W0710 17:00:51.305129 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f682381_8753_4143_b130_4496b861e3c7.slice/crio-f3a16c0fef423b52ecd878f43194e3e77211b428b9f99261caf6970f10022e3e WatchSource:0}: Error finding container f3a16c0fef423b52ecd878f43194e3e77211b428b9f99261caf6970f10022e3e: Status 404 returned error can't find the container with id f3a16c0fef423b52ecd878f43194e3e77211b428b9f99261caf6970f10022e3e Jul 10 17:00:51.340635 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:51.340590 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 10 17:00:51.340752 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:51.340669 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 10 17:00:51.375364 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:51.375318 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7bcc59c859-67cq9" event={"ID":"5f682381-8753-4143-b130-4496b861e3c7","Type":"ContainerStarted","Data":"f3a16c0fef423b52ecd878f43194e3e77211b428b9f99261caf6970f10022e3e"} Jul 10 17:00:52.386817 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:52.386718 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2tm7x" event={"ID":"973dcb52-e499-429c-a73d-471e3cd87ffc","Type":"ContainerStarted","Data":"bf74d08fafa294645889706bbfbdb0c5c9ee5519ee2780c983993ceb8ca788be"} Jul 10 17:00:52.387537 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:52.387506 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2tm7x" Jul 10 17:00:52.389421 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:52.389387 2578 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-2tm7x container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 10 17:00:52.389648 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:52.389446 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2tm7x" podUID="973dcb52-e499-429c-a73d-471e3cd87ffc" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 10 17:00:52.426327 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:52.426261 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2tm7x" podStartSLOduration=1.995449449 podStartE2EDuration="4.426233355s" totalImagesPullingTime="2.430783906s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 17:00:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:00:52.419514811 +0000 UTC m=+396.508843861" watchObservedRunningTime="2026-07-10 17:00:52.426233355 +0000 UTC m=+396.515562407" Jul 10 17:00:53.392675 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:53.392644 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2tm7x" Jul 10 17:00:54.397624 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:54.397529 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7bcc59c859-67cq9" event={"ID":"5f682381-8753-4143-b130-4496b861e3c7","Type":"ContainerStarted","Data":"25a32635ee669ef31c9440d412a946eabfe270b8002ba1f0cb9806f8508162fd"} Jul 10 17:00:54.397980 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:54.397791 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7bcc59c859-67cq9" Jul 10 17:00:54.445110 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:00:54.445026 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7bcc59c859-67cq9" podStartSLOduration=2.665376152 podStartE2EDuration="5.44500673s" totalImagesPullingTime="2.779630578s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 17:00:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:00:54.443768346 +0000 UTC m=+398.533097397" watchObservedRunningTime="2026-07-10 17:00:54.44500673 +0000 UTC m=+398.534335780" Jul 10 17:01:05.405143 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:01:05.405108 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7bcc59c859-67cq9" Jul 10 17:04:16.332172 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:04:16.332140 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-5j4ws_27742f15-b602-4554-84dc-ff9d46a9adda/console-operator/2.log" Jul 10 17:04:16.332783 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:04:16.332766 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-5j4ws_27742f15-b602-4554-84dc-ff9d46a9adda/console-operator/2.log" Jul 10 17:04:16.338698 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:04:16.338678 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gqgkg_b9a53126-f278-4a9d-9735-5f73dd0c1964/ovn-acl-logging/0.log" Jul 10 17:04:16.339426 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:04:16.339408 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gqgkg_b9a53126-f278-4a9d-9735-5f73dd0c1964/ovn-acl-logging/0.log" Jul 10 17:09:16.360183 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:09:16.360099 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-5j4ws_27742f15-b602-4554-84dc-ff9d46a9adda/console-operator/2.log" Jul 10 17:09:16.361749 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:09:16.361724 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-5j4ws_27742f15-b602-4554-84dc-ff9d46a9adda/console-operator/2.log" Jul 10 17:09:16.366723 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:09:16.366699 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gqgkg_b9a53126-f278-4a9d-9735-5f73dd0c1964/ovn-acl-logging/0.log" Jul 10 17:09:16.368790 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:09:16.368770 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gqgkg_b9a53126-f278-4a9d-9735-5f73dd0c1964/ovn-acl-logging/0.log" Jul 10 17:14:16.387664 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:14:16.387631 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-5j4ws_27742f15-b602-4554-84dc-ff9d46a9adda/console-operator/2.log" Jul 10 17:14:16.390597 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:14:16.390572 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-5j4ws_27742f15-b602-4554-84dc-ff9d46a9adda/console-operator/2.log" Jul 10 17:14:16.393973 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:14:16.393951 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gqgkg_b9a53126-f278-4a9d-9735-5f73dd0c1964/ovn-acl-logging/0.log" Jul 10 17:14:16.397269 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:14:16.397246 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gqgkg_b9a53126-f278-4a9d-9735-5f73dd0c1964/ovn-acl-logging/0.log" Jul 10 17:19:16.414482 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:19:16.414448 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-5j4ws_27742f15-b602-4554-84dc-ff9d46a9adda/console-operator/2.log" Jul 10 17:19:16.418720 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:19:16.418692 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-5j4ws_27742f15-b602-4554-84dc-ff9d46a9adda/console-operator/2.log" Jul 10 17:19:16.421561 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:19:16.421534 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gqgkg_b9a53126-f278-4a9d-9735-5f73dd0c1964/ovn-acl-logging/0.log" Jul 10 17:19:16.425923 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:19:16.425902 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gqgkg_b9a53126-f278-4a9d-9735-5f73dd0c1964/ovn-acl-logging/0.log" Jul 10 17:24:16.451005 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:24:16.450879 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-5j4ws_27742f15-b602-4554-84dc-ff9d46a9adda/console-operator/2.log" Jul 10 17:24:16.456113 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:24:16.454008 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-5j4ws_27742f15-b602-4554-84dc-ff9d46a9adda/console-operator/2.log" Jul 10 17:24:16.457636 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:24:16.457611 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gqgkg_b9a53126-f278-4a9d-9735-5f73dd0c1964/ovn-acl-logging/0.log" Jul 10 17:24:16.460282 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:24:16.460266 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gqgkg_b9a53126-f278-4a9d-9735-5f73dd0c1964/ovn-acl-logging/0.log" Jul 10 17:27:30.632173 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:27:30.632128 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-fc13d1/test-model-fc13d1-kserve-6d669f6669-86d8g"] Jul 10 17:27:30.635524 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:27:30.635504 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-fc13d1/test-model-fc13d1-kserve-6d669f6669-86d8g" Jul 10 17:27:30.639163 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:27:30.639141 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-fc13d1\"/\"default-dockercfg-r4sxx\"" Jul 10 17:27:30.639299 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:27:30.639141 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-fc13d1\"/\"test-model-fc13d1-kserve-self-signed-certs\"" Jul 10 17:27:30.639299 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:27:30.639142 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-fc13d1\"/\"kube-root-ca.crt\"" Jul 10 17:27:30.639299 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:27:30.639142 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-fc13d1\"/\"openshift-service-ca.crt\"" Jul 10 17:27:30.645208 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:27:30.645188 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-fc13d1/test-model-fc13d1-kserve-6d669f6669-86d8g"] Jul 10 17:27:30.811504 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:27:30.811470 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9c7db97-85ec-4b86-a56a-35a83107a934-tls-certs\") pod \"test-model-fc13d1-kserve-6d669f6669-86d8g\" (UID: \"d9c7db97-85ec-4b86-a56a-35a83107a934\") " pod="ai-tenant-e2e-inf-b-fc13d1/test-model-fc13d1-kserve-6d669f6669-86d8g" Jul 10 17:27:30.811711 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:27:30.811510 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9c7db97-85ec-4b86-a56a-35a83107a934-home\") pod \"test-model-fc13d1-kserve-6d669f6669-86d8g\" (UID: \"d9c7db97-85ec-4b86-a56a-35a83107a934\") " pod="ai-tenant-e2e-inf-b-fc13d1/test-model-fc13d1-kserve-6d669f6669-86d8g" Jul 10 17:27:30.811711 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:27:30.811605 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9c7db97-85ec-4b86-a56a-35a83107a934-dshm\") pod \"test-model-fc13d1-kserve-6d669f6669-86d8g\" (UID: \"d9c7db97-85ec-4b86-a56a-35a83107a934\") " pod="ai-tenant-e2e-inf-b-fc13d1/test-model-fc13d1-kserve-6d669f6669-86d8g" Jul 10 17:27:30.811711 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:27:30.811648 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9c7db97-85ec-4b86-a56a-35a83107a934-model-cache\") pod \"test-model-fc13d1-kserve-6d669f6669-86d8g\" (UID: \"d9c7db97-85ec-4b86-a56a-35a83107a934\") " pod="ai-tenant-e2e-inf-b-fc13d1/test-model-fc13d1-kserve-6d669f6669-86d8g" Jul 10 17:27:30.811711 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:27:30.811682 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9c7db97-85ec-4b86-a56a-35a83107a934-tmp-dir\") pod \"test-model-fc13d1-kserve-6d669f6669-86d8g\" (UID: \"d9c7db97-85ec-4b86-a56a-35a83107a934\") " pod="ai-tenant-e2e-inf-b-fc13d1/test-model-fc13d1-kserve-6d669f6669-86d8g" Jul 10 17:27:30.811711 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:27:30.811702 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kj4x5\" (UniqueName: \"kubernetes.io/projected/d9c7db97-85ec-4b86-a56a-35a83107a934-kube-api-access-kj4x5\") pod \"test-model-fc13d1-kserve-6d669f6669-86d8g\" (UID: \"d9c7db97-85ec-4b86-a56a-35a83107a934\") " pod="ai-tenant-e2e-inf-b-fc13d1/test-model-fc13d1-kserve-6d669f6669-86d8g" Jul 10 17:27:30.912745 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:27:30.912659 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9c7db97-85ec-4b86-a56a-35a83107a934-tls-certs\") pod \"test-model-fc13d1-kserve-6d669f6669-86d8g\" (UID: \"d9c7db97-85ec-4b86-a56a-35a83107a934\") " pod="ai-tenant-e2e-inf-b-fc13d1/test-model-fc13d1-kserve-6d669f6669-86d8g" Jul 10 17:27:30.912745 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:27:30.912693 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9c7db97-85ec-4b86-a56a-35a83107a934-home\") pod \"test-model-fc13d1-kserve-6d669f6669-86d8g\" (UID: \"d9c7db97-85ec-4b86-a56a-35a83107a934\") " pod="ai-tenant-e2e-inf-b-fc13d1/test-model-fc13d1-kserve-6d669f6669-86d8g" Jul 10 17:27:30.912745 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:27:30.912723 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9c7db97-85ec-4b86-a56a-35a83107a934-dshm\") pod \"test-model-fc13d1-kserve-6d669f6669-86d8g\" (UID: \"d9c7db97-85ec-4b86-a56a-35a83107a934\") " pod="ai-tenant-e2e-inf-b-fc13d1/test-model-fc13d1-kserve-6d669f6669-86d8g" Jul 10 17:27:30.913013 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:27:30.912749 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9c7db97-85ec-4b86-a56a-35a83107a934-model-cache\") pod \"test-model-fc13d1-kserve-6d669f6669-86d8g\" (UID: \"d9c7db97-85ec-4b86-a56a-35a83107a934\") " pod="ai-tenant-e2e-inf-b-fc13d1/test-model-fc13d1-kserve-6d669f6669-86d8g" Jul 10 17:27:30.913013 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:27:30.912766 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9c7db97-85ec-4b86-a56a-35a83107a934-tmp-dir\") pod \"test-model-fc13d1-kserve-6d669f6669-86d8g\" (UID: \"d9c7db97-85ec-4b86-a56a-35a83107a934\") " pod="ai-tenant-e2e-inf-b-fc13d1/test-model-fc13d1-kserve-6d669f6669-86d8g" Jul 10 17:27:30.913013 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:27:30.912783 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kj4x5\" (UniqueName: \"kubernetes.io/projected/d9c7db97-85ec-4b86-a56a-35a83107a934-kube-api-access-kj4x5\") pod \"test-model-fc13d1-kserve-6d669f6669-86d8g\" (UID: \"d9c7db97-85ec-4b86-a56a-35a83107a934\") " pod="ai-tenant-e2e-inf-b-fc13d1/test-model-fc13d1-kserve-6d669f6669-86d8g" Jul 10 17:27:30.913260 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:27:30.913155 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9c7db97-85ec-4b86-a56a-35a83107a934-home\") pod \"test-model-fc13d1-kserve-6d669f6669-86d8g\" (UID: \"d9c7db97-85ec-4b86-a56a-35a83107a934\") " pod="ai-tenant-e2e-inf-b-fc13d1/test-model-fc13d1-kserve-6d669f6669-86d8g" Jul 10 17:27:30.913260 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:27:30.913179 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9c7db97-85ec-4b86-a56a-35a83107a934-tmp-dir\") pod \"test-model-fc13d1-kserve-6d669f6669-86d8g\" (UID: \"d9c7db97-85ec-4b86-a56a-35a83107a934\") " pod="ai-tenant-e2e-inf-b-fc13d1/test-model-fc13d1-kserve-6d669f6669-86d8g" Jul 10 17:27:30.913260 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:27:30.913221 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9c7db97-85ec-4b86-a56a-35a83107a934-model-cache\") pod \"test-model-fc13d1-kserve-6d669f6669-86d8g\" (UID: \"d9c7db97-85ec-4b86-a56a-35a83107a934\") " pod="ai-tenant-e2e-inf-b-fc13d1/test-model-fc13d1-kserve-6d669f6669-86d8g" Jul 10 17:27:30.915179 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:27:30.915159 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9c7db97-85ec-4b86-a56a-35a83107a934-dshm\") pod \"test-model-fc13d1-kserve-6d669f6669-86d8g\" (UID: \"d9c7db97-85ec-4b86-a56a-35a83107a934\") " pod="ai-tenant-e2e-inf-b-fc13d1/test-model-fc13d1-kserve-6d669f6669-86d8g" Jul 10 17:27:30.915443 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:27:30.915423 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9c7db97-85ec-4b86-a56a-35a83107a934-tls-certs\") pod \"test-model-fc13d1-kserve-6d669f6669-86d8g\" (UID: \"d9c7db97-85ec-4b86-a56a-35a83107a934\") " pod="ai-tenant-e2e-inf-b-fc13d1/test-model-fc13d1-kserve-6d669f6669-86d8g" Jul 10 17:27:30.920543 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:27:30.920515 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kj4x5\" (UniqueName: \"kubernetes.io/projected/d9c7db97-85ec-4b86-a56a-35a83107a934-kube-api-access-kj4x5\") pod \"test-model-fc13d1-kserve-6d669f6669-86d8g\" (UID: \"d9c7db97-85ec-4b86-a56a-35a83107a934\") " pod="ai-tenant-e2e-inf-b-fc13d1/test-model-fc13d1-kserve-6d669f6669-86d8g" Jul 10 17:27:30.960220 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:27:30.960188 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-fc13d1/test-model-fc13d1-kserve-6d669f6669-86d8g" Jul 10 17:27:31.094567 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:27:31.094534 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-fc13d1/test-model-fc13d1-kserve-6d669f6669-86d8g"] Jul 10 17:27:31.096841 ip-10-0-136-7 kubenswrapper[2578]: W0710 17:27:31.096816 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd9c7db97_85ec_4b86_a56a_35a83107a934.slice/crio-847909554c77644d76a165165e59c7d2007234d6f3576814ea9e5758ee402267 WatchSource:0}: Error finding container 847909554c77644d76a165165e59c7d2007234d6f3576814ea9e5758ee402267: Status 404 returned error can't find the container with id 847909554c77644d76a165165e59c7d2007234d6f3576814ea9e5758ee402267 Jul 10 17:27:31.098696 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:27:31.098677 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 17:27:31.456494 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:27:31.456454 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-fc13d1/test-model-fc13d1-kserve-6d669f6669-86d8g" event={"ID":"d9c7db97-85ec-4b86-a56a-35a83107a934","Type":"ContainerStarted","Data":"847909554c77644d76a165165e59c7d2007234d6f3576814ea9e5758ee402267"} Jul 10 17:27:32.461744 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:27:32.461657 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-fc13d1/test-model-fc13d1-kserve-6d669f6669-86d8g" event={"ID":"d9c7db97-85ec-4b86-a56a-35a83107a934","Type":"ContainerStarted","Data":"6c22244a2a64fa0edb247fe9873928093faec81db94ee5ffb699afcbca0b6b71"} Jul 10 17:27:32.478441 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:27:32.478394 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-fc13d1/test-model-fc13d1-kserve-6d669f6669-86d8g" podStartSLOduration=1.426789727 podStartE2EDuration="2.478376285s" totalImagesPullingTime="1.051586558s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 17:27:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:27:32.477544208 +0000 UTC m=+1996.566873258" watchObservedRunningTime="2026-07-10 17:27:32.478376285 +0000 UTC m=+1996.567705333" Jul 10 17:27:40.961263 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:27:40.961222 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-fc13d1/test-model-fc13d1-kserve-6d669f6669-86d8g" Jul 10 17:27:40.961652 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:27:40.961279 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-fc13d1/test-model-fc13d1-kserve-6d669f6669-86d8g" Jul 10 17:27:40.974726 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:27:40.974693 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-fc13d1/test-model-fc13d1-kserve-6d669f6669-86d8g" Jul 10 17:27:41.513754 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:27:41.513727 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-fc13d1/test-model-fc13d1-kserve-6d669f6669-86d8g" Jul 10 17:27:49.380203 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:27:49.380161 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-fc13d1/test-model-fc13d1-kserve-6d669f6669-86d8g"] Jul 10 17:28:04.401488 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:28:04.401440 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-fc13d1/test-model-fc13d1-kserve-6d669f6669-86d8g" podUID="d9c7db97-85ec-4b86-a56a-35a83107a934" containerName="main" containerID="cri-o://6c22244a2a64fa0edb247fe9873928093faec81db94ee5ffb699afcbca0b6b71" gracePeriod=45 Jul 10 17:28:04.606341 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:28:04.606308 2578 generic.go:356] "Generic (PLEG): container finished" podID="d9c7db97-85ec-4b86-a56a-35a83107a934" containerID="6c22244a2a64fa0edb247fe9873928093faec81db94ee5ffb699afcbca0b6b71" exitCode=0 Jul 10 17:28:04.606341 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:28:04.606343 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-fc13d1/test-model-fc13d1-kserve-6d669f6669-86d8g" event={"ID":"d9c7db97-85ec-4b86-a56a-35a83107a934","Type":"ContainerDied","Data":"6c22244a2a64fa0edb247fe9873928093faec81db94ee5ffb699afcbca0b6b71"} Jul 10 17:28:04.661989 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:28:04.661915 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-fc13d1/test-model-fc13d1-kserve-6d669f6669-86d8g" Jul 10 17:28:04.819568 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:28:04.819517 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kj4x5\" (UniqueName: \"kubernetes.io/projected/d9c7db97-85ec-4b86-a56a-35a83107a934-kube-api-access-kj4x5\") pod \"d9c7db97-85ec-4b86-a56a-35a83107a934\" (UID: \"d9c7db97-85ec-4b86-a56a-35a83107a934\") " Jul 10 17:28:04.819744 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:28:04.819593 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9c7db97-85ec-4b86-a56a-35a83107a934-dshm\") pod \"d9c7db97-85ec-4b86-a56a-35a83107a934\" (UID: \"d9c7db97-85ec-4b86-a56a-35a83107a934\") " Jul 10 17:28:04.819744 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:28:04.819621 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9c7db97-85ec-4b86-a56a-35a83107a934-model-cache\") pod \"d9c7db97-85ec-4b86-a56a-35a83107a934\" (UID: \"d9c7db97-85ec-4b86-a56a-35a83107a934\") " Jul 10 17:28:04.819744 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:28:04.819644 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9c7db97-85ec-4b86-a56a-35a83107a934-tmp-dir\") pod \"d9c7db97-85ec-4b86-a56a-35a83107a934\" (UID: \"d9c7db97-85ec-4b86-a56a-35a83107a934\") " Jul 10 17:28:04.819744 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:28:04.819670 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9c7db97-85ec-4b86-a56a-35a83107a934-tls-certs\") pod \"d9c7db97-85ec-4b86-a56a-35a83107a934\" (UID: \"d9c7db97-85ec-4b86-a56a-35a83107a934\") " Jul 10 17:28:04.819744 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:28:04.819692 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9c7db97-85ec-4b86-a56a-35a83107a934-home\") pod \"d9c7db97-85ec-4b86-a56a-35a83107a934\" (UID: \"d9c7db97-85ec-4b86-a56a-35a83107a934\") " Jul 10 17:28:04.819984 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:28:04.819948 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9c7db97-85ec-4b86-a56a-35a83107a934-model-cache" (OuterVolumeSpecName: "model-cache") pod "d9c7db97-85ec-4b86-a56a-35a83107a934" (UID: "d9c7db97-85ec-4b86-a56a-35a83107a934"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 17:28:04.820043 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:28:04.819982 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9c7db97-85ec-4b86-a56a-35a83107a934-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d9c7db97-85ec-4b86-a56a-35a83107a934" (UID: "d9c7db97-85ec-4b86-a56a-35a83107a934"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 17:28:04.820130 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:28:04.820031 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9c7db97-85ec-4b86-a56a-35a83107a934-home" (OuterVolumeSpecName: "home") pod "d9c7db97-85ec-4b86-a56a-35a83107a934" (UID: "d9c7db97-85ec-4b86-a56a-35a83107a934"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 17:28:04.822128 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:28:04.822107 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9c7db97-85ec-4b86-a56a-35a83107a934-kube-api-access-kj4x5" (OuterVolumeSpecName: "kube-api-access-kj4x5") pod "d9c7db97-85ec-4b86-a56a-35a83107a934" (UID: "d9c7db97-85ec-4b86-a56a-35a83107a934"). InnerVolumeSpecName "kube-api-access-kj4x5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 17:28:04.822194 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:28:04.822129 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9c7db97-85ec-4b86-a56a-35a83107a934-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d9c7db97-85ec-4b86-a56a-35a83107a934" (UID: "d9c7db97-85ec-4b86-a56a-35a83107a934"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 17:28:04.822194 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:28:04.822159 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9c7db97-85ec-4b86-a56a-35a83107a934-dshm" (OuterVolumeSpecName: "dshm") pod "d9c7db97-85ec-4b86-a56a-35a83107a934" (UID: "d9c7db97-85ec-4b86-a56a-35a83107a934"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 17:28:04.920949 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:28:04.920860 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kj4x5\" (UniqueName: \"kubernetes.io/projected/d9c7db97-85ec-4b86-a56a-35a83107a934-kube-api-access-kj4x5\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 17:28:04.920949 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:28:04.920896 2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9c7db97-85ec-4b86-a56a-35a83107a934-dshm\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 17:28:04.920949 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:28:04.920905 2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9c7db97-85ec-4b86-a56a-35a83107a934-model-cache\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 17:28:04.920949 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:28:04.920913 2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9c7db97-85ec-4b86-a56a-35a83107a934-tmp-dir\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 17:28:04.920949 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:28:04.920921 2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9c7db97-85ec-4b86-a56a-35a83107a934-tls-certs\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 17:28:04.920949 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:28:04.920928 2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9c7db97-85ec-4b86-a56a-35a83107a934-home\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 17:28:05.613244 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:28:05.613217 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-fc13d1/test-model-fc13d1-kserve-6d669f6669-86d8g" Jul 10 17:28:05.613645 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:28:05.613212 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-fc13d1/test-model-fc13d1-kserve-6d669f6669-86d8g" event={"ID":"d9c7db97-85ec-4b86-a56a-35a83107a934","Type":"ContainerDied","Data":"847909554c77644d76a165165e59c7d2007234d6f3576814ea9e5758ee402267"} Jul 10 17:28:05.613645 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:28:05.613342 2578 scope.go:117] "RemoveContainer" containerID="6c22244a2a64fa0edb247fe9873928093faec81db94ee5ffb699afcbca0b6b71" Jul 10 17:28:05.633546 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:28:05.633513 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-fc13d1/test-model-fc13d1-kserve-6d669f6669-86d8g"] Jul 10 17:28:05.637573 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:28:05.637548 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-fc13d1/test-model-fc13d1-kserve-6d669f6669-86d8g"] Jul 10 17:28:06.425197 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:28:06.425163 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9c7db97-85ec-4b86-a56a-35a83107a934" path="/var/lib/kubelet/pods/d9c7db97-85ec-4b86-a56a-35a83107a934/volumes" Jul 10 17:29:16.478383 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:29:16.478261 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-5j4ws_27742f15-b602-4554-84dc-ff9d46a9adda/console-operator/2.log" Jul 10 17:29:16.482517 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:29:16.481544 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-5j4ws_27742f15-b602-4554-84dc-ff9d46a9adda/console-operator/2.log" Jul 10 17:29:16.484654 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:29:16.484635 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gqgkg_b9a53126-f278-4a9d-9735-5f73dd0c1964/ovn-acl-logging/0.log" Jul 10 17:29:16.487956 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:29:16.487936 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gqgkg_b9a53126-f278-4a9d-9735-5f73dd0c1964/ovn-acl-logging/0.log" Jul 10 17:30:00.132010 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:00.131974 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728410-5vbcn"] Jul 10 17:30:00.132494 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:00.132362 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9c7db97-85ec-4b86-a56a-35a83107a934" containerName="main" Jul 10 17:30:00.132494 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:00.132375 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9c7db97-85ec-4b86-a56a-35a83107a934" containerName="main" Jul 10 17:30:00.132494 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:00.132440 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9c7db97-85ec-4b86-a56a-35a83107a934" containerName="main" Jul 10 17:30:00.135368 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:00.135348 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728410-5vbcn" Jul 10 17:30:00.138122 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:00.138096 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kqj5t\"" Jul 10 17:30:00.138249 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:00.138207 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-2482f8-29728410-62lzd"] Jul 10 17:30:00.142076 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:00.142022 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-43091a-29728410-svrlw"] Jul 10 17:30:00.142197 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:00.142155 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-2482f8-29728410-62lzd" Jul 10 17:30:00.146729 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:00.146705 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728410-5vbcn" Jul 10 17:30:00.147436 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:00.147416 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728410-5vbcn"] Jul 10 17:30:00.147689 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:00.147673 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-43091a-29728410-svrlw" Jul 10 17:30:00.149524 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:00.149501 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-43091a-29728410-svrlw"] Jul 10 17:30:00.154688 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:00.154655 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-2482f8-29728410-62lzd"] Jul 10 17:30:00.174798 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:00.174768 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-2482f8-29728410-62lzd" Jul 10 17:30:00.181351 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:00.181332 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-43091a-29728410-svrlw" Jul 10 17:30:00.319850 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:00.319755 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728410-5vbcn"] Jul 10 17:30:00.320113 ip-10-0-136-7 kubenswrapper[2578]: W0710 17:30:00.320012 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce0815eb_f110_45e7_9105_1606ac710e3d.slice/crio-df2c546d5280859209b81c23573a271f7a1356ded36768bf180d83cf6e2dd5cc WatchSource:0}: Error finding container df2c546d5280859209b81c23573a271f7a1356ded36768bf180d83cf6e2dd5cc: Status 404 returned error can't find the container with id df2c546d5280859209b81c23573a271f7a1356ded36768bf180d83cf6e2dd5cc Jul 10 17:30:00.338982 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:00.338889 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-2482f8-29728410-62lzd"] Jul 10 17:30:00.344581 ip-10-0-136-7 kubenswrapper[2578]: W0710 17:30:00.344553 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod576426fa_ca08_400c_a020_7740a8e866a6.slice/crio-2a388da02d3fc1c5ed418e82aeb4bb5f74e49e8e2936d07425d4ab4a770d1be4 WatchSource:0}: Error finding container 2a388da02d3fc1c5ed418e82aeb4bb5f74e49e8e2936d07425d4ab4a770d1be4: Status 404 returned error can't find the container with id 2a388da02d3fc1c5ed418e82aeb4bb5f74e49e8e2936d07425d4ab4a770d1be4 Jul 10 17:30:00.365686 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:00.365661 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-43091a-29728410-svrlw"] Jul 10 17:30:00.368106 ip-10-0-136-7 kubenswrapper[2578]: W0710 17:30:00.368067 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f2fa9d6_c2a6_4f16_98e6_28e485912e76.slice/crio-172024fde919575175b79553f8479084816a1ee07d618d42211c8f1924d90435 WatchSource:0}: Error finding container 172024fde919575175b79553f8479084816a1ee07d618d42211c8f1924d90435: Status 404 returned error can't find the container with id 172024fde919575175b79553f8479084816a1ee07d618d42211c8f1924d90435 Jul 10 17:30:01.125858 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:01.125814 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-2482f8-29728410-62lzd" event={"ID":"576426fa-ca08-400c-a020-7740a8e866a6","Type":"ContainerStarted","Data":"2a388da02d3fc1c5ed418e82aeb4bb5f74e49e8e2936d07425d4ab4a770d1be4"} Jul 10 17:30:01.126948 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:01.126919 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728410-5vbcn" event={"ID":"ce0815eb-f110-45e7-9105-1606ac710e3d","Type":"ContainerStarted","Data":"df2c546d5280859209b81c23573a271f7a1356ded36768bf180d83cf6e2dd5cc"} Jul 10 17:30:01.127973 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:01.127954 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-43091a-29728410-svrlw" event={"ID":"4f2fa9d6-c2a6-4f16-98e6-28e485912e76","Type":"ContainerStarted","Data":"172024fde919575175b79553f8479084816a1ee07d618d42211c8f1924d90435"} Jul 10 17:30:03.138618 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:03.138578 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728410-5vbcn" event={"ID":"ce0815eb-f110-45e7-9105-1606ac710e3d","Type":"ContainerStarted","Data":"dbbbe14f018b87f3b72357a80a71a7e38f314a260d870fa546492b56cc2bfc3d"} Jul 10 17:30:03.140217 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:03.140190 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-43091a-29728410-svrlw" event={"ID":"4f2fa9d6-c2a6-4f16-98e6-28e485912e76","Type":"ContainerStarted","Data":"65702e281ccef712a35017e25c76d62c827e0cdb0d33d1d8b77629723349e8c2"} Jul 10 17:30:03.153988 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:03.153940 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728410-5vbcn" podStartSLOduration=0.767501145 podStartE2EDuration="3.153913367s" totalImagesPullingTime="2.386412222s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 17:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:30:03.152548737 +0000 UTC m=+2147.241877788" watchObservedRunningTime="2026-07-10 17:30:03.153913367 +0000 UTC m=+2147.243242416" Jul 10 17:30:03.197098 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:03.197009 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-43091a-29728410-svrlw" podStartSLOduration=0.80361959 podStartE2EDuration="3.196992014s" totalImagesPullingTime="2.393372424s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 17:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:30:03.196015626 +0000 UTC m=+2147.285344675" watchObservedRunningTime="2026-07-10 17:30:03.196992014 +0000 UTC m=+2147.286321065" Jul 10 17:30:04.146612 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:04.146574 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-2482f8-29728410-62lzd" event={"ID":"576426fa-ca08-400c-a020-7740a8e866a6","Type":"ContainerStarted","Data":"ec665536af133b321afa2d3dff10521a186b4391066e7cba1de466c2baf39c47"} Jul 10 17:30:04.163244 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:04.163187 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-2482f8-29728410-62lzd" podStartSLOduration=1.742720495 podStartE2EDuration="4.16316657s" totalImagesPullingTime="2.420446075s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 17:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:30:04.162875567 +0000 UTC m=+2148.252204616" watchObservedRunningTime="2026-07-10 17:30:04.16316657 +0000 UTC m=+2148.252495621" Jul 10 17:30:30.938534 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:30.938497 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-2482f8/auth-test-model-2482f8-kserve-7d768b9459-nwfrs"] Jul 10 17:30:30.945976 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:30.945948 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2482f8/auth-test-model-2482f8-kserve-7d768b9459-nwfrs" Jul 10 17:30:30.950424 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:30.950062 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-2482f8\"/\"auth-test-model-2482f8-kserve-self-signed-certs\"" Jul 10 17:30:30.950424 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:30.950080 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-2482f8\"/\"default-dockercfg-8hgnp\"" Jul 10 17:30:30.950424 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:30.950095 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-2482f8\"/\"openshift-service-ca.crt\"" Jul 10 17:30:30.950424 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:30.950166 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-2482f8\"/\"kube-root-ca.crt\"" Jul 10 17:30:30.952681 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:30.952658 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-2482f8/auth-test-model-2482f8-kserve-7d768b9459-nwfrs"] Jul 10 17:30:30.990157 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:30.990125 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c2705ea-824e-4f04-875b-972bd2f595d9-tls-certs\") pod \"auth-test-model-2482f8-kserve-7d768b9459-nwfrs\" (UID: \"8c2705ea-824e-4f04-875b-972bd2f595d9\") " pod="ai-tenant-e2e-shared-b-2482f8/auth-test-model-2482f8-kserve-7d768b9459-nwfrs" Jul 10 17:30:30.990337 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:30.990172 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvnpn\" (UniqueName: \"kubernetes.io/projected/8c2705ea-824e-4f04-875b-972bd2f595d9-kube-api-access-xvnpn\") pod \"auth-test-model-2482f8-kserve-7d768b9459-nwfrs\" (UID: \"8c2705ea-824e-4f04-875b-972bd2f595d9\") " pod="ai-tenant-e2e-shared-b-2482f8/auth-test-model-2482f8-kserve-7d768b9459-nwfrs" Jul 10 17:30:30.990337 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:30.990206 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c2705ea-824e-4f04-875b-972bd2f595d9-tmp-dir\") pod \"auth-test-model-2482f8-kserve-7d768b9459-nwfrs\" (UID: \"8c2705ea-824e-4f04-875b-972bd2f595d9\") " pod="ai-tenant-e2e-shared-b-2482f8/auth-test-model-2482f8-kserve-7d768b9459-nwfrs" Jul 10 17:30:30.990337 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:30.990224 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c2705ea-824e-4f04-875b-972bd2f595d9-dshm\") pod \"auth-test-model-2482f8-kserve-7d768b9459-nwfrs\" (UID: \"8c2705ea-824e-4f04-875b-972bd2f595d9\") " pod="ai-tenant-e2e-shared-b-2482f8/auth-test-model-2482f8-kserve-7d768b9459-nwfrs" Jul 10 17:30:30.990337 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:30.990248 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c2705ea-824e-4f04-875b-972bd2f595d9-model-cache\") pod \"auth-test-model-2482f8-kserve-7d768b9459-nwfrs\" (UID: \"8c2705ea-824e-4f04-875b-972bd2f595d9\") " pod="ai-tenant-e2e-shared-b-2482f8/auth-test-model-2482f8-kserve-7d768b9459-nwfrs" Jul 10 17:30:30.990337 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:30.990268 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c2705ea-824e-4f04-875b-972bd2f595d9-home\") pod \"auth-test-model-2482f8-kserve-7d768b9459-nwfrs\" (UID: \"8c2705ea-824e-4f04-875b-972bd2f595d9\") " pod="ai-tenant-e2e-shared-b-2482f8/auth-test-model-2482f8-kserve-7d768b9459-nwfrs" Jul 10 17:30:31.091261 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:31.091228 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c2705ea-824e-4f04-875b-972bd2f595d9-model-cache\") pod \"auth-test-model-2482f8-kserve-7d768b9459-nwfrs\" (UID: \"8c2705ea-824e-4f04-875b-972bd2f595d9\") " pod="ai-tenant-e2e-shared-b-2482f8/auth-test-model-2482f8-kserve-7d768b9459-nwfrs" Jul 10 17:30:31.091449 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:31.091288 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c2705ea-824e-4f04-875b-972bd2f595d9-home\") pod \"auth-test-model-2482f8-kserve-7d768b9459-nwfrs\" (UID: \"8c2705ea-824e-4f04-875b-972bd2f595d9\") " pod="ai-tenant-e2e-shared-b-2482f8/auth-test-model-2482f8-kserve-7d768b9459-nwfrs" Jul 10 17:30:31.091449 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:31.091372 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c2705ea-824e-4f04-875b-972bd2f595d9-tls-certs\") pod \"auth-test-model-2482f8-kserve-7d768b9459-nwfrs\" (UID: \"8c2705ea-824e-4f04-875b-972bd2f595d9\") " pod="ai-tenant-e2e-shared-b-2482f8/auth-test-model-2482f8-kserve-7d768b9459-nwfrs" Jul 10 17:30:31.091449 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:31.091413 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvnpn\" (UniqueName: \"kubernetes.io/projected/8c2705ea-824e-4f04-875b-972bd2f595d9-kube-api-access-xvnpn\") pod \"auth-test-model-2482f8-kserve-7d768b9459-nwfrs\" (UID: \"8c2705ea-824e-4f04-875b-972bd2f595d9\") " pod="ai-tenant-e2e-shared-b-2482f8/auth-test-model-2482f8-kserve-7d768b9459-nwfrs" Jul 10 17:30:31.091614 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:31.091454 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c2705ea-824e-4f04-875b-972bd2f595d9-tmp-dir\") pod \"auth-test-model-2482f8-kserve-7d768b9459-nwfrs\" (UID: \"8c2705ea-824e-4f04-875b-972bd2f595d9\") " pod="ai-tenant-e2e-shared-b-2482f8/auth-test-model-2482f8-kserve-7d768b9459-nwfrs" Jul 10 17:30:31.091614 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:31.091481 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c2705ea-824e-4f04-875b-972bd2f595d9-dshm\") pod \"auth-test-model-2482f8-kserve-7d768b9459-nwfrs\" (UID: \"8c2705ea-824e-4f04-875b-972bd2f595d9\") " pod="ai-tenant-e2e-shared-b-2482f8/auth-test-model-2482f8-kserve-7d768b9459-nwfrs" Jul 10 17:30:31.091744 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:31.091708 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c2705ea-824e-4f04-875b-972bd2f595d9-model-cache\") pod \"auth-test-model-2482f8-kserve-7d768b9459-nwfrs\" (UID: \"8c2705ea-824e-4f04-875b-972bd2f595d9\") " pod="ai-tenant-e2e-shared-b-2482f8/auth-test-model-2482f8-kserve-7d768b9459-nwfrs" Jul 10 17:30:31.092112 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:31.091855 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c2705ea-824e-4f04-875b-972bd2f595d9-home\") pod \"auth-test-model-2482f8-kserve-7d768b9459-nwfrs\" (UID: \"8c2705ea-824e-4f04-875b-972bd2f595d9\") " pod="ai-tenant-e2e-shared-b-2482f8/auth-test-model-2482f8-kserve-7d768b9459-nwfrs" Jul 10 17:30:31.092112 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:31.091855 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c2705ea-824e-4f04-875b-972bd2f595d9-tmp-dir\") pod \"auth-test-model-2482f8-kserve-7d768b9459-nwfrs\" (UID: \"8c2705ea-824e-4f04-875b-972bd2f595d9\") " pod="ai-tenant-e2e-shared-b-2482f8/auth-test-model-2482f8-kserve-7d768b9459-nwfrs" Jul 10 17:30:31.094175 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:31.094149 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c2705ea-824e-4f04-875b-972bd2f595d9-dshm\") pod \"auth-test-model-2482f8-kserve-7d768b9459-nwfrs\" (UID: \"8c2705ea-824e-4f04-875b-972bd2f595d9\") " pod="ai-tenant-e2e-shared-b-2482f8/auth-test-model-2482f8-kserve-7d768b9459-nwfrs" Jul 10 17:30:31.094366 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:31.094349 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c2705ea-824e-4f04-875b-972bd2f595d9-tls-certs\") pod \"auth-test-model-2482f8-kserve-7d768b9459-nwfrs\" (UID: \"8c2705ea-824e-4f04-875b-972bd2f595d9\") " pod="ai-tenant-e2e-shared-b-2482f8/auth-test-model-2482f8-kserve-7d768b9459-nwfrs" Jul 10 17:30:31.101485 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:31.101455 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvnpn\" (UniqueName: \"kubernetes.io/projected/8c2705ea-824e-4f04-875b-972bd2f595d9-kube-api-access-xvnpn\") pod \"auth-test-model-2482f8-kserve-7d768b9459-nwfrs\" (UID: \"8c2705ea-824e-4f04-875b-972bd2f595d9\") " pod="ai-tenant-e2e-shared-b-2482f8/auth-test-model-2482f8-kserve-7d768b9459-nwfrs" Jul 10 17:30:31.270502 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:31.270417 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2482f8/auth-test-model-2482f8-kserve-7d768b9459-nwfrs" Jul 10 17:30:31.407257 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:31.407228 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-2482f8/auth-test-model-2482f8-kserve-7d768b9459-nwfrs"] Jul 10 17:30:31.409355 ip-10-0-136-7 kubenswrapper[2578]: W0710 17:30:31.409331 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8c2705ea_824e_4f04_875b_972bd2f595d9.slice/crio-e0cf4c170cd456697238648e44ddc3b308abc35cef19dfa6844149a2850d57a0 WatchSource:0}: Error finding container e0cf4c170cd456697238648e44ddc3b308abc35cef19dfa6844149a2850d57a0: Status 404 returned error can't find the container with id e0cf4c170cd456697238648e44ddc3b308abc35cef19dfa6844149a2850d57a0 Jul 10 17:30:32.275432 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:32.275399 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2482f8/auth-test-model-2482f8-kserve-7d768b9459-nwfrs" event={"ID":"8c2705ea-824e-4f04-875b-972bd2f595d9","Type":"ContainerStarted","Data":"0da392d31a228a19bc373e89f033dad764e642326d65fc75f5db8d5a1d685462"} Jul 10 17:30:32.275924 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:32.275438 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2482f8/auth-test-model-2482f8-kserve-7d768b9459-nwfrs" event={"ID":"8c2705ea-824e-4f04-875b-972bd2f595d9","Type":"ContainerStarted","Data":"e0cf4c170cd456697238648e44ddc3b308abc35cef19dfa6844149a2850d57a0"} Jul 10 17:30:32.292508 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:32.292457 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-2482f8/auth-test-model-2482f8-kserve-7d768b9459-nwfrs" podStartSLOduration=2.292441805 podStartE2EDuration="2.292441805s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 17:30:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:30:32.292325822 +0000 UTC m=+2176.381654875" watchObservedRunningTime="2026-07-10 17:30:32.292441805 +0000 UTC m=+2176.381770854" Jul 10 17:30:41.271605 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:41.271566 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-2482f8/auth-test-model-2482f8-kserve-7d768b9459-nwfrs" Jul 10 17:30:41.272019 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:41.271621 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-2482f8/auth-test-model-2482f8-kserve-7d768b9459-nwfrs" Jul 10 17:30:41.284426 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:41.284394 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-2482f8/auth-test-model-2482f8-kserve-7d768b9459-nwfrs" Jul 10 17:30:41.332070 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:30:41.332027 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-2482f8/auth-test-model-2482f8-kserve-7d768b9459-nwfrs" Jul 10 17:31:00.010302 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:31:00.010259 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-2482f8-29728410-62lzd"] Jul 10 17:31:00.010845 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:31:00.010471 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-2482f8-29728410-62lzd" podUID="576426fa-ca08-400c-a020-7740a8e866a6" containerName="cleanup" containerID="cri-o://ec665536af133b321afa2d3dff10521a186b4391066e7cba1de466c2baf39c47" gracePeriod=30 Jul 10 17:31:00.013956 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:31:00.013922 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-43091a-29728410-svrlw"] Jul 10 17:31:00.014176 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:31:00.014139 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-43091a-29728410-svrlw" podUID="4f2fa9d6-c2a6-4f16-98e6-28e485912e76" containerName="cleanup" containerID="cri-o://65702e281ccef712a35017e25c76d62c827e0cdb0d33d1d8b77629723349e8c2" gracePeriod=30 Jul 10 17:31:00.015455 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:31:00.015432 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728410-5vbcn"] Jul 10 17:31:00.015663 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:31:00.015624 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728410-5vbcn" podUID="ce0815eb-f110-45e7-9105-1606ac710e3d" containerName="cleanup" containerID="cri-o://dbbbe14f018b87f3b72357a80a71a7e38f314a260d870fa546492b56cc2bfc3d" gracePeriod=30 Jul 10 17:31:30.204917 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:31:30.204880 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-2482f8-29728410-62lzd" Jul 10 17:31:30.248428 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:31:30.248402 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728410-5vbcn" Jul 10 17:31:30.251400 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:31:30.251379 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-43091a-29728410-svrlw" Jul 10 17:31:30.542351 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:31:30.542316 2578 generic.go:356] "Generic (PLEG): container finished" podID="576426fa-ca08-400c-a020-7740a8e866a6" containerID="ec665536af133b321afa2d3dff10521a186b4391066e7cba1de466c2baf39c47" exitCode=137 Jul 10 17:31:30.542533 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:31:30.542364 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-2482f8-29728410-62lzd" Jul 10 17:31:30.542533 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:31:30.542404 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-2482f8-29728410-62lzd" event={"ID":"576426fa-ca08-400c-a020-7740a8e866a6","Type":"ContainerDied","Data":"ec665536af133b321afa2d3dff10521a186b4391066e7cba1de466c2baf39c47"} Jul 10 17:31:30.542533 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:31:30.542471 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-2482f8-29728410-62lzd" event={"ID":"576426fa-ca08-400c-a020-7740a8e866a6","Type":"ContainerDied","Data":"2a388da02d3fc1c5ed418e82aeb4bb5f74e49e8e2936d07425d4ab4a770d1be4"} Jul 10 17:31:30.542533 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:31:30.542506 2578 scope.go:117] "RemoveContainer" containerID="ec665536af133b321afa2d3dff10521a186b4391066e7cba1de466c2baf39c47" Jul 10 17:31:30.543703 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:31:30.543659 2578 generic.go:356] "Generic (PLEG): container finished" podID="ce0815eb-f110-45e7-9105-1606ac710e3d" containerID="dbbbe14f018b87f3b72357a80a71a7e38f314a260d870fa546492b56cc2bfc3d" exitCode=137 Jul 10 17:31:30.543703 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:31:30.543691 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728410-5vbcn" event={"ID":"ce0815eb-f110-45e7-9105-1606ac710e3d","Type":"ContainerDied","Data":"dbbbe14f018b87f3b72357a80a71a7e38f314a260d870fa546492b56cc2bfc3d"} Jul 10 17:31:30.543824 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:31:30.543704 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728410-5vbcn" Jul 10 17:31:30.543824 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:31:30.543750 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728410-5vbcn" event={"ID":"ce0815eb-f110-45e7-9105-1606ac710e3d","Type":"ContainerDied","Data":"df2c546d5280859209b81c23573a271f7a1356ded36768bf180d83cf6e2dd5cc"} Jul 10 17:31:30.545331 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:31:30.545306 2578 generic.go:356] "Generic (PLEG): container finished" podID="4f2fa9d6-c2a6-4f16-98e6-28e485912e76" containerID="65702e281ccef712a35017e25c76d62c827e0cdb0d33d1d8b77629723349e8c2" exitCode=137 Jul 10 17:31:30.545457 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:31:30.545353 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-43091a-29728410-svrlw" Jul 10 17:31:30.545457 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:31:30.545383 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-43091a-29728410-svrlw" event={"ID":"4f2fa9d6-c2a6-4f16-98e6-28e485912e76","Type":"ContainerDied","Data":"65702e281ccef712a35017e25c76d62c827e0cdb0d33d1d8b77629723349e8c2"} Jul 10 17:31:30.545457 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:31:30.545433 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-43091a-29728410-svrlw" event={"ID":"4f2fa9d6-c2a6-4f16-98e6-28e485912e76","Type":"ContainerDied","Data":"172024fde919575175b79553f8479084816a1ee07d618d42211c8f1924d90435"} Jul 10 17:31:30.552610 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:31:30.552592 2578 scope.go:117] "RemoveContainer" containerID="ec665536af133b321afa2d3dff10521a186b4391066e7cba1de466c2baf39c47" Jul 10 17:31:30.552901 ip-10-0-136-7 kubenswrapper[2578]: E0710 17:31:30.552882 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec665536af133b321afa2d3dff10521a186b4391066e7cba1de466c2baf39c47\": container with ID starting with ec665536af133b321afa2d3dff10521a186b4391066e7cba1de466c2baf39c47 not found: ID does not exist" containerID="ec665536af133b321afa2d3dff10521a186b4391066e7cba1de466c2baf39c47" Jul 10 17:31:30.552943 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:31:30.552910 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec665536af133b321afa2d3dff10521a186b4391066e7cba1de466c2baf39c47"} err="failed to get container status \"ec665536af133b321afa2d3dff10521a186b4391066e7cba1de466c2baf39c47\": rpc error: code = NotFound desc = could not find container \"ec665536af133b321afa2d3dff10521a186b4391066e7cba1de466c2baf39c47\": container with ID starting with ec665536af133b321afa2d3dff10521a186b4391066e7cba1de466c2baf39c47 not found: ID does not exist" Jul 10 17:31:30.552943 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:31:30.552927 2578 scope.go:117] "RemoveContainer" containerID="dbbbe14f018b87f3b72357a80a71a7e38f314a260d870fa546492b56cc2bfc3d" Jul 10 17:31:30.560599 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:31:30.560578 2578 scope.go:117] "RemoveContainer" containerID="dbbbe14f018b87f3b72357a80a71a7e38f314a260d870fa546492b56cc2bfc3d" Jul 10 17:31:30.560855 ip-10-0-136-7 kubenswrapper[2578]: E0710 17:31:30.560836 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dbbbe14f018b87f3b72357a80a71a7e38f314a260d870fa546492b56cc2bfc3d\": container with ID starting with dbbbe14f018b87f3b72357a80a71a7e38f314a260d870fa546492b56cc2bfc3d not found: ID does not exist" containerID="dbbbe14f018b87f3b72357a80a71a7e38f314a260d870fa546492b56cc2bfc3d" Jul 10 17:31:30.560900 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:31:30.560865 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbbbe14f018b87f3b72357a80a71a7e38f314a260d870fa546492b56cc2bfc3d"} err="failed to get container status \"dbbbe14f018b87f3b72357a80a71a7e38f314a260d870fa546492b56cc2bfc3d\": rpc error: code = NotFound desc = could not find container \"dbbbe14f018b87f3b72357a80a71a7e38f314a260d870fa546492b56cc2bfc3d\": container with ID starting with dbbbe14f018b87f3b72357a80a71a7e38f314a260d870fa546492b56cc2bfc3d not found: ID does not exist" Jul 10 17:31:30.560900 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:31:30.560883 2578 scope.go:117] "RemoveContainer" containerID="65702e281ccef712a35017e25c76d62c827e0cdb0d33d1d8b77629723349e8c2" Jul 10 17:31:30.564755 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:31:30.564733 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728410-5vbcn"] Jul 10 17:31:30.568673 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:31:30.568649 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728410-5vbcn"] Jul 10 17:31:30.568964 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:31:30.568946 2578 scope.go:117] "RemoveContainer" containerID="65702e281ccef712a35017e25c76d62c827e0cdb0d33d1d8b77629723349e8c2" Jul 10 17:31:30.569258 ip-10-0-136-7 kubenswrapper[2578]: E0710 17:31:30.569240 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65702e281ccef712a35017e25c76d62c827e0cdb0d33d1d8b77629723349e8c2\": container with ID starting with 65702e281ccef712a35017e25c76d62c827e0cdb0d33d1d8b77629723349e8c2 not found: ID does not exist" containerID="65702e281ccef712a35017e25c76d62c827e0cdb0d33d1d8b77629723349e8c2" Jul 10 17:31:30.569326 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:31:30.569264 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65702e281ccef712a35017e25c76d62c827e0cdb0d33d1d8b77629723349e8c2"} err="failed to get container status \"65702e281ccef712a35017e25c76d62c827e0cdb0d33d1d8b77629723349e8c2\": rpc error: code = NotFound desc = could not find container \"65702e281ccef712a35017e25c76d62c827e0cdb0d33d1d8b77629723349e8c2\": container with ID starting with 65702e281ccef712a35017e25c76d62c827e0cdb0d33d1d8b77629723349e8c2 not found: ID does not exist" Jul 10 17:31:30.578509 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:31:30.578487 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-2482f8-29728410-62lzd"] Jul 10 17:31:30.582167 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:31:30.582143 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-2482f8-29728410-62lzd"] Jul 10 17:31:30.592826 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:31:30.592804 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-43091a-29728410-svrlw"] Jul 10 17:31:30.594182 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:31:30.594160 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-43091a-29728410-svrlw"] Jul 10 17:31:32.425268 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:31:32.425231 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f2fa9d6-c2a6-4f16-98e6-28e485912e76" path="/var/lib/kubelet/pods/4f2fa9d6-c2a6-4f16-98e6-28e485912e76/volumes" Jul 10 17:31:32.425674 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:31:32.425486 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="576426fa-ca08-400c-a020-7740a8e866a6" path="/var/lib/kubelet/pods/576426fa-ca08-400c-a020-7740a8e866a6/volumes" Jul 10 17:31:32.425715 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:31:32.425674 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce0815eb-f110-45e7-9105-1606ac710e3d" path="/var/lib/kubelet/pods/ce0815eb-f110-45e7-9105-1606ac710e3d/volumes" Jul 10 17:32:38.333795 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:38.333762 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-2482f8/auth-test-model-2482f8-kserve-7d768b9459-nwfrs"] Jul 10 17:32:44.042192 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:44.042154 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-43091a/sub-test-model-43091a-kserve-77655999f5-xnwgj"] Jul 10 17:32:44.042639 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:44.042494 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="576426fa-ca08-400c-a020-7740a8e866a6" containerName="cleanup" Jul 10 17:32:44.042639 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:44.042505 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="576426fa-ca08-400c-a020-7740a8e866a6" containerName="cleanup" Jul 10 17:32:44.042639 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:44.042527 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f2fa9d6-c2a6-4f16-98e6-28e485912e76" containerName="cleanup" Jul 10 17:32:44.042639 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:44.042532 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f2fa9d6-c2a6-4f16-98e6-28e485912e76" containerName="cleanup" Jul 10 17:32:44.042639 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:44.042541 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce0815eb-f110-45e7-9105-1606ac710e3d" containerName="cleanup" Jul 10 17:32:44.042639 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:44.042546 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce0815eb-f110-45e7-9105-1606ac710e3d" containerName="cleanup" Jul 10 17:32:44.042639 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:44.042595 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="576426fa-ca08-400c-a020-7740a8e866a6" containerName="cleanup" Jul 10 17:32:44.042639 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:44.042605 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce0815eb-f110-45e7-9105-1606ac710e3d" containerName="cleanup" Jul 10 17:32:44.042639 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:44.042611 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f2fa9d6-c2a6-4f16-98e6-28e485912e76" containerName="cleanup" Jul 10 17:32:44.045510 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:44.045492 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-43091a/sub-test-model-43091a-kserve-77655999f5-xnwgj" Jul 10 17:32:44.049514 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:44.049487 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-43091a\"/\"openshift-service-ca.crt\"" Jul 10 17:32:44.049659 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:44.049532 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-43091a\"/\"default-dockercfg-644pv\"" Jul 10 17:32:44.049659 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:44.049532 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-43091a\"/\"sub-test-model-43091a-kserve-self-signed-certs\"" Jul 10 17:32:44.049756 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:44.049695 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-43091a\"/\"kube-root-ca.crt\"" Jul 10 17:32:44.055120 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:44.055087 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-43091a/sub-test-model-43091a-kserve-77655999f5-xnwgj"] Jul 10 17:32:44.106224 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:44.106183 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/455fc0dc-c5fa-4014-9f96-d5edda5bbba2-dshm\") pod \"sub-test-model-43091a-kserve-77655999f5-xnwgj\" (UID: \"455fc0dc-c5fa-4014-9f96-d5edda5bbba2\") " pod="ai-tenant-e2e-shared-a-43091a/sub-test-model-43091a-kserve-77655999f5-xnwgj" Jul 10 17:32:44.106398 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:44.106252 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/455fc0dc-c5fa-4014-9f96-d5edda5bbba2-tmp-dir\") pod \"sub-test-model-43091a-kserve-77655999f5-xnwgj\" (UID: \"455fc0dc-c5fa-4014-9f96-d5edda5bbba2\") " pod="ai-tenant-e2e-shared-a-43091a/sub-test-model-43091a-kserve-77655999f5-xnwgj" Jul 10 17:32:44.106398 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:44.106293 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2s2q\" (UniqueName: \"kubernetes.io/projected/455fc0dc-c5fa-4014-9f96-d5edda5bbba2-kube-api-access-j2s2q\") pod \"sub-test-model-43091a-kserve-77655999f5-xnwgj\" (UID: \"455fc0dc-c5fa-4014-9f96-d5edda5bbba2\") " pod="ai-tenant-e2e-shared-a-43091a/sub-test-model-43091a-kserve-77655999f5-xnwgj" Jul 10 17:32:44.106398 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:44.106325 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/455fc0dc-c5fa-4014-9f96-d5edda5bbba2-tls-certs\") pod \"sub-test-model-43091a-kserve-77655999f5-xnwgj\" (UID: \"455fc0dc-c5fa-4014-9f96-d5edda5bbba2\") " pod="ai-tenant-e2e-shared-a-43091a/sub-test-model-43091a-kserve-77655999f5-xnwgj" Jul 10 17:32:44.106398 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:44.106347 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/455fc0dc-c5fa-4014-9f96-d5edda5bbba2-home\") pod \"sub-test-model-43091a-kserve-77655999f5-xnwgj\" (UID: \"455fc0dc-c5fa-4014-9f96-d5edda5bbba2\") " pod="ai-tenant-e2e-shared-a-43091a/sub-test-model-43091a-kserve-77655999f5-xnwgj" Jul 10 17:32:44.106398 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:44.106392 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/455fc0dc-c5fa-4014-9f96-d5edda5bbba2-model-cache\") pod \"sub-test-model-43091a-kserve-77655999f5-xnwgj\" (UID: \"455fc0dc-c5fa-4014-9f96-d5edda5bbba2\") " pod="ai-tenant-e2e-shared-a-43091a/sub-test-model-43091a-kserve-77655999f5-xnwgj" Jul 10 17:32:44.207261 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:44.207220 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2s2q\" (UniqueName: \"kubernetes.io/projected/455fc0dc-c5fa-4014-9f96-d5edda5bbba2-kube-api-access-j2s2q\") pod \"sub-test-model-43091a-kserve-77655999f5-xnwgj\" (UID: \"455fc0dc-c5fa-4014-9f96-d5edda5bbba2\") " pod="ai-tenant-e2e-shared-a-43091a/sub-test-model-43091a-kserve-77655999f5-xnwgj" Jul 10 17:32:44.207466 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:44.207270 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/455fc0dc-c5fa-4014-9f96-d5edda5bbba2-tls-certs\") pod \"sub-test-model-43091a-kserve-77655999f5-xnwgj\" (UID: \"455fc0dc-c5fa-4014-9f96-d5edda5bbba2\") " pod="ai-tenant-e2e-shared-a-43091a/sub-test-model-43091a-kserve-77655999f5-xnwgj" Jul 10 17:32:44.207466 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:44.207297 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/455fc0dc-c5fa-4014-9f96-d5edda5bbba2-home\") pod \"sub-test-model-43091a-kserve-77655999f5-xnwgj\" (UID: \"455fc0dc-c5fa-4014-9f96-d5edda5bbba2\") " pod="ai-tenant-e2e-shared-a-43091a/sub-test-model-43091a-kserve-77655999f5-xnwgj" Jul 10 17:32:44.207466 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:44.207342 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/455fc0dc-c5fa-4014-9f96-d5edda5bbba2-model-cache\") pod \"sub-test-model-43091a-kserve-77655999f5-xnwgj\" (UID: \"455fc0dc-c5fa-4014-9f96-d5edda5bbba2\") " pod="ai-tenant-e2e-shared-a-43091a/sub-test-model-43091a-kserve-77655999f5-xnwgj" Jul 10 17:32:44.207466 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:44.207385 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/455fc0dc-c5fa-4014-9f96-d5edda5bbba2-dshm\") pod \"sub-test-model-43091a-kserve-77655999f5-xnwgj\" (UID: \"455fc0dc-c5fa-4014-9f96-d5edda5bbba2\") " pod="ai-tenant-e2e-shared-a-43091a/sub-test-model-43091a-kserve-77655999f5-xnwgj" Jul 10 17:32:44.207466 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:44.207444 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/455fc0dc-c5fa-4014-9f96-d5edda5bbba2-tmp-dir\") pod \"sub-test-model-43091a-kserve-77655999f5-xnwgj\" (UID: \"455fc0dc-c5fa-4014-9f96-d5edda5bbba2\") " pod="ai-tenant-e2e-shared-a-43091a/sub-test-model-43091a-kserve-77655999f5-xnwgj" Jul 10 17:32:44.207870 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:44.207845 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/455fc0dc-c5fa-4014-9f96-d5edda5bbba2-home\") pod \"sub-test-model-43091a-kserve-77655999f5-xnwgj\" (UID: \"455fc0dc-c5fa-4014-9f96-d5edda5bbba2\") " pod="ai-tenant-e2e-shared-a-43091a/sub-test-model-43091a-kserve-77655999f5-xnwgj" Jul 10 17:32:44.207967 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:44.207879 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/455fc0dc-c5fa-4014-9f96-d5edda5bbba2-model-cache\") pod \"sub-test-model-43091a-kserve-77655999f5-xnwgj\" (UID: \"455fc0dc-c5fa-4014-9f96-d5edda5bbba2\") " pod="ai-tenant-e2e-shared-a-43091a/sub-test-model-43091a-kserve-77655999f5-xnwgj" Jul 10 17:32:44.208008 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:44.207969 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/455fc0dc-c5fa-4014-9f96-d5edda5bbba2-tmp-dir\") pod \"sub-test-model-43091a-kserve-77655999f5-xnwgj\" (UID: \"455fc0dc-c5fa-4014-9f96-d5edda5bbba2\") " pod="ai-tenant-e2e-shared-a-43091a/sub-test-model-43091a-kserve-77655999f5-xnwgj" Jul 10 17:32:44.210393 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:44.210366 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/455fc0dc-c5fa-4014-9f96-d5edda5bbba2-dshm\") pod \"sub-test-model-43091a-kserve-77655999f5-xnwgj\" (UID: \"455fc0dc-c5fa-4014-9f96-d5edda5bbba2\") " pod="ai-tenant-e2e-shared-a-43091a/sub-test-model-43091a-kserve-77655999f5-xnwgj" Jul 10 17:32:44.210582 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:44.210562 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/455fc0dc-c5fa-4014-9f96-d5edda5bbba2-tls-certs\") pod \"sub-test-model-43091a-kserve-77655999f5-xnwgj\" (UID: \"455fc0dc-c5fa-4014-9f96-d5edda5bbba2\") " pod="ai-tenant-e2e-shared-a-43091a/sub-test-model-43091a-kserve-77655999f5-xnwgj" Jul 10 17:32:44.215396 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:44.215375 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2s2q\" (UniqueName: \"kubernetes.io/projected/455fc0dc-c5fa-4014-9f96-d5edda5bbba2-kube-api-access-j2s2q\") pod \"sub-test-model-43091a-kserve-77655999f5-xnwgj\" (UID: \"455fc0dc-c5fa-4014-9f96-d5edda5bbba2\") " pod="ai-tenant-e2e-shared-a-43091a/sub-test-model-43091a-kserve-77655999f5-xnwgj" Jul 10 17:32:44.355749 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:44.355658 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-43091a/sub-test-model-43091a-kserve-77655999f5-xnwgj" Jul 10 17:32:44.493983 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:44.493954 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-43091a/sub-test-model-43091a-kserve-77655999f5-xnwgj"] Jul 10 17:32:44.496820 ip-10-0-136-7 kubenswrapper[2578]: W0710 17:32:44.496795 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod455fc0dc_c5fa_4014_9f96_d5edda5bbba2.slice/crio-c404a1e8501baf4190485d49055e2c1e7dd7656e6fe9d6f0b313889fc21e494c WatchSource:0}: Error finding container c404a1e8501baf4190485d49055e2c1e7dd7656e6fe9d6f0b313889fc21e494c: Status 404 returned error can't find the container with id c404a1e8501baf4190485d49055e2c1e7dd7656e6fe9d6f0b313889fc21e494c Jul 10 17:32:44.892094 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:44.892017 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-43091a/sub-test-model-43091a-kserve-77655999f5-xnwgj" event={"ID":"455fc0dc-c5fa-4014-9f96-d5edda5bbba2","Type":"ContainerStarted","Data":"58618b8a62a68229dc5cb2c61779132861acaa407be4a23b34b35810f493fbd0"} Jul 10 17:32:44.892285 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:44.892107 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-43091a/sub-test-model-43091a-kserve-77655999f5-xnwgj" event={"ID":"455fc0dc-c5fa-4014-9f96-d5edda5bbba2","Type":"ContainerStarted","Data":"c404a1e8501baf4190485d49055e2c1e7dd7656e6fe9d6f0b313889fc21e494c"} Jul 10 17:32:44.910083 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:44.910014 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-43091a/sub-test-model-43091a-kserve-77655999f5-xnwgj" podStartSLOduration=0.909999205 podStartE2EDuration="909.999205ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 17:32:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:32:44.90808616 +0000 UTC m=+2308.997415206" watchObservedRunningTime="2026-07-10 17:32:44.909999205 +0000 UTC m=+2308.999328254" Jul 10 17:32:53.354756 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:53.354697 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-2482f8/auth-test-model-2482f8-kserve-7d768b9459-nwfrs" podUID="8c2705ea-824e-4f04-875b-972bd2f595d9" containerName="main" containerID="cri-o://0da392d31a228a19bc373e89f033dad764e642326d65fc75f5db8d5a1d685462" gracePeriod=45 Jul 10 17:32:53.604630 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:53.604600 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2482f8/auth-test-model-2482f8-kserve-7d768b9459-nwfrs" Jul 10 17:32:53.692185 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:53.692090 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c2705ea-824e-4f04-875b-972bd2f595d9-model-cache\") pod \"8c2705ea-824e-4f04-875b-972bd2f595d9\" (UID: \"8c2705ea-824e-4f04-875b-972bd2f595d9\") " Jul 10 17:32:53.692185 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:53.692141 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c2705ea-824e-4f04-875b-972bd2f595d9-tls-certs\") pod \"8c2705ea-824e-4f04-875b-972bd2f595d9\" (UID: \"8c2705ea-824e-4f04-875b-972bd2f595d9\") " Jul 10 17:32:53.692185 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:53.692163 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c2705ea-824e-4f04-875b-972bd2f595d9-dshm\") pod \"8c2705ea-824e-4f04-875b-972bd2f595d9\" (UID: \"8c2705ea-824e-4f04-875b-972bd2f595d9\") " Jul 10 17:32:53.692475 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:53.692243 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c2705ea-824e-4f04-875b-972bd2f595d9-tmp-dir\") pod \"8c2705ea-824e-4f04-875b-972bd2f595d9\" (UID: \"8c2705ea-824e-4f04-875b-972bd2f595d9\") " Jul 10 17:32:53.692475 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:53.692277 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c2705ea-824e-4f04-875b-972bd2f595d9-home\") pod \"8c2705ea-824e-4f04-875b-972bd2f595d9\" (UID: \"8c2705ea-824e-4f04-875b-972bd2f595d9\") " Jul 10 17:32:53.692475 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:53.692362 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xvnpn\" (UniqueName: \"kubernetes.io/projected/8c2705ea-824e-4f04-875b-972bd2f595d9-kube-api-access-xvnpn\") pod \"8c2705ea-824e-4f04-875b-972bd2f595d9\" (UID: \"8c2705ea-824e-4f04-875b-972bd2f595d9\") " Jul 10 17:32:53.692475 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:53.692389 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c2705ea-824e-4f04-875b-972bd2f595d9-model-cache" (OuterVolumeSpecName: "model-cache") pod "8c2705ea-824e-4f04-875b-972bd2f595d9" (UID: "8c2705ea-824e-4f04-875b-972bd2f595d9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 17:32:53.692657 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:53.692592 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c2705ea-824e-4f04-875b-972bd2f595d9-home" (OuterVolumeSpecName: "home") pod "8c2705ea-824e-4f04-875b-972bd2f595d9" (UID: "8c2705ea-824e-4f04-875b-972bd2f595d9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 17:32:53.692657 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:53.692588 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c2705ea-824e-4f04-875b-972bd2f595d9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8c2705ea-824e-4f04-875b-972bd2f595d9" (UID: "8c2705ea-824e-4f04-875b-972bd2f595d9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 17:32:53.692657 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:53.692627 2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c2705ea-824e-4f04-875b-972bd2f595d9-model-cache\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 17:32:53.694574 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:53.694549 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c2705ea-824e-4f04-875b-972bd2f595d9-dshm" (OuterVolumeSpecName: "dshm") pod "8c2705ea-824e-4f04-875b-972bd2f595d9" (UID: "8c2705ea-824e-4f04-875b-972bd2f595d9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 17:32:53.694668 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:53.694614 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c2705ea-824e-4f04-875b-972bd2f595d9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8c2705ea-824e-4f04-875b-972bd2f595d9" (UID: "8c2705ea-824e-4f04-875b-972bd2f595d9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 17:32:53.694668 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:53.694630 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c2705ea-824e-4f04-875b-972bd2f595d9-kube-api-access-xvnpn" (OuterVolumeSpecName: "kube-api-access-xvnpn") pod "8c2705ea-824e-4f04-875b-972bd2f595d9" (UID: "8c2705ea-824e-4f04-875b-972bd2f595d9"). InnerVolumeSpecName "kube-api-access-xvnpn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 17:32:53.793526 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:53.793486 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xvnpn\" (UniqueName: \"kubernetes.io/projected/8c2705ea-824e-4f04-875b-972bd2f595d9-kube-api-access-xvnpn\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 17:32:53.793526 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:53.793519 2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c2705ea-824e-4f04-875b-972bd2f595d9-tls-certs\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 17:32:53.793526 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:53.793528 2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c2705ea-824e-4f04-875b-972bd2f595d9-dshm\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 17:32:53.793526 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:53.793536 2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c2705ea-824e-4f04-875b-972bd2f595d9-tmp-dir\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 17:32:53.793526 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:53.793543 2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c2705ea-824e-4f04-875b-972bd2f595d9-home\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 17:32:53.934665 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:53.934631 2578 generic.go:356] "Generic (PLEG): container finished" podID="8c2705ea-824e-4f04-875b-972bd2f595d9" containerID="0da392d31a228a19bc373e89f033dad764e642326d65fc75f5db8d5a1d685462" exitCode=0 Jul 10 17:32:53.934864 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:53.934680 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2482f8/auth-test-model-2482f8-kserve-7d768b9459-nwfrs" event={"ID":"8c2705ea-824e-4f04-875b-972bd2f595d9","Type":"ContainerDied","Data":"0da392d31a228a19bc373e89f033dad764e642326d65fc75f5db8d5a1d685462"} Jul 10 17:32:53.934864 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:53.934685 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2482f8/auth-test-model-2482f8-kserve-7d768b9459-nwfrs" Jul 10 17:32:53.934864 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:53.934713 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2482f8/auth-test-model-2482f8-kserve-7d768b9459-nwfrs" event={"ID":"8c2705ea-824e-4f04-875b-972bd2f595d9","Type":"ContainerDied","Data":"e0cf4c170cd456697238648e44ddc3b308abc35cef19dfa6844149a2850d57a0"} Jul 10 17:32:53.934864 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:53.934730 2578 scope.go:117] "RemoveContainer" containerID="0da392d31a228a19bc373e89f033dad764e642326d65fc75f5db8d5a1d685462" Jul 10 17:32:53.943838 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:53.943810 2578 scope.go:117] "RemoveContainer" containerID="0da392d31a228a19bc373e89f033dad764e642326d65fc75f5db8d5a1d685462" Jul 10 17:32:53.944145 ip-10-0-136-7 kubenswrapper[2578]: E0710 17:32:53.944121 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0da392d31a228a19bc373e89f033dad764e642326d65fc75f5db8d5a1d685462\": container with ID starting with 0da392d31a228a19bc373e89f033dad764e642326d65fc75f5db8d5a1d685462 not found: ID does not exist" containerID="0da392d31a228a19bc373e89f033dad764e642326d65fc75f5db8d5a1d685462" Jul 10 17:32:53.944211 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:53.944154 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0da392d31a228a19bc373e89f033dad764e642326d65fc75f5db8d5a1d685462"} err="failed to get container status \"0da392d31a228a19bc373e89f033dad764e642326d65fc75f5db8d5a1d685462\": rpc error: code = NotFound desc = could not find container \"0da392d31a228a19bc373e89f033dad764e642326d65fc75f5db8d5a1d685462\": container with ID starting with 0da392d31a228a19bc373e89f033dad764e642326d65fc75f5db8d5a1d685462 not found: ID does not exist" Jul 10 17:32:53.955105 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:53.955074 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-2482f8/auth-test-model-2482f8-kserve-7d768b9459-nwfrs"] Jul 10 17:32:53.957216 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:53.957195 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-2482f8/auth-test-model-2482f8-kserve-7d768b9459-nwfrs"] Jul 10 17:32:54.356845 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:54.356786 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-43091a/sub-test-model-43091a-kserve-77655999f5-xnwgj" Jul 10 17:32:54.357322 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:54.356936 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-43091a/sub-test-model-43091a-kserve-77655999f5-xnwgj" Jul 10 17:32:54.369187 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:54.369155 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-43091a/sub-test-model-43091a-kserve-77655999f5-xnwgj" Jul 10 17:32:54.425153 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:54.425124 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c2705ea-824e-4f04-875b-972bd2f595d9" path="/var/lib/kubelet/pods/8c2705ea-824e-4f04-875b-972bd2f595d9/volumes" Jul 10 17:32:54.951601 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:32:54.951573 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-43091a/sub-test-model-43091a-kserve-77655999f5-xnwgj" Jul 10 17:33:51.350685 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:33:51.350645 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-43091a/sub-test-model-43091a-kserve-77655999f5-xnwgj"] Jul 10 17:34:00.929452 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:00.929421 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-43091a/rate-test-model-43091a-kserve-6f88876547-mwbms"] Jul 10 17:34:00.929909 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:00.929753 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c2705ea-824e-4f04-875b-972bd2f595d9" containerName="main" Jul 10 17:34:00.929909 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:00.929764 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c2705ea-824e-4f04-875b-972bd2f595d9" containerName="main" Jul 10 17:34:00.929909 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:00.929812 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c2705ea-824e-4f04-875b-972bd2f595d9" containerName="main" Jul 10 17:34:00.932668 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:00.932642 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-43091a/rate-test-model-43091a-kserve-6f88876547-mwbms" Jul 10 17:34:00.935242 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:00.935215 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-43091a\"/\"rate-test-model-43091a-kserve-self-signed-certs\"" Jul 10 17:34:00.942418 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:00.942398 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-43091a/rate-test-model-43091a-kserve-6f88876547-mwbms"] Jul 10 17:34:01.100969 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:01.100929 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b831e616-8574-46b4-a2e2-08df475cc4fe-tmp-dir\") pod \"rate-test-model-43091a-kserve-6f88876547-mwbms\" (UID: \"b831e616-8574-46b4-a2e2-08df475cc4fe\") " pod="ai-tenant-e2e-shared-a-43091a/rate-test-model-43091a-kserve-6f88876547-mwbms" Jul 10 17:34:01.101192 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:01.100992 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b831e616-8574-46b4-a2e2-08df475cc4fe-dshm\") pod \"rate-test-model-43091a-kserve-6f88876547-mwbms\" (UID: \"b831e616-8574-46b4-a2e2-08df475cc4fe\") " pod="ai-tenant-e2e-shared-a-43091a/rate-test-model-43091a-kserve-6f88876547-mwbms" Jul 10 17:34:01.101192 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:01.101097 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b831e616-8574-46b4-a2e2-08df475cc4fe-home\") pod \"rate-test-model-43091a-kserve-6f88876547-mwbms\" (UID: \"b831e616-8574-46b4-a2e2-08df475cc4fe\") " pod="ai-tenant-e2e-shared-a-43091a/rate-test-model-43091a-kserve-6f88876547-mwbms" Jul 10 17:34:01.101192 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:01.101134 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b831e616-8574-46b4-a2e2-08df475cc4fe-tls-certs\") pod \"rate-test-model-43091a-kserve-6f88876547-mwbms\" (UID: \"b831e616-8574-46b4-a2e2-08df475cc4fe\") " pod="ai-tenant-e2e-shared-a-43091a/rate-test-model-43091a-kserve-6f88876547-mwbms" Jul 10 17:34:01.101340 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:01.101217 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b831e616-8574-46b4-a2e2-08df475cc4fe-model-cache\") pod \"rate-test-model-43091a-kserve-6f88876547-mwbms\" (UID: \"b831e616-8574-46b4-a2e2-08df475cc4fe\") " pod="ai-tenant-e2e-shared-a-43091a/rate-test-model-43091a-kserve-6f88876547-mwbms" Jul 10 17:34:01.101340 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:01.101238 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7ptn\" (UniqueName: \"kubernetes.io/projected/b831e616-8574-46b4-a2e2-08df475cc4fe-kube-api-access-n7ptn\") pod \"rate-test-model-43091a-kserve-6f88876547-mwbms\" (UID: \"b831e616-8574-46b4-a2e2-08df475cc4fe\") " pod="ai-tenant-e2e-shared-a-43091a/rate-test-model-43091a-kserve-6f88876547-mwbms" Jul 10 17:34:01.202509 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:01.202419 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b831e616-8574-46b4-a2e2-08df475cc4fe-tls-certs\") pod \"rate-test-model-43091a-kserve-6f88876547-mwbms\" (UID: \"b831e616-8574-46b4-a2e2-08df475cc4fe\") " pod="ai-tenant-e2e-shared-a-43091a/rate-test-model-43091a-kserve-6f88876547-mwbms" Jul 10 17:34:01.202509 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:01.202489 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b831e616-8574-46b4-a2e2-08df475cc4fe-model-cache\") pod \"rate-test-model-43091a-kserve-6f88876547-mwbms\" (UID: \"b831e616-8574-46b4-a2e2-08df475cc4fe\") " pod="ai-tenant-e2e-shared-a-43091a/rate-test-model-43091a-kserve-6f88876547-mwbms" Jul 10 17:34:01.202748 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:01.202523 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7ptn\" (UniqueName: \"kubernetes.io/projected/b831e616-8574-46b4-a2e2-08df475cc4fe-kube-api-access-n7ptn\") pod \"rate-test-model-43091a-kserve-6f88876547-mwbms\" (UID: \"b831e616-8574-46b4-a2e2-08df475cc4fe\") " pod="ai-tenant-e2e-shared-a-43091a/rate-test-model-43091a-kserve-6f88876547-mwbms" Jul 10 17:34:01.202748 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:01.202572 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b831e616-8574-46b4-a2e2-08df475cc4fe-tmp-dir\") pod \"rate-test-model-43091a-kserve-6f88876547-mwbms\" (UID: \"b831e616-8574-46b4-a2e2-08df475cc4fe\") " pod="ai-tenant-e2e-shared-a-43091a/rate-test-model-43091a-kserve-6f88876547-mwbms" Jul 10 17:34:01.202748 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:01.202621 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b831e616-8574-46b4-a2e2-08df475cc4fe-dshm\") pod \"rate-test-model-43091a-kserve-6f88876547-mwbms\" (UID: \"b831e616-8574-46b4-a2e2-08df475cc4fe\") " pod="ai-tenant-e2e-shared-a-43091a/rate-test-model-43091a-kserve-6f88876547-mwbms" Jul 10 17:34:01.202748 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:01.202655 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b831e616-8574-46b4-a2e2-08df475cc4fe-home\") pod \"rate-test-model-43091a-kserve-6f88876547-mwbms\" (UID: \"b831e616-8574-46b4-a2e2-08df475cc4fe\") " pod="ai-tenant-e2e-shared-a-43091a/rate-test-model-43091a-kserve-6f88876547-mwbms" Jul 10 17:34:01.202963 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:01.202905 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b831e616-8574-46b4-a2e2-08df475cc4fe-model-cache\") pod \"rate-test-model-43091a-kserve-6f88876547-mwbms\" (UID: \"b831e616-8574-46b4-a2e2-08df475cc4fe\") " pod="ai-tenant-e2e-shared-a-43091a/rate-test-model-43091a-kserve-6f88876547-mwbms" Jul 10 17:34:01.202963 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:01.202908 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b831e616-8574-46b4-a2e2-08df475cc4fe-tmp-dir\") pod \"rate-test-model-43091a-kserve-6f88876547-mwbms\" (UID: \"b831e616-8574-46b4-a2e2-08df475cc4fe\") " pod="ai-tenant-e2e-shared-a-43091a/rate-test-model-43091a-kserve-6f88876547-mwbms" Jul 10 17:34:01.203081 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:01.202965 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b831e616-8574-46b4-a2e2-08df475cc4fe-home\") pod \"rate-test-model-43091a-kserve-6f88876547-mwbms\" (UID: \"b831e616-8574-46b4-a2e2-08df475cc4fe\") " pod="ai-tenant-e2e-shared-a-43091a/rate-test-model-43091a-kserve-6f88876547-mwbms" Jul 10 17:34:01.205001 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:01.204972 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b831e616-8574-46b4-a2e2-08df475cc4fe-dshm\") pod \"rate-test-model-43091a-kserve-6f88876547-mwbms\" (UID: \"b831e616-8574-46b4-a2e2-08df475cc4fe\") " pod="ai-tenant-e2e-shared-a-43091a/rate-test-model-43091a-kserve-6f88876547-mwbms" Jul 10 17:34:01.205189 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:01.205171 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b831e616-8574-46b4-a2e2-08df475cc4fe-tls-certs\") pod \"rate-test-model-43091a-kserve-6f88876547-mwbms\" (UID: \"b831e616-8574-46b4-a2e2-08df475cc4fe\") " pod="ai-tenant-e2e-shared-a-43091a/rate-test-model-43091a-kserve-6f88876547-mwbms" Jul 10 17:34:01.211342 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:01.211318 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7ptn\" (UniqueName: \"kubernetes.io/projected/b831e616-8574-46b4-a2e2-08df475cc4fe-kube-api-access-n7ptn\") pod \"rate-test-model-43091a-kserve-6f88876547-mwbms\" (UID: \"b831e616-8574-46b4-a2e2-08df475cc4fe\") " pod="ai-tenant-e2e-shared-a-43091a/rate-test-model-43091a-kserve-6f88876547-mwbms" Jul 10 17:34:01.242392 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:01.242366 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-43091a/rate-test-model-43091a-kserve-6f88876547-mwbms" Jul 10 17:34:01.377839 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:01.377814 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-43091a/rate-test-model-43091a-kserve-6f88876547-mwbms"] Jul 10 17:34:01.379631 ip-10-0-136-7 kubenswrapper[2578]: W0710 17:34:01.379606 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb831e616_8574_46b4_a2e2_08df475cc4fe.slice/crio-e2bf4946da9e15f05ab2052e94a630b57c02f11fea72fd9a21d9a0a4cc3140aa WatchSource:0}: Error finding container e2bf4946da9e15f05ab2052e94a630b57c02f11fea72fd9a21d9a0a4cc3140aa: Status 404 returned error can't find the container with id e2bf4946da9e15f05ab2052e94a630b57c02f11fea72fd9a21d9a0a4cc3140aa Jul 10 17:34:02.240736 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:02.240697 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-43091a/rate-test-model-43091a-kserve-6f88876547-mwbms" event={"ID":"b831e616-8574-46b4-a2e2-08df475cc4fe","Type":"ContainerStarted","Data":"e3c832c9b9d2922c71f45676ece23934efb1919d4602a2dfd9c40c773cfb9e30"} Jul 10 17:34:02.241123 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:02.240749 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-43091a/rate-test-model-43091a-kserve-6f88876547-mwbms" event={"ID":"b831e616-8574-46b4-a2e2-08df475cc4fe","Type":"ContainerStarted","Data":"e2bf4946da9e15f05ab2052e94a630b57c02f11fea72fd9a21d9a0a4cc3140aa"} Jul 10 17:34:02.257572 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:02.257515 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-43091a/rate-test-model-43091a-kserve-6f88876547-mwbms" podStartSLOduration=2.257499007 podStartE2EDuration="2.257499007s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 17:34:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:34:02.256411804 +0000 UTC m=+2386.345740855" watchObservedRunningTime="2026-07-10 17:34:02.257499007 +0000 UTC m=+2386.346828056" Jul 10 17:34:06.371612 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:06.371563 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-43091a/sub-test-model-43091a-kserve-77655999f5-xnwgj" podUID="455fc0dc-c5fa-4014-9f96-d5edda5bbba2" containerName="main" containerID="cri-o://58618b8a62a68229dc5cb2c61779132861acaa407be4a23b34b35810f493fbd0" gracePeriod=45 Jul 10 17:34:06.628506 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:06.628443 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-43091a/sub-test-model-43091a-kserve-77655999f5-xnwgj" Jul 10 17:34:06.648979 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:06.648950 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j2s2q\" (UniqueName: \"kubernetes.io/projected/455fc0dc-c5fa-4014-9f96-d5edda5bbba2-kube-api-access-j2s2q\") pod \"455fc0dc-c5fa-4014-9f96-d5edda5bbba2\" (UID: \"455fc0dc-c5fa-4014-9f96-d5edda5bbba2\") " Jul 10 17:34:06.648979 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:06.648984 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/455fc0dc-c5fa-4014-9f96-d5edda5bbba2-model-cache\") pod \"455fc0dc-c5fa-4014-9f96-d5edda5bbba2\" (UID: \"455fc0dc-c5fa-4014-9f96-d5edda5bbba2\") " Jul 10 17:34:06.649308 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:06.649078 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/455fc0dc-c5fa-4014-9f96-d5edda5bbba2-dshm\") pod \"455fc0dc-c5fa-4014-9f96-d5edda5bbba2\" (UID: \"455fc0dc-c5fa-4014-9f96-d5edda5bbba2\") " Jul 10 17:34:06.649308 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:06.649106 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/455fc0dc-c5fa-4014-9f96-d5edda5bbba2-tmp-dir\") pod \"455fc0dc-c5fa-4014-9f96-d5edda5bbba2\" (UID: \"455fc0dc-c5fa-4014-9f96-d5edda5bbba2\") " Jul 10 17:34:06.649308 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:06.649127 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/455fc0dc-c5fa-4014-9f96-d5edda5bbba2-tls-certs\") pod \"455fc0dc-c5fa-4014-9f96-d5edda5bbba2\" (UID: \"455fc0dc-c5fa-4014-9f96-d5edda5bbba2\") " Jul 10 17:34:06.649308 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:06.649156 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/455fc0dc-c5fa-4014-9f96-d5edda5bbba2-home\") pod \"455fc0dc-c5fa-4014-9f96-d5edda5bbba2\" (UID: \"455fc0dc-c5fa-4014-9f96-d5edda5bbba2\") " Jul 10 17:34:06.649574 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:06.649388 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/455fc0dc-c5fa-4014-9f96-d5edda5bbba2-model-cache" (OuterVolumeSpecName: "model-cache") pod "455fc0dc-c5fa-4014-9f96-d5edda5bbba2" (UID: "455fc0dc-c5fa-4014-9f96-d5edda5bbba2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 17:34:06.649636 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:06.649605 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/455fc0dc-c5fa-4014-9f96-d5edda5bbba2-home" (OuterVolumeSpecName: "home") pod "455fc0dc-c5fa-4014-9f96-d5edda5bbba2" (UID: "455fc0dc-c5fa-4014-9f96-d5edda5bbba2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 17:34:06.649703 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:06.649682 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/455fc0dc-c5fa-4014-9f96-d5edda5bbba2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "455fc0dc-c5fa-4014-9f96-d5edda5bbba2" (UID: "455fc0dc-c5fa-4014-9f96-d5edda5bbba2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 17:34:06.651541 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:06.651517 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/455fc0dc-c5fa-4014-9f96-d5edda5bbba2-kube-api-access-j2s2q" (OuterVolumeSpecName: "kube-api-access-j2s2q") pod "455fc0dc-c5fa-4014-9f96-d5edda5bbba2" (UID: "455fc0dc-c5fa-4014-9f96-d5edda5bbba2"). InnerVolumeSpecName "kube-api-access-j2s2q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 17:34:06.651761 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:06.651742 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/455fc0dc-c5fa-4014-9f96-d5edda5bbba2-dshm" (OuterVolumeSpecName: "dshm") pod "455fc0dc-c5fa-4014-9f96-d5edda5bbba2" (UID: "455fc0dc-c5fa-4014-9f96-d5edda5bbba2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 17:34:06.651809 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:06.651741 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/455fc0dc-c5fa-4014-9f96-d5edda5bbba2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "455fc0dc-c5fa-4014-9f96-d5edda5bbba2" (UID: "455fc0dc-c5fa-4014-9f96-d5edda5bbba2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 17:34:06.750115 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:06.750084 2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/455fc0dc-c5fa-4014-9f96-d5edda5bbba2-dshm\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 17:34:06.750293 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:06.750133 2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/455fc0dc-c5fa-4014-9f96-d5edda5bbba2-tmp-dir\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 17:34:06.750293 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:06.750144 2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/455fc0dc-c5fa-4014-9f96-d5edda5bbba2-tls-certs\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 17:34:06.750293 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:06.750151 2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/455fc0dc-c5fa-4014-9f96-d5edda5bbba2-home\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 17:34:06.750293 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:06.750160 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j2s2q\" (UniqueName: \"kubernetes.io/projected/455fc0dc-c5fa-4014-9f96-d5edda5bbba2-kube-api-access-j2s2q\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 17:34:06.750293 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:06.750168 2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/455fc0dc-c5fa-4014-9f96-d5edda5bbba2-model-cache\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 17:34:07.264291 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:07.264254 2578 generic.go:356] "Generic (PLEG): container finished" podID="455fc0dc-c5fa-4014-9f96-d5edda5bbba2" containerID="58618b8a62a68229dc5cb2c61779132861acaa407be4a23b34b35810f493fbd0" exitCode=0 Jul 10 17:34:07.264470 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:07.264300 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-43091a/sub-test-model-43091a-kserve-77655999f5-xnwgj" Jul 10 17:34:07.264470 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:07.264331 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-43091a/sub-test-model-43091a-kserve-77655999f5-xnwgj" event={"ID":"455fc0dc-c5fa-4014-9f96-d5edda5bbba2","Type":"ContainerDied","Data":"58618b8a62a68229dc5cb2c61779132861acaa407be4a23b34b35810f493fbd0"} Jul 10 17:34:07.264470 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:07.264377 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-43091a/sub-test-model-43091a-kserve-77655999f5-xnwgj" event={"ID":"455fc0dc-c5fa-4014-9f96-d5edda5bbba2","Type":"ContainerDied","Data":"c404a1e8501baf4190485d49055e2c1e7dd7656e6fe9d6f0b313889fc21e494c"} Jul 10 17:34:07.264470 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:07.264397 2578 scope.go:117] "RemoveContainer" containerID="58618b8a62a68229dc5cb2c61779132861acaa407be4a23b34b35810f493fbd0" Jul 10 17:34:07.273345 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:07.273329 2578 scope.go:117] "RemoveContainer" containerID="58618b8a62a68229dc5cb2c61779132861acaa407be4a23b34b35810f493fbd0" Jul 10 17:34:07.273586 ip-10-0-136-7 kubenswrapper[2578]: E0710 17:34:07.273567 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58618b8a62a68229dc5cb2c61779132861acaa407be4a23b34b35810f493fbd0\": container with ID starting with 58618b8a62a68229dc5cb2c61779132861acaa407be4a23b34b35810f493fbd0 not found: ID does not exist" containerID="58618b8a62a68229dc5cb2c61779132861acaa407be4a23b34b35810f493fbd0" Jul 10 17:34:07.273631 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:07.273596 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58618b8a62a68229dc5cb2c61779132861acaa407be4a23b34b35810f493fbd0"} err="failed to get container status \"58618b8a62a68229dc5cb2c61779132861acaa407be4a23b34b35810f493fbd0\": rpc error: code = NotFound desc = could not find container \"58618b8a62a68229dc5cb2c61779132861acaa407be4a23b34b35810f493fbd0\": container with ID starting with 58618b8a62a68229dc5cb2c61779132861acaa407be4a23b34b35810f493fbd0 not found: ID does not exist" Jul 10 17:34:07.284816 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:07.284793 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-43091a/sub-test-model-43091a-kserve-77655999f5-xnwgj"] Jul 10 17:34:07.286977 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:07.286955 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-43091a/sub-test-model-43091a-kserve-77655999f5-xnwgj"] Jul 10 17:34:08.425428 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:08.425393 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="455fc0dc-c5fa-4014-9f96-d5edda5bbba2" path="/var/lib/kubelet/pods/455fc0dc-c5fa-4014-9f96-d5edda5bbba2/volumes" Jul 10 17:34:11.242869 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:11.242830 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-43091a/rate-test-model-43091a-kserve-6f88876547-mwbms" Jul 10 17:34:11.242869 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:11.242880 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-43091a/rate-test-model-43091a-kserve-6f88876547-mwbms" Jul 10 17:34:11.255318 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:11.255286 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-43091a/rate-test-model-43091a-kserve-6f88876547-mwbms" Jul 10 17:34:11.298501 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:11.298476 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-43091a/rate-test-model-43091a-kserve-6f88876547-mwbms" Jul 10 17:34:16.506230 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:16.506112 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-5j4ws_27742f15-b602-4554-84dc-ff9d46a9adda/console-operator/2.log" Jul 10 17:34:16.520488 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:16.510490 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-5j4ws_27742f15-b602-4554-84dc-ff9d46a9adda/console-operator/2.log" Jul 10 17:34:16.520488 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:16.512743 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gqgkg_b9a53126-f278-4a9d-9735-5f73dd0c1964/ovn-acl-logging/0.log" Jul 10 17:34:16.520488 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:34:16.517120 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gqgkg_b9a53126-f278-4a9d-9735-5f73dd0c1964/ovn-acl-logging/0.log" Jul 10 17:35:41.590265 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:41.590231 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-43091a/rate-test-model-43091a-kserve-6f88876547-mwbms"] Jul 10 17:35:47.071112 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:47.071074 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7463-140489726480048"] Jul 10 17:35:47.071561 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:47.071412 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="455fc0dc-c5fa-4014-9f96-d5edda5bbba2" containerName="main" Jul 10 17:35:47.071561 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:47.071424 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="455fc0dc-c5fa-4014-9f96-d5edda5bbba2" containerName="main" Jul 10 17:35:47.071561 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:47.071511 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="455fc0dc-c5fa-4014-9f96-d5edda5bbba2" containerName="main" Jul 10 17:35:47.074447 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:47.074429 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7463-140489726480048" Jul 10 17:35:47.077439 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:47.077408 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-fg4vv\"" Jul 10 17:35:47.078762 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:47.078739 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7463-140489726480048"] Jul 10 17:35:47.148639 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:47.148607 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkcck\" (UniqueName: \"kubernetes.io/projected/d42f3ab9-2f31-41fb-b2d8-64ff9ba4bc75-kube-api-access-vkcck\") pod \"test-curl-7463-140489726480048\" (UID: \"d42f3ab9-2f31-41fb-b2d8-64ff9ba4bc75\") " pod="openshift-ingress/test-curl-7463-140489726480048" Jul 10 17:35:47.249121 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:47.249038 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkcck\" (UniqueName: \"kubernetes.io/projected/d42f3ab9-2f31-41fb-b2d8-64ff9ba4bc75-kube-api-access-vkcck\") pod \"test-curl-7463-140489726480048\" (UID: \"d42f3ab9-2f31-41fb-b2d8-64ff9ba4bc75\") " pod="openshift-ingress/test-curl-7463-140489726480048" Jul 10 17:35:47.257664 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:47.257630 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkcck\" (UniqueName: \"kubernetes.io/projected/d42f3ab9-2f31-41fb-b2d8-64ff9ba4bc75-kube-api-access-vkcck\") pod \"test-curl-7463-140489726480048\" (UID: \"d42f3ab9-2f31-41fb-b2d8-64ff9ba4bc75\") " pod="openshift-ingress/test-curl-7463-140489726480048" Jul 10 17:35:47.385391 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:47.385294 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7463-140489726480048" Jul 10 17:35:47.525735 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:47.525676 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7463-140489726480048"] Jul 10 17:35:47.528098 ip-10-0-136-7 kubenswrapper[2578]: W0710 17:35:47.528066 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd42f3ab9_2f31_41fb_b2d8_64ff9ba4bc75.slice/crio-1a53c0430c4853d0597d297b7f162f0fb78d0b5bc80aad7995ad621f351cd303 WatchSource:0}: Error finding container 1a53c0430c4853d0597d297b7f162f0fb78d0b5bc80aad7995ad621f351cd303: Status 404 returned error can't find the container with id 1a53c0430c4853d0597d297b7f162f0fb78d0b5bc80aad7995ad621f351cd303 Jul 10 17:35:47.529576 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:47.529555 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 17:35:47.700276 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:47.700190 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7463-140489726480048" event={"ID":"d42f3ab9-2f31-41fb-b2d8-64ff9ba4bc75","Type":"ContainerStarted","Data":"1a53c0430c4853d0597d297b7f162f0fb78d0b5bc80aad7995ad621f351cd303"} Jul 10 17:35:48.706001 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:48.705968 2578 generic.go:356] "Generic (PLEG): container finished" podID="d42f3ab9-2f31-41fb-b2d8-64ff9ba4bc75" containerID="500a51e995e969e1adf449b8b95e228c10192b170b425b53a2ac3c88832ee2af" exitCode=0 Jul 10 17:35:48.706419 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:48.706080 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7463-140489726480048" event={"ID":"d42f3ab9-2f31-41fb-b2d8-64ff9ba4bc75","Type":"ContainerDied","Data":"500a51e995e969e1adf449b8b95e228c10192b170b425b53a2ac3c88832ee2af"} Jul 10 17:35:49.850654 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:49.850618 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7463-140489726480048" Jul 10 17:35:49.878307 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:49.878271 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vkcck\" (UniqueName: \"kubernetes.io/projected/d42f3ab9-2f31-41fb-b2d8-64ff9ba4bc75-kube-api-access-vkcck\") pod \"d42f3ab9-2f31-41fb-b2d8-64ff9ba4bc75\" (UID: \"d42f3ab9-2f31-41fb-b2d8-64ff9ba4bc75\") " Jul 10 17:35:49.880627 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:49.880593 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d42f3ab9-2f31-41fb-b2d8-64ff9ba4bc75-kube-api-access-vkcck" (OuterVolumeSpecName: "kube-api-access-vkcck") pod "d42f3ab9-2f31-41fb-b2d8-64ff9ba4bc75" (UID: "d42f3ab9-2f31-41fb-b2d8-64ff9ba4bc75"). InnerVolumeSpecName "kube-api-access-vkcck". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 17:35:49.885647 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:49.885626 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7463-140489726480048_d42f3ab9-2f31-41fb-b2d8-64ff9ba4bc75/test-curl-7463-140489726480048/0.log" Jul 10 17:35:49.907103 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:49.907069 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7463-140489726480048"] Jul 10 17:35:49.910347 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:49.910319 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7463-140489726480048"] Jul 10 17:35:49.979089 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:49.978975 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vkcck\" (UniqueName: \"kubernetes.io/projected/d42f3ab9-2f31-41fb-b2d8-64ff9ba4bc75-kube-api-access-vkcck\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 17:35:50.425257 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:50.425223 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d42f3ab9-2f31-41fb-b2d8-64ff9ba4bc75" path="/var/lib/kubelet/pods/d42f3ab9-2f31-41fb-b2d8-64ff9ba4bc75/volumes" Jul 10 17:35:50.716216 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:50.716128 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7463-140489726480048" Jul 10 17:35:50.716216 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:50.716154 2578 scope.go:117] "RemoveContainer" containerID="500a51e995e969e1adf449b8b95e228c10192b170b425b53a2ac3c88832ee2af" Jul 10 17:35:53.098397 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:53.098360 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7463-140489726516224"] Jul 10 17:35:53.098775 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:53.098690 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d42f3ab9-2f31-41fb-b2d8-64ff9ba4bc75" containerName="test-curl-7463-140489726480048" Jul 10 17:35:53.098775 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:53.098702 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d42f3ab9-2f31-41fb-b2d8-64ff9ba4bc75" containerName="test-curl-7463-140489726480048" Jul 10 17:35:53.098775 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:53.098766 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d42f3ab9-2f31-41fb-b2d8-64ff9ba4bc75" containerName="test-curl-7463-140489726480048" Jul 10 17:35:53.101687 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:53.101670 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7463-140489726516224" Jul 10 17:35:53.104417 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:53.104391 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-fg4vv\"" Jul 10 17:35:53.105707 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:53.105686 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7463-140489726516224"] Jul 10 17:35:53.209073 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:53.209012 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mgk7\" (UniqueName: \"kubernetes.io/projected/f95a9359-5a07-4d38-a743-cb3034bbb236-kube-api-access-9mgk7\") pod \"test-curl-7463-140489726516224\" (UID: \"f95a9359-5a07-4d38-a743-cb3034bbb236\") " pod="openshift-ingress/test-curl-7463-140489726516224" Jul 10 17:35:53.309844 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:53.309810 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mgk7\" (UniqueName: \"kubernetes.io/projected/f95a9359-5a07-4d38-a743-cb3034bbb236-kube-api-access-9mgk7\") pod \"test-curl-7463-140489726516224\" (UID: \"f95a9359-5a07-4d38-a743-cb3034bbb236\") " pod="openshift-ingress/test-curl-7463-140489726516224" Jul 10 17:35:53.318171 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:53.318140 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mgk7\" (UniqueName: \"kubernetes.io/projected/f95a9359-5a07-4d38-a743-cb3034bbb236-kube-api-access-9mgk7\") pod \"test-curl-7463-140489726516224\" (UID: \"f95a9359-5a07-4d38-a743-cb3034bbb236\") " pod="openshift-ingress/test-curl-7463-140489726516224" Jul 10 17:35:53.411026 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:53.410935 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7463-140489726516224" Jul 10 17:35:53.547979 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:53.547954 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7463-140489726516224"] Jul 10 17:35:53.550500 ip-10-0-136-7 kubenswrapper[2578]: W0710 17:35:53.550468 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf95a9359_5a07_4d38_a743_cb3034bbb236.slice/crio-0b01235663b193a15b49f19f225c4db3c1616909806187afd5be951cad6858fb WatchSource:0}: Error finding container 0b01235663b193a15b49f19f225c4db3c1616909806187afd5be951cad6858fb: Status 404 returned error can't find the container with id 0b01235663b193a15b49f19f225c4db3c1616909806187afd5be951cad6858fb Jul 10 17:35:53.730741 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:53.730648 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7463-140489726516224" event={"ID":"f95a9359-5a07-4d38-a743-cb3034bbb236","Type":"ContainerStarted","Data":"0b01235663b193a15b49f19f225c4db3c1616909806187afd5be951cad6858fb"} Jul 10 17:35:54.736885 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:54.736793 2578 generic.go:356] "Generic (PLEG): container finished" podID="f95a9359-5a07-4d38-a743-cb3034bbb236" containerID="ecdf83b5598df5ccb705512d9e8e8b0ac645dca608cdfc333b8295ef8f3bf0b0" exitCode=0 Jul 10 17:35:54.737299 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:54.736884 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7463-140489726516224" event={"ID":"f95a9359-5a07-4d38-a743-cb3034bbb236","Type":"ContainerDied","Data":"ecdf83b5598df5ccb705512d9e8e8b0ac645dca608cdfc333b8295ef8f3bf0b0"} Jul 10 17:35:55.884232 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:55.884209 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7463-140489726516224" Jul 10 17:35:55.903202 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:55.903178 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7463-140489726516224_f95a9359-5a07-4d38-a743-cb3034bbb236/test-curl-7463-140489726516224/0.log" Jul 10 17:35:55.925254 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:55.925203 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7463-140489726516224"] Jul 10 17:35:55.926797 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:55.926771 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7463-140489726516224"] Jul 10 17:35:55.934529 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:55.934509 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9mgk7\" (UniqueName: \"kubernetes.io/projected/f95a9359-5a07-4d38-a743-cb3034bbb236-kube-api-access-9mgk7\") pod \"f95a9359-5a07-4d38-a743-cb3034bbb236\" (UID: \"f95a9359-5a07-4d38-a743-cb3034bbb236\") " Jul 10 17:35:55.936702 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:55.936673 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f95a9359-5a07-4d38-a743-cb3034bbb236-kube-api-access-9mgk7" (OuterVolumeSpecName: "kube-api-access-9mgk7") pod "f95a9359-5a07-4d38-a743-cb3034bbb236" (UID: "f95a9359-5a07-4d38-a743-cb3034bbb236"). InnerVolumeSpecName "kube-api-access-9mgk7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 17:35:56.035447 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:56.035410 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9mgk7\" (UniqueName: \"kubernetes.io/projected/f95a9359-5a07-4d38-a743-cb3034bbb236-kube-api-access-9mgk7\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 17:35:56.430214 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:56.430133 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f95a9359-5a07-4d38-a743-cb3034bbb236" path="/var/lib/kubelet/pods/f95a9359-5a07-4d38-a743-cb3034bbb236/volumes" Jul 10 17:35:56.612592 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:56.612555 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-43091a/rate-test-model-43091a-kserve-6f88876547-mwbms" podUID="b831e616-8574-46b4-a2e2-08df475cc4fe" containerName="main" containerID="cri-o://e3c832c9b9d2922c71f45676ece23934efb1919d4602a2dfd9c40c773cfb9e30" gracePeriod=45 Jul 10 17:35:56.746781 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:56.746752 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7463-140489726516224" Jul 10 17:35:56.746967 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:56.746809 2578 scope.go:117] "RemoveContainer" containerID="ecdf83b5598df5ccb705512d9e8e8b0ac645dca608cdfc333b8295ef8f3bf0b0" Jul 10 17:35:56.748587 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:56.748567 2578 generic.go:356] "Generic (PLEG): container finished" podID="b831e616-8574-46b4-a2e2-08df475cc4fe" containerID="e3c832c9b9d2922c71f45676ece23934efb1919d4602a2dfd9c40c773cfb9e30" exitCode=0 Jul 10 17:35:56.748755 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:56.748611 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-43091a/rate-test-model-43091a-kserve-6f88876547-mwbms" event={"ID":"b831e616-8574-46b4-a2e2-08df475cc4fe","Type":"ContainerDied","Data":"e3c832c9b9d2922c71f45676ece23934efb1919d4602a2dfd9c40c773cfb9e30"} Jul 10 17:35:56.859982 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:56.859957 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-43091a/rate-test-model-43091a-kserve-6f88876547-mwbms" Jul 10 17:35:56.942588 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:56.942557 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b831e616-8574-46b4-a2e2-08df475cc4fe-tmp-dir\") pod \"b831e616-8574-46b4-a2e2-08df475cc4fe\" (UID: \"b831e616-8574-46b4-a2e2-08df475cc4fe\") " Jul 10 17:35:56.943043 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:56.942598 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b831e616-8574-46b4-a2e2-08df475cc4fe-tls-certs\") pod \"b831e616-8574-46b4-a2e2-08df475cc4fe\" (UID: \"b831e616-8574-46b4-a2e2-08df475cc4fe\") " Jul 10 17:35:56.943043 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:56.942644 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n7ptn\" (UniqueName: \"kubernetes.io/projected/b831e616-8574-46b4-a2e2-08df475cc4fe-kube-api-access-n7ptn\") pod \"b831e616-8574-46b4-a2e2-08df475cc4fe\" (UID: \"b831e616-8574-46b4-a2e2-08df475cc4fe\") " Jul 10 17:35:56.943043 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:56.942672 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b831e616-8574-46b4-a2e2-08df475cc4fe-home\") pod \"b831e616-8574-46b4-a2e2-08df475cc4fe\" (UID: \"b831e616-8574-46b4-a2e2-08df475cc4fe\") " Jul 10 17:35:56.943043 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:56.942691 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b831e616-8574-46b4-a2e2-08df475cc4fe-model-cache\") pod \"b831e616-8574-46b4-a2e2-08df475cc4fe\" (UID: \"b831e616-8574-46b4-a2e2-08df475cc4fe\") " Jul 10 17:35:56.943043 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:56.942751 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b831e616-8574-46b4-a2e2-08df475cc4fe-dshm\") pod \"b831e616-8574-46b4-a2e2-08df475cc4fe\" (UID: \"b831e616-8574-46b4-a2e2-08df475cc4fe\") " Jul 10 17:35:56.943043 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:56.942889 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b831e616-8574-46b4-a2e2-08df475cc4fe-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b831e616-8574-46b4-a2e2-08df475cc4fe" (UID: "b831e616-8574-46b4-a2e2-08df475cc4fe"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 17:35:56.943043 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:56.942984 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b831e616-8574-46b4-a2e2-08df475cc4fe-home" (OuterVolumeSpecName: "home") pod "b831e616-8574-46b4-a2e2-08df475cc4fe" (UID: "b831e616-8574-46b4-a2e2-08df475cc4fe"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 17:35:56.943043 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:56.943038 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b831e616-8574-46b4-a2e2-08df475cc4fe-model-cache" (OuterVolumeSpecName: "model-cache") pod "b831e616-8574-46b4-a2e2-08df475cc4fe" (UID: "b831e616-8574-46b4-a2e2-08df475cc4fe"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 17:35:56.945024 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:56.944999 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b831e616-8574-46b4-a2e2-08df475cc4fe-dshm" (OuterVolumeSpecName: "dshm") pod "b831e616-8574-46b4-a2e2-08df475cc4fe" (UID: "b831e616-8574-46b4-a2e2-08df475cc4fe"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 17:35:56.945024 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:56.945001 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b831e616-8574-46b4-a2e2-08df475cc4fe-kube-api-access-n7ptn" (OuterVolumeSpecName: "kube-api-access-n7ptn") pod "b831e616-8574-46b4-a2e2-08df475cc4fe" (UID: "b831e616-8574-46b4-a2e2-08df475cc4fe"). InnerVolumeSpecName "kube-api-access-n7ptn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 17:35:56.945195 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:56.945028 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b831e616-8574-46b4-a2e2-08df475cc4fe-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b831e616-8574-46b4-a2e2-08df475cc4fe" (UID: "b831e616-8574-46b4-a2e2-08df475cc4fe"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 17:35:57.043720 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:57.043688 2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b831e616-8574-46b4-a2e2-08df475cc4fe-dshm\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 17:35:57.043880 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:57.043738 2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b831e616-8574-46b4-a2e2-08df475cc4fe-tmp-dir\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 17:35:57.043880 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:57.043748 2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b831e616-8574-46b4-a2e2-08df475cc4fe-tls-certs\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 17:35:57.043880 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:57.043757 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n7ptn\" (UniqueName: \"kubernetes.io/projected/b831e616-8574-46b4-a2e2-08df475cc4fe-kube-api-access-n7ptn\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 17:35:57.043880 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:57.043767 2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b831e616-8574-46b4-a2e2-08df475cc4fe-home\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 17:35:57.043880 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:57.043775 2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b831e616-8574-46b4-a2e2-08df475cc4fe-model-cache\") on node \"ip-10-0-136-7.ec2.internal\" DevicePath \"\"" Jul 10 17:35:57.754727 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:57.754699 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-43091a/rate-test-model-43091a-kserve-6f88876547-mwbms" Jul 10 17:35:57.754727 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:57.754713 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-43091a/rate-test-model-43091a-kserve-6f88876547-mwbms" event={"ID":"b831e616-8574-46b4-a2e2-08df475cc4fe","Type":"ContainerDied","Data":"e2bf4946da9e15f05ab2052e94a630b57c02f11fea72fd9a21d9a0a4cc3140aa"} Jul 10 17:35:57.754978 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:57.754763 2578 scope.go:117] "RemoveContainer" containerID="e3c832c9b9d2922c71f45676ece23934efb1919d4602a2dfd9c40c773cfb9e30" Jul 10 17:35:57.775542 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:57.775501 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-43091a/rate-test-model-43091a-kserve-6f88876547-mwbms"] Jul 10 17:35:57.777978 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:57.777952 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-43091a/rate-test-model-43091a-kserve-6f88876547-mwbms"] Jul 10 17:35:58.425595 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:35:58.425560 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b831e616-8574-46b4-a2e2-08df475cc4fe" path="/var/lib/kubelet/pods/b831e616-8574-46b4-a2e2-08df475cc4fe/volumes" Jul 10 17:36:22.297705 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:22.297675 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67764b8b78-tmqk6_fcde3330-8614-47ac-9c0f-93c34a0cfc17/copy-manifests/0.log" Jul 10 17:36:22.310674 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:22.310644 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67764b8b78-tmqk6_fcde3330-8614-47ac-9c0f-93c34a0cfc17/manager/0.log" Jul 10 17:36:25.444789 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:25.444760 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-2tm7x_973dcb52-e499-429c-a73d-471e3cd87ffc/discovery/0.log" Jul 10 17:36:25.682523 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:25.682452 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-558b5454d6-rb5qx_2a999b9a-8edf-4057-afbd-8761b62ae720/kube-auth-proxy/0.log" Jul 10 17:36:26.159803 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:26.159775 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-bb9cb67d8-ddrbp_3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84/router/0.log" Jul 10 17:36:34.035814 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:34.035777 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-dgk2w_a0b23658-26fb-454a-a42e-88432bf12b0a/global-pull-secret-syncer/0.log" Jul 10 17:36:34.227368 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:34.227322 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-t6rhv_08351a06-c5eb-4c51-923c-283f5b28a7d2/konnectivity-agent/0.log" Jul 10 17:36:34.271173 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:34.271139 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-7.ec2.internal_374e81600297ab5c614c91e79771b865/haproxy/0.log" Jul 10 17:36:41.756554 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:41.756528 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-zfbfz_40988796-3d52-4414-89dc-53c322b4d514/cluster-monitoring-operator/0.log" Jul 10 17:36:42.019839 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:42.019812 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-t6dps_b67ee4fa-977b-42e9-b192-6ebcc6471bbe/node-exporter/0.log" Jul 10 17:36:42.047634 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:42.047610 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-t6dps_b67ee4fa-977b-42e9-b192-6ebcc6471bbe/kube-rbac-proxy/0.log" Jul 10 17:36:42.069474 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:42.069451 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-t6dps_b67ee4fa-977b-42e9-b192-6ebcc6471bbe/init-textfile/0.log" Jul 10 17:36:42.272039 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:42.271968 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99/prometheus/0.log" Jul 10 17:36:42.298661 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:42.298635 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99/config-reloader/0.log" Jul 10 17:36:42.322302 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:42.322272 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99/thanos-sidecar/0.log" Jul 10 17:36:42.346103 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:42.346081 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99/kube-rbac-proxy-web/0.log" Jul 10 17:36:42.369135 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:42.369110 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99/kube-rbac-proxy/0.log" Jul 10 17:36:42.391950 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:42.391928 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99/kube-rbac-proxy-thanos/0.log" Jul 10 17:36:42.415854 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:42.415826 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_82c44fc5-e1fa-487c-a4ce-d2ec8ecf4c99/init-config-reloader/0.log" Jul 10 17:36:42.443391 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:42.443360 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-gnmbh_04f0588a-4a06-4776-9ab6-8f60cdd61155/prometheus-operator/0.log" Jul 10 17:36:42.462734 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:42.462711 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-gnmbh_04f0588a-4a06-4776-9ab6-8f60cdd61155/kube-rbac-proxy/0.log" Jul 10 17:36:42.528109 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:42.528017 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-66d47b6679-hxmgv_399a676f-ae4a-42e5-aa7f-ac8a0428131b/telemeter-client/0.log" Jul 10 17:36:42.549307 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:42.549285 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-66d47b6679-hxmgv_399a676f-ae4a-42e5-aa7f-ac8a0428131b/reload/0.log" Jul 10 17:36:42.570683 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:42.570662 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-66d47b6679-hxmgv_399a676f-ae4a-42e5-aa7f-ac8a0428131b/kube-rbac-proxy/0.log" Jul 10 17:36:44.497351 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:44.497302 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-5j4ws_27742f15-b602-4554-84dc-ff9d46a9adda/console-operator/2.log" Jul 10 17:36:44.501769 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:44.501745 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-5j4ws_27742f15-b602-4554-84dc-ff9d46a9adda/console-operator/3.log" Jul 10 17:36:44.684066 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:44.684014 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-dnfk7/perf-node-gather-daemonset-7cjzz"] Jul 10 17:36:44.684389 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:44.684376 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b831e616-8574-46b4-a2e2-08df475cc4fe" containerName="main" Jul 10 17:36:44.684436 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:44.684391 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b831e616-8574-46b4-a2e2-08df475cc4fe" containerName="main" Jul 10 17:36:44.684436 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:44.684411 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f95a9359-5a07-4d38-a743-cb3034bbb236" containerName="test-curl-7463-140489726516224" Jul 10 17:36:44.684436 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:44.684421 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f95a9359-5a07-4d38-a743-cb3034bbb236" containerName="test-curl-7463-140489726516224" Jul 10 17:36:44.684530 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:44.684476 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b831e616-8574-46b4-a2e2-08df475cc4fe" containerName="main" Jul 10 17:36:44.684530 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:44.684486 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="f95a9359-5a07-4d38-a743-cb3034bbb236" containerName="test-curl-7463-140489726516224" Jul 10 17:36:44.686440 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:44.686425 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dnfk7/perf-node-gather-daemonset-7cjzz" Jul 10 17:36:44.689346 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:44.689314 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dnfk7\"/\"kube-root-ca.crt\"" Jul 10 17:36:44.689469 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:44.689389 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dnfk7\"/\"openshift-service-ca.crt\"" Jul 10 17:36:44.690188 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:44.690176 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-dnfk7\"/\"default-dockercfg-gdhhc\"" Jul 10 17:36:44.697715 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:44.697692 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dnfk7/perf-node-gather-daemonset-7cjzz"] Jul 10 17:36:44.760859 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:44.760773 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0c5184ca-a9af-415c-9bcb-29818e49bb3d-sys\") pod \"perf-node-gather-daemonset-7cjzz\" (UID: \"0c5184ca-a9af-415c-9bcb-29818e49bb3d\") " pod="openshift-must-gather-dnfk7/perf-node-gather-daemonset-7cjzz" Jul 10 17:36:44.760859 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:44.760810 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zrpg\" (UniqueName: \"kubernetes.io/projected/0c5184ca-a9af-415c-9bcb-29818e49bb3d-kube-api-access-9zrpg\") pod \"perf-node-gather-daemonset-7cjzz\" (UID: \"0c5184ca-a9af-415c-9bcb-29818e49bb3d\") " pod="openshift-must-gather-dnfk7/perf-node-gather-daemonset-7cjzz" Jul 10 17:36:44.761036 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:44.760904 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0c5184ca-a9af-415c-9bcb-29818e49bb3d-podres\") pod \"perf-node-gather-daemonset-7cjzz\" (UID: \"0c5184ca-a9af-415c-9bcb-29818e49bb3d\") " pod="openshift-must-gather-dnfk7/perf-node-gather-daemonset-7cjzz" Jul 10 17:36:44.761036 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:44.760938 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0c5184ca-a9af-415c-9bcb-29818e49bb3d-proc\") pod \"perf-node-gather-daemonset-7cjzz\" (UID: \"0c5184ca-a9af-415c-9bcb-29818e49bb3d\") " pod="openshift-must-gather-dnfk7/perf-node-gather-daemonset-7cjzz" Jul 10 17:36:44.761036 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:44.760964 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0c5184ca-a9af-415c-9bcb-29818e49bb3d-lib-modules\") pod \"perf-node-gather-daemonset-7cjzz\" (UID: \"0c5184ca-a9af-415c-9bcb-29818e49bb3d\") " pod="openshift-must-gather-dnfk7/perf-node-gather-daemonset-7cjzz" Jul 10 17:36:44.862344 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:44.862306 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0c5184ca-a9af-415c-9bcb-29818e49bb3d-podres\") pod \"perf-node-gather-daemonset-7cjzz\" (UID: \"0c5184ca-a9af-415c-9bcb-29818e49bb3d\") " pod="openshift-must-gather-dnfk7/perf-node-gather-daemonset-7cjzz" Jul 10 17:36:44.862516 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:44.862351 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0c5184ca-a9af-415c-9bcb-29818e49bb3d-proc\") pod \"perf-node-gather-daemonset-7cjzz\" (UID: \"0c5184ca-a9af-415c-9bcb-29818e49bb3d\") " pod="openshift-must-gather-dnfk7/perf-node-gather-daemonset-7cjzz" Jul 10 17:36:44.862516 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:44.862370 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0c5184ca-a9af-415c-9bcb-29818e49bb3d-lib-modules\") pod \"perf-node-gather-daemonset-7cjzz\" (UID: \"0c5184ca-a9af-415c-9bcb-29818e49bb3d\") " pod="openshift-must-gather-dnfk7/perf-node-gather-daemonset-7cjzz" Jul 10 17:36:44.862516 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:44.862410 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0c5184ca-a9af-415c-9bcb-29818e49bb3d-sys\") pod \"perf-node-gather-daemonset-7cjzz\" (UID: \"0c5184ca-a9af-415c-9bcb-29818e49bb3d\") " pod="openshift-must-gather-dnfk7/perf-node-gather-daemonset-7cjzz" Jul 10 17:36:44.862516 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:44.862429 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zrpg\" (UniqueName: \"kubernetes.io/projected/0c5184ca-a9af-415c-9bcb-29818e49bb3d-kube-api-access-9zrpg\") pod \"perf-node-gather-daemonset-7cjzz\" (UID: \"0c5184ca-a9af-415c-9bcb-29818e49bb3d\") " pod="openshift-must-gather-dnfk7/perf-node-gather-daemonset-7cjzz" Jul 10 17:36:44.862516 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:44.862441 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0c5184ca-a9af-415c-9bcb-29818e49bb3d-proc\") pod \"perf-node-gather-daemonset-7cjzz\" (UID: \"0c5184ca-a9af-415c-9bcb-29818e49bb3d\") " pod="openshift-must-gather-dnfk7/perf-node-gather-daemonset-7cjzz" Jul 10 17:36:44.862516 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:44.862477 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0c5184ca-a9af-415c-9bcb-29818e49bb3d-podres\") pod \"perf-node-gather-daemonset-7cjzz\" (UID: \"0c5184ca-a9af-415c-9bcb-29818e49bb3d\") " pod="openshift-must-gather-dnfk7/perf-node-gather-daemonset-7cjzz" Jul 10 17:36:44.862716 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:44.862518 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0c5184ca-a9af-415c-9bcb-29818e49bb3d-sys\") pod \"perf-node-gather-daemonset-7cjzz\" (UID: \"0c5184ca-a9af-415c-9bcb-29818e49bb3d\") " pod="openshift-must-gather-dnfk7/perf-node-gather-daemonset-7cjzz" Jul 10 17:36:44.862716 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:44.862541 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0c5184ca-a9af-415c-9bcb-29818e49bb3d-lib-modules\") pod \"perf-node-gather-daemonset-7cjzz\" (UID: \"0c5184ca-a9af-415c-9bcb-29818e49bb3d\") " pod="openshift-must-gather-dnfk7/perf-node-gather-daemonset-7cjzz" Jul 10 17:36:44.870817 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:44.870796 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zrpg\" (UniqueName: \"kubernetes.io/projected/0c5184ca-a9af-415c-9bcb-29818e49bb3d-kube-api-access-9zrpg\") pod \"perf-node-gather-daemonset-7cjzz\" (UID: \"0c5184ca-a9af-415c-9bcb-29818e49bb3d\") " pod="openshift-must-gather-dnfk7/perf-node-gather-daemonset-7cjzz" Jul 10 17:36:44.996875 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:44.996845 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dnfk7/perf-node-gather-daemonset-7cjzz" Jul 10 17:36:45.126534 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:45.126508 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dnfk7/perf-node-gather-daemonset-7cjzz"] Jul 10 17:36:45.129164 ip-10-0-136-7 kubenswrapper[2578]: W0710 17:36:45.129131 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod0c5184ca_a9af_415c_9bcb_29818e49bb3d.slice/crio-5e99cead6ea0a1bcc9fca431de10a536b558d775436b62924ba6bda531bc8df8 WatchSource:0}: Error finding container 5e99cead6ea0a1bcc9fca431de10a536b558d775436b62924ba6bda531bc8df8: Status 404 returned error can't find the container with id 5e99cead6ea0a1bcc9fca431de10a536b558d775436b62924ba6bda531bc8df8 Jul 10 17:36:45.646868 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:45.646837 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-xc54t_54cce7d7-9921-4a19-aaf7-d909fbfafa8f/volume-data-source-validator/0.log" Jul 10 17:36:45.965512 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:45.965411 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dnfk7/perf-node-gather-daemonset-7cjzz" event={"ID":"0c5184ca-a9af-415c-9bcb-29818e49bb3d","Type":"ContainerStarted","Data":"ff60fa77c578a9d2529f590936d842d9c7e0ffdd2cb0ad06ea6224cf4d0b93ea"} Jul 10 17:36:45.965512 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:45.965498 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dnfk7/perf-node-gather-daemonset-7cjzz" event={"ID":"0c5184ca-a9af-415c-9bcb-29818e49bb3d","Type":"ContainerStarted","Data":"5e99cead6ea0a1bcc9fca431de10a536b558d775436b62924ba6bda531bc8df8"} Jul 10 17:36:45.965753 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:45.965560 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-dnfk7/perf-node-gather-daemonset-7cjzz" Jul 10 17:36:45.981941 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:45.981892 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-dnfk7/perf-node-gather-daemonset-7cjzz" podStartSLOduration=1.981877465 podStartE2EDuration="1.981877465s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 17:36:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 17:36:45.980884022 +0000 UTC m=+2550.070213085" watchObservedRunningTime="2026-07-10 17:36:45.981877465 +0000 UTC m=+2550.071206516" Jul 10 17:36:46.658898 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:46.658868 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-g5nt8_55d7dbb1-c8db-40fc-b416-7b781aae9aab/dns/0.log" Jul 10 17:36:46.680521 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:46.680495 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-g5nt8_55d7dbb1-c8db-40fc-b416-7b781aae9aab/kube-rbac-proxy/0.log" Jul 10 17:36:46.797731 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:46.797706 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6kk9g_1d85dbfa-1b16-4e26-b207-6bb7064345ee/dns-node-resolver/0.log" Jul 10 17:36:47.472664 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:47.472636 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-w9kb4_53987bf8-a205-4326-8757-ea8b3a6c4514/node-ca/0.log" Jul 10 17:36:48.510750 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:48.510719 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-2tm7x_973dcb52-e499-429c-a73d-471e3cd87ffc/discovery/0.log" Jul 10 17:36:48.565390 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:48.565362 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-558b5454d6-rb5qx_2a999b9a-8edf-4057-afbd-8761b62ae720/kube-auth-proxy/0.log" Jul 10 17:36:48.703277 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:48.703251 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-bb9cb67d8-ddrbp_3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84/router/0.log" Jul 10 17:36:49.464992 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:49.464966 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hwfb8_633753a4-3340-4473-9639-4299c06b53a8/serve-healthcheck-canary/0.log" Jul 10 17:36:49.924217 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:49.924188 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-d5bgt_ef2eb691-dbbf-4cfc-9fc1-bba4cd71895b/insights-operator/0.log" Jul 10 17:36:49.924633 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:49.924606 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-d5bgt_ef2eb691-dbbf-4cfc-9fc1-bba4cd71895b/insights-operator/1.log" Jul 10 17:36:50.088020 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:50.087991 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qzmtz_634f2940-fb9b-4a74-ae85-4fba5ee47d8d/kube-rbac-proxy/0.log" Jul 10 17:36:50.107477 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:50.107457 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qzmtz_634f2940-fb9b-4a74-ae85-4fba5ee47d8d/exporter/0.log" Jul 10 17:36:50.127978 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:50.127954 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qzmtz_634f2940-fb9b-4a74-ae85-4fba5ee47d8d/extractor/0.log" Jul 10 17:36:51.979987 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:51.979960 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-dnfk7/perf-node-gather-daemonset-7cjzz" Jul 10 17:36:52.748228 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:52.748196 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67764b8b78-tmqk6_fcde3330-8614-47ac-9c0f-93c34a0cfc17/manager/0.log" Jul 10 17:36:52.765551 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:52.765527 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67764b8b78-tmqk6_fcde3330-8614-47ac-9c0f-93c34a0cfc17/copy-manifests/0.log" Jul 10 17:36:54.086568 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:54.086543 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7bcc59c859-67cq9_5f682381-8753-4143-b130-4496b861e3c7/manager/0.log" Jul 10 17:36:55.226585 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:55.226550 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-2tm7x_973dcb52-e499-429c-a73d-471e3cd87ffc/discovery/0.log" Jul 10 17:36:55.249204 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:55.249181 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-558b5454d6-rb5qx_2a999b9a-8edf-4057-afbd-8761b62ae720/kube-auth-proxy/0.log" Jul 10 17:36:55.346432 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:55.346407 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-bb9cb67d8-ddrbp_3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84/router/0.log" Jul 10 17:36:56.752161 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:56.752079 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-2tm7x_973dcb52-e499-429c-a73d-471e3cd87ffc/discovery/0.log" Jul 10 17:36:56.779595 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:56.779562 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-558b5454d6-rb5qx_2a999b9a-8edf-4057-afbd-8761b62ae720/kube-auth-proxy/0.log" Jul 10 17:36:56.890634 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:56.890606 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-bb9cb67d8-ddrbp_3c7fb6e2-bbb4-45a7-b717-7bd085ad9a84/router/0.log" Jul 10 17:36:59.695925 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:59.695888 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-8r74k_73c4ea4d-fc8b-4fca-b4bd-2fc9102d4347/kube-storage-version-migrator-operator/1.log" Jul 10 17:36:59.696716 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:36:59.696699 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-8r74k_73c4ea4d-fc8b-4fca-b4bd-2fc9102d4347/kube-storage-version-migrator-operator/0.log" Jul 10 17:37:00.685881 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:37:00.685851 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-9qtfh_a26f297b-fb0e-4bd2-8cab-e3ec3aff00c1/kube-multus/0.log" Jul 10 17:37:00.872739 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:37:00.872716 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5d4nh_6e1806e2-a453-4929-8632-267349af86c7/kube-multus-additional-cni-plugins/0.log" Jul 10 17:37:00.892724 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:37:00.892701 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5d4nh_6e1806e2-a453-4929-8632-267349af86c7/egress-router-binary-copy/0.log" Jul 10 17:37:00.912604 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:37:00.912583 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5d4nh_6e1806e2-a453-4929-8632-267349af86c7/cni-plugins/0.log" Jul 10 17:37:00.932304 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:37:00.932271 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5d4nh_6e1806e2-a453-4929-8632-267349af86c7/bond-cni-plugin/0.log" Jul 10 17:37:00.953308 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:37:00.953230 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5d4nh_6e1806e2-a453-4929-8632-267349af86c7/routeoverride-cni/0.log" Jul 10 17:37:00.975135 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:37:00.975107 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5d4nh_6e1806e2-a453-4929-8632-267349af86c7/whereabouts-cni-bincopy/0.log" Jul 10 17:37:00.996904 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:37:00.996876 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5d4nh_6e1806e2-a453-4929-8632-267349af86c7/whereabouts-cni/0.log" Jul 10 17:37:01.279559 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:37:01.279462 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-jr8jk_d43ce12f-a8fd-4a94-bb94-09e6562e1250/network-metrics-daemon/0.log" Jul 10 17:37:01.299153 ip-10-0-136-7 kubenswrapper[2578]: I0710 17:37:01.299126 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-jr8jk_d43ce12f-a8fd-4a94-bb94-09e6562e1250/kube-rbac-proxy/0.log"