Jul 15 19:21:26.297593 ip-10-0-141-62 systemd[1]: Starting Kubernetes Kubelet... Jul 15 19:21:26.768455 ip-10-0-141-62 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 19:21:26.768455 ip-10-0-141-62 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 15 19:21:26.768455 ip-10-0-141-62 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 19:21:26.768455 ip-10-0-141-62 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 15 19:21:26.768455 ip-10-0-141-62 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 19:21:26.770147 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.769994 2577 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 15 19:21:26.866446 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775061 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 19:21:26.866446 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775074 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 19:21:26.866446 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775077 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 19:21:26.866446 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775081 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 19:21:26.866446 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775084 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 19:21:26.866446 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775086 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 19:21:26.866446 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775089 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 19:21:26.866446 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775092 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 19:21:26.866446 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775096 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 19:21:26.866446 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775100 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 19:21:26.866446 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775103 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 19:21:26.866446 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775105 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 19:21:26.866446 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775108 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 19:21:26.866446 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775110 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 19:21:26.866446 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775113 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 19:21:26.866446 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775115 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 19:21:26.866446 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775118 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 19:21:26.866446 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775121 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 19:21:26.866446 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775124 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 19:21:26.866446 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775127 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 19:21:26.842649 ip-10-0-141-62 systemd[1]: Started Kubernetes Kubelet. Jul 15 19:21:26.867333 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775130 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 19:21:26.867333 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775134 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 19:21:26.867333 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775137 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 19:21:26.867333 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775140 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 19:21:26.867333 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775143 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 19:21:26.867333 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775146 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 19:21:26.867333 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775148 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 19:21:26.867333 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775151 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 19:21:26.867333 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775154 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 19:21:26.867333 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775157 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 19:21:26.867333 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775159 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 19:21:26.867333 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775162 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 19:21:26.867333 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775164 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 19:21:26.867333 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775168 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 19:21:26.867333 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775170 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 19:21:26.867333 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775173 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 19:21:26.867333 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775175 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 19:21:26.867333 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775178 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 19:21:26.867333 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775180 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 19:21:26.868357 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775182 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 19:21:26.868357 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775185 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 19:21:26.868357 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775187 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 19:21:26.868357 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775189 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 19:21:26.868357 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775192 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 19:21:26.868357 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775194 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 19:21:26.868357 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775197 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 19:21:26.868357 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775199 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 19:21:26.868357 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775203 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 19:21:26.868357 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775207 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 19:21:26.868357 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775211 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 19:21:26.868357 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775214 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 19:21:26.868357 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775217 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 19:21:26.868357 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775220 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 19:21:26.868357 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775222 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 19:21:26.868357 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775225 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 19:21:26.868357 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775227 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 19:21:26.868357 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775230 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 19:21:26.868357 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775233 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 19:21:26.869148 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775236 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 19:21:26.869148 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775240 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 19:21:26.869148 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775242 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 19:21:26.869148 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775245 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 19:21:26.869148 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775247 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 19:21:26.869148 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775250 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 19:21:26.869148 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775252 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 19:21:26.869148 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775255 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 19:21:26.869148 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775257 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 19:21:26.869148 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775259 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 19:21:26.869148 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775262 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 19:21:26.869148 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775265 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 19:21:26.869148 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775267 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 19:21:26.869148 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775270 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 19:21:26.869148 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775272 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 19:21:26.869148 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775275 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 19:21:26.869148 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775277 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 19:21:26.869148 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775280 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 19:21:26.869148 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775282 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 19:21:26.869148 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775284 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 19:21:26.869996 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775287 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 19:21:26.869996 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775289 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 19:21:26.869996 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775292 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 19:21:26.869996 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775294 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 19:21:26.869996 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775297 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 19:21:26.869996 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775300 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 19:21:26.869996 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775302 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 19:21:26.869996 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775306 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 19:21:26.869996 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775322 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 15 19:21:26.869996 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775325 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 19:21:26.869996 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775328 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 19:21:26.869996 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775330 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 19:21:26.869996 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775333 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 19:21:26.869996 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775336 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 19:21:26.869996 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775339 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 19:21:26.869996 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775341 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 19:21:26.869996 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775344 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 19:21:26.869996 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775346 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 19:21:26.869996 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775349 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 19:21:26.869996 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775351 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 19:21:26.870807 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.775354 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 19:21:26.870807 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776033 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 19:21:26.870807 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776040 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 19:21:26.870807 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776042 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 19:21:26.870807 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776045 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 19:21:26.870807 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776048 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 19:21:26.870807 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776050 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 19:21:26.870807 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776053 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 19:21:26.870807 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776055 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 19:21:26.870807 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776058 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 19:21:26.870807 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776060 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 19:21:26.870807 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776063 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 19:21:26.870807 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776065 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 15 19:21:26.870807 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776068 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 19:21:26.870807 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776070 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 19:21:26.870807 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776073 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 19:21:26.870807 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776075 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 19:21:26.870807 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776078 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 19:21:26.870807 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776080 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 19:21:26.870807 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776083 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 19:21:26.871823 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776085 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 19:21:26.871823 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776088 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 19:21:26.871823 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776093 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 19:21:26.871823 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776097 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 19:21:26.871823 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776100 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 19:21:26.871823 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776104 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 19:21:26.871823 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776107 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 19:21:26.871823 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776110 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 19:21:26.871823 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776113 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 19:21:26.871823 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776116 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 19:21:26.871823 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776119 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 19:21:26.871823 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776121 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 19:21:26.871823 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776124 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 19:21:26.871823 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776127 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 19:21:26.871823 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776129 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 19:21:26.871823 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776131 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 19:21:26.871823 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776134 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 19:21:26.871823 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776136 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 19:21:26.871823 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776139 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 19:21:26.872399 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776142 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 19:21:26.872399 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776144 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 19:21:26.872399 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776147 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 19:21:26.872399 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776149 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 19:21:26.872399 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776152 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 19:21:26.872399 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776155 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 19:21:26.872399 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776158 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 19:21:26.872399 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776160 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 19:21:26.872399 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776163 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 19:21:26.872399 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776166 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 19:21:26.872399 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776170 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 19:21:26.872399 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776173 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 19:21:26.872399 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776176 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 19:21:26.872399 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776179 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 19:21:26.872399 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776182 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 19:21:26.872399 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776186 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 19:21:26.872399 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776189 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 19:21:26.872399 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776192 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 19:21:26.872399 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776194 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 19:21:26.872399 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776198 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 19:21:26.873030 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776200 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 19:21:26.873030 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776203 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 19:21:26.873030 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776206 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 19:21:26.873030 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776209 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 19:21:26.873030 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776213 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 19:21:26.873030 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776216 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 19:21:26.873030 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776219 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 19:21:26.873030 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776222 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 19:21:26.873030 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776224 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 19:21:26.873030 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776227 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 19:21:26.873030 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776229 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 19:21:26.873030 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776232 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 19:21:26.873030 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776235 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 19:21:26.873030 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776237 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 19:21:26.873030 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776240 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 19:21:26.873030 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776242 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 19:21:26.873030 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776245 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 19:21:26.873030 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776248 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 19:21:26.873030 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776251 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 19:21:26.873567 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776253 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 19:21:26.873567 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776255 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 19:21:26.873567 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776258 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 19:21:26.873567 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776260 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 19:21:26.873567 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776262 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 19:21:26.873567 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776265 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 19:21:26.873567 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776268 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 19:21:26.873567 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776271 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 19:21:26.873567 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776274 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 19:21:26.873567 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776276 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 19:21:26.873567 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776279 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 19:21:26.873567 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776281 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 19:21:26.873567 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776284 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 19:21:26.873567 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776286 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 19:21:26.873567 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776289 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 19:21:26.873567 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776291 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 19:21:26.873567 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776294 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 19:21:26.873567 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776296 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 19:21:26.873567 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776299 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 19:21:26.873567 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776302 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 19:21:26.874295 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776304 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 19:21:26.874295 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776307 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 19:21:26.874295 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776652 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 19:21:26.874295 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776657 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 19:21:26.874295 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776661 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 19:21:26.874295 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776664 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 15 19:21:26.874295 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776667 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 19:21:26.874295 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776670 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 19:21:26.874295 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776672 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 19:21:26.874295 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776675 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 19:21:26.874295 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776677 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 19:21:26.874295 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776680 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 19:21:26.874295 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776682 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 19:21:26.874295 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776685 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 19:21:26.874295 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776688 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 19:21:26.874295 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776691 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 19:21:26.874295 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776693 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 19:21:26.874295 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776696 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 19:21:26.874295 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776699 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 19:21:26.875229 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776703 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 19:21:26.875229 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776707 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 19:21:26.875229 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776710 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 19:21:26.875229 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776713 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 19:21:26.875229 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776716 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 19:21:26.875229 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776718 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 19:21:26.875229 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776721 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 19:21:26.875229 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776725 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 19:21:26.875229 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776728 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 19:21:26.875229 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776731 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 19:21:26.875229 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776733 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 19:21:26.875229 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776736 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 19:21:26.875229 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776739 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 19:21:26.875229 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776741 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 19:21:26.875229 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776744 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 19:21:26.875229 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776748 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 19:21:26.875229 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776751 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 19:21:26.875229 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776756 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 19:21:26.875229 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776760 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 19:21:26.875869 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776763 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 19:21:26.875869 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776767 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 19:21:26.875869 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776770 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 19:21:26.875869 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776772 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 19:21:26.875869 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776776 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 19:21:26.875869 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776778 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 19:21:26.875869 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776781 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 19:21:26.875869 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776783 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 19:21:26.875869 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776786 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 19:21:26.875869 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776788 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 19:21:26.875869 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776791 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 19:21:26.875869 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776793 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 19:21:26.875869 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776797 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 19:21:26.875869 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776799 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 19:21:26.875869 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776802 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 19:21:26.875869 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776804 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 19:21:26.875869 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776806 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 19:21:26.875869 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776809 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 19:21:26.875869 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776811 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 19:21:26.875869 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776814 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 19:21:27.016149 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776818 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 19:21:27.016149 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776821 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 19:21:27.016149 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776823 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 19:21:27.016149 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776827 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 19:21:27.016149 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776830 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 19:21:27.016149 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776832 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 19:21:27.016149 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776835 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 19:21:27.016149 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776838 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 19:21:27.016149 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776840 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 19:21:27.016149 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776844 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 19:21:27.016149 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776847 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 19:21:27.016149 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776850 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 19:21:27.016149 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776852 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 19:21:27.016149 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776855 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 19:21:27.016149 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776857 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 19:21:27.016149 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776860 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 19:21:27.016149 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776863 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 19:21:27.016149 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776865 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 19:21:27.016149 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776868 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 19:21:27.016149 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776871 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 19:21:27.019731 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776873 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 19:21:27.019731 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776877 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 19:21:27.019731 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776880 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 19:21:27.019731 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776882 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 19:21:27.019731 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776885 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 19:21:27.019731 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776892 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 19:21:27.019731 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776894 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 19:21:27.019731 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776897 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 19:21:27.019731 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776899 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 19:21:27.019731 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776902 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 19:21:27.019731 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776904 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 19:21:27.019731 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776906 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 19:21:27.019731 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776909 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 19:21:27.019731 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776911 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 19:21:27.019731 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776914 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 19:21:27.019731 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776916 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 19:21:27.019731 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776919 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 19:21:27.019731 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776921 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 19:21:27.019731 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776924 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 19:21:27.019731 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776926 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 19:21:27.020488 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776928 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 19:21:27.020488 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776931 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 19:21:27.020488 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.776936 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 19:21:27.020488 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.776996 2577 flags.go:64] FLAG: --address="0.0.0.0" Jul 15 19:21:27.020488 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777002 2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 15 19:21:27.020488 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777008 2577 flags.go:64] FLAG: --anonymous-auth="true" Jul 15 19:21:27.020488 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777013 2577 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 15 19:21:27.020488 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777016 2577 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 15 19:21:27.020488 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777020 2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 15 19:21:27.020488 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777025 2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 15 19:21:27.020488 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777032 2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 15 19:21:27.020488 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777036 2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 15 19:21:27.020488 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777040 2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 15 19:21:27.020488 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777043 2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 15 19:21:27.020488 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777047 2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 15 19:21:27.020488 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777050 2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 15 19:21:27.020488 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777052 2577 flags.go:64] FLAG: --cgroup-root="" Jul 15 19:21:27.020488 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777055 2577 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 15 19:21:27.020488 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777059 2577 flags.go:64] FLAG: --client-ca-file="" Jul 15 19:21:27.020488 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777062 2577 flags.go:64] FLAG: --cloud-provider="external" Jul 15 19:21:27.020488 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777065 2577 flags.go:64] FLAG: --cluster-dns="[]" Jul 15 19:21:27.020488 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777068 2577 flags.go:64] FLAG: --cluster-domain="" Jul 15 19:21:27.020488 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777071 2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 15 19:21:27.021256 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777074 2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 15 19:21:27.021256 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777077 2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 15 19:21:27.021256 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777080 2577 flags.go:64] FLAG: --container-log-max-files="5" Jul 15 19:21:27.021256 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777084 2577 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 15 19:21:27.021256 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777087 2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 15 19:21:27.021256 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777090 2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 15 19:21:27.021256 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777093 2577 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 15 19:21:27.021256 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777096 2577 flags.go:64] FLAG: --contention-profiling="false" Jul 15 19:21:27.021256 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777099 2577 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 15 19:21:27.021256 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777101 2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 15 19:21:27.021256 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777105 2577 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 15 19:21:27.021256 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777112 2577 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 15 19:21:27.021256 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777116 2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 15 19:21:27.021256 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777119 2577 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 15 19:21:27.021256 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777122 2577 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 15 19:21:27.021256 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777125 2577 flags.go:64] FLAG: --enable-load-reader="false" Jul 15 19:21:27.021256 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777128 2577 flags.go:64] FLAG: --enable-server="true" Jul 15 19:21:27.021256 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777130 2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 15 19:21:27.021256 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777135 2577 flags.go:64] FLAG: --event-burst="100" Jul 15 19:21:27.021256 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777138 2577 flags.go:64] FLAG: --event-qps="50" Jul 15 19:21:27.021256 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777141 2577 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 15 19:21:27.021256 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777143 2577 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 15 19:21:27.021256 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777146 2577 flags.go:64] FLAG: --eviction-hard="" Jul 15 19:21:27.021256 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777149 2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 15 19:21:27.022002 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777152 2577 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 15 19:21:27.022002 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777155 2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 15 19:21:27.022002 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777158 2577 flags.go:64] FLAG: --eviction-soft="" Jul 15 19:21:27.022002 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777160 2577 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 15 19:21:27.022002 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777164 2577 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 15 19:21:27.022002 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777166 2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 15 19:21:27.022002 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777169 2577 flags.go:64] FLAG: --experimental-mounter-path="" Jul 15 19:21:27.022002 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777172 2577 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 15 19:21:27.022002 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777175 2577 flags.go:64] FLAG: --fail-swap-on="true" Jul 15 19:21:27.022002 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777178 2577 flags.go:64] FLAG: --feature-gates="" Jul 15 19:21:27.022002 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777181 2577 flags.go:64] FLAG: --file-check-frequency="20s" Jul 15 19:21:27.022002 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777184 2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 15 19:21:27.022002 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777187 2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 15 19:21:27.022002 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777190 2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 15 19:21:27.022002 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777193 2577 flags.go:64] FLAG: --healthz-port="10248" Jul 15 19:21:27.022002 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777196 2577 flags.go:64] FLAG: --help="false" Jul 15 19:21:27.022002 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777199 2577 flags.go:64] FLAG: --hostname-override="ip-10-0-141-62.ec2.internal" Jul 15 19:21:27.022002 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777202 2577 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 15 19:21:27.022002 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777204 2577 flags.go:64] FLAG: --http-check-frequency="20s" Jul 15 19:21:27.022002 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777209 2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 15 19:21:27.022002 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777213 2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 15 19:21:27.022002 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777216 2577 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 15 19:21:27.022002 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777219 2577 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 15 19:21:27.022002 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777222 2577 flags.go:64] FLAG: --image-service-endpoint="" Jul 15 19:21:27.022780 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777224 2577 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 15 19:21:27.022780 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777227 2577 flags.go:64] FLAG: --kube-api-burst="100" Jul 15 19:21:27.022780 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777230 2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 15 19:21:27.022780 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777233 2577 flags.go:64] FLAG: --kube-api-qps="50" Jul 15 19:21:27.022780 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777236 2577 flags.go:64] FLAG: --kube-reserved="" Jul 15 19:21:27.022780 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777238 2577 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 15 19:21:27.022780 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777241 2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 15 19:21:27.022780 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777244 2577 flags.go:64] FLAG: --kubelet-cgroups="" Jul 15 19:21:27.022780 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777247 2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 15 19:21:27.022780 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777250 2577 flags.go:64] FLAG: --lock-file="" Jul 15 19:21:27.022780 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777252 2577 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 15 19:21:27.022780 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777255 2577 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 15 19:21:27.022780 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777258 2577 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 15 19:21:27.022780 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777263 2577 flags.go:64] FLAG: --log-json-split-stream="false" Jul 15 19:21:27.022780 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777266 2577 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 15 19:21:27.022780 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777269 2577 flags.go:64] FLAG: --log-text-split-stream="false" Jul 15 19:21:27.022780 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777272 2577 flags.go:64] FLAG: --logging-format="text" Jul 15 19:21:27.022780 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777275 2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 15 19:21:27.022780 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777278 2577 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 15 19:21:27.022780 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777281 2577 flags.go:64] FLAG: --manifest-url="" Jul 15 19:21:27.022780 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777283 2577 flags.go:64] FLAG: --manifest-url-header="" Jul 15 19:21:27.022780 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777287 2577 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 15 19:21:27.022780 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777290 2577 flags.go:64] FLAG: --max-open-files="1000000" Jul 15 19:21:27.022780 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777294 2577 flags.go:64] FLAG: --max-pods="110" Jul 15 19:21:27.022780 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777297 2577 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 15 19:21:27.023560 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777299 2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 15 19:21:27.023560 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777302 2577 flags.go:64] FLAG: --memory-manager-policy="None" Jul 15 19:21:27.023560 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777329 2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 15 19:21:27.023560 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777342 2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 15 19:21:27.023560 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777345 2577 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 15 19:21:27.023560 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777348 2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 15 19:21:27.023560 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777355 2577 flags.go:64] FLAG: --node-status-max-images="50" Jul 15 19:21:27.023560 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777358 2577 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 15 19:21:27.023560 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777361 2577 flags.go:64] FLAG: --oom-score-adj="-999" Jul 15 19:21:27.023560 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777364 2577 flags.go:64] FLAG: --pod-cidr="" Jul 15 19:21:27.023560 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777366 2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 15 19:21:27.023560 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777372 2577 flags.go:64] FLAG: --pod-manifest-path="" Jul 15 19:21:27.023560 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777375 2577 flags.go:64] FLAG: --pod-max-pids="-1" Jul 15 19:21:27.023560 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777378 2577 flags.go:64] FLAG: --pods-per-core="0" Jul 15 19:21:27.023560 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777381 2577 flags.go:64] FLAG: --port="10250" Jul 15 19:21:27.023560 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777384 2577 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 15 19:21:27.023560 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777387 2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0046700050fee61d9" Jul 15 19:21:27.023560 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777390 2577 flags.go:64] FLAG: --qos-reserved="" Jul 15 19:21:27.023560 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777393 2577 flags.go:64] FLAG: --read-only-port="10255" Jul 15 19:21:27.023560 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777396 2577 flags.go:64] FLAG: --register-node="true" Jul 15 19:21:27.023560 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777399 2577 flags.go:64] FLAG: --register-with-taints="" Jul 15 19:21:27.023560 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777403 2577 flags.go:64] FLAG: --registry-burst="10" Jul 15 19:21:27.023560 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777406 2577 flags.go:64] FLAG: --registry-qps="5" Jul 15 19:21:27.023560 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777408 2577 flags.go:64] FLAG: --reserved-cpus="" Jul 15 19:21:27.024306 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777411 2577 flags.go:64] FLAG: --reserved-memory="" Jul 15 19:21:27.024306 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777415 2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 15 19:21:27.024306 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777418 2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 15 19:21:27.024306 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777420 2577 flags.go:64] FLAG: --rotate-certificates="false" Jul 15 19:21:27.024306 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777423 2577 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 15 19:21:27.024306 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777426 2577 flags.go:64] FLAG: --runonce="false" Jul 15 19:21:27.024306 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777429 2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 15 19:21:27.024306 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777432 2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 15 19:21:27.024306 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777435 2577 flags.go:64] FLAG: --seccomp-default="false" Jul 15 19:21:27.024306 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777438 2577 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 15 19:21:27.024306 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777441 2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 15 19:21:27.024306 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777446 2577 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 15 19:21:27.024306 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777449 2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 15 19:21:27.024306 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777452 2577 flags.go:64] FLAG: --storage-driver-password="root" Jul 15 19:21:27.024306 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777455 2577 flags.go:64] FLAG: --storage-driver-secure="false" Jul 15 19:21:27.024306 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777458 2577 flags.go:64] FLAG: --storage-driver-table="stats" Jul 15 19:21:27.024306 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777461 2577 flags.go:64] FLAG: --storage-driver-user="root" Jul 15 19:21:27.024306 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777464 2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 15 19:21:27.024306 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777467 2577 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 15 19:21:27.024306 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777469 2577 flags.go:64] FLAG: --system-cgroups="" Jul 15 19:21:27.024306 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777472 2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 15 19:21:27.024306 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777478 2577 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 15 19:21:27.024306 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777480 2577 flags.go:64] FLAG: --tls-cert-file="" Jul 15 19:21:27.024306 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777483 2577 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 15 19:21:27.024306 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777487 2577 flags.go:64] FLAG: --tls-min-version="" Jul 15 19:21:27.025042 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777489 2577 flags.go:64] FLAG: --tls-private-key-file="" Jul 15 19:21:27.025042 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777492 2577 flags.go:64] FLAG: --topology-manager-policy="none" Jul 15 19:21:27.025042 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777495 2577 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 15 19:21:27.025042 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777498 2577 flags.go:64] FLAG: --topology-manager-scope="container" Jul 15 19:21:27.025042 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777500 2577 flags.go:64] FLAG: --v="2" Jul 15 19:21:27.025042 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777504 2577 flags.go:64] FLAG: --version="false" Jul 15 19:21:27.025042 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777508 2577 flags.go:64] FLAG: --vmodule="" Jul 15 19:21:27.025042 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777512 2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 15 19:21:27.025042 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777515 2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 15 19:21:27.025042 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777623 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 19:21:27.025042 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777627 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 19:21:27.025042 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777631 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 19:21:27.025042 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777634 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 19:21:27.025042 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777637 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 19:21:27.025042 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777639 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 19:21:27.025042 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777642 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 19:21:27.025042 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777644 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 19:21:27.025042 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777647 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 19:21:27.025042 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777652 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 19:21:27.025042 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777655 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 19:21:27.025042 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777657 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 19:21:27.025042 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777661 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 19:21:27.025754 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777665 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 19:21:27.025754 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777668 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 19:21:27.025754 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777671 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 19:21:27.025754 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777673 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 19:21:27.025754 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777676 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 19:21:27.025754 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777679 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 19:21:27.025754 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777681 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 19:21:27.025754 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777684 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 19:21:27.025754 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777686 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 19:21:27.025754 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777689 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 19:21:27.025754 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777692 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 19:21:27.025754 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777694 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 19:21:27.025754 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777696 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 19:21:27.025754 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777699 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 19:21:27.025754 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777701 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 19:21:27.025754 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777704 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 19:21:27.025754 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777707 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 19:21:27.025754 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777710 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 19:21:27.025754 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777712 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 19:21:27.025754 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777715 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 19:21:27.026332 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777718 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 19:21:27.026332 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777720 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 19:21:27.026332 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777723 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 19:21:27.026332 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777725 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 19:21:27.026332 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777728 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 19:21:27.026332 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777730 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 19:21:27.026332 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777733 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 19:21:27.026332 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777735 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 19:21:27.026332 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777740 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 19:21:27.026332 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777742 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 19:21:27.026332 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777745 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 19:21:27.026332 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777747 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 19:21:27.026332 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777750 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 19:21:27.026332 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777752 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 19:21:27.026332 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777755 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 19:21:27.026332 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777757 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 19:21:27.026332 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777759 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 19:21:27.026332 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777762 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 19:21:27.026332 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777764 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 19:21:27.026332 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777767 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 19:21:27.026969 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777770 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 19:21:27.026969 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777772 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 19:21:27.026969 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777775 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 19:21:27.026969 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777777 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 19:21:27.026969 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777779 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 19:21:27.026969 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777782 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 19:21:27.026969 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777784 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 19:21:27.026969 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777787 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 19:21:27.026969 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777789 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 19:21:27.026969 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777792 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 19:21:27.026969 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777794 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 19:21:27.026969 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777797 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 19:21:27.026969 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777799 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 19:21:27.026969 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777802 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 19:21:27.026969 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777804 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 19:21:27.026969 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777806 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 19:21:27.026969 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777809 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 19:21:27.026969 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777811 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 19:21:27.026969 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777814 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 19:21:27.026969 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777816 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 19:21:27.027559 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777821 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 19:21:27.027559 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777823 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 19:21:27.027559 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777826 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 19:21:27.027559 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777829 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 19:21:27.027559 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777831 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 19:21:27.027559 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777834 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 19:21:27.027559 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777837 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 19:21:27.027559 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777839 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 19:21:27.027559 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777842 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 19:21:27.027559 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777844 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 19:21:27.027559 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777847 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 19:21:27.027559 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777849 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 19:21:27.027559 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777852 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 19:21:27.027559 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777855 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 19:21:27.027559 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777857 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 19:21:27.027559 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777861 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 19:21:27.027559 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777864 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 19:21:27.027559 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777867 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 19:21:27.027559 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777869 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 15 19:21:27.027559 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777872 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 19:21:27.028182 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777874 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 19:21:27.028182 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777877 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 19:21:27.028182 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777880 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 19:21:27.028182 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777882 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 19:21:27.028182 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777885 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 19:21:27.028182 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.777889 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 19:21:27.028182 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.777895 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 19:21:27.028182 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.786868 2577 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 15 19:21:27.028182 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.786883 2577 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 15 19:21:27.028182 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.786903 2577 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 15 19:21:27.028182 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.786908 2577 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 15 19:21:27.028182 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.786965 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 19:21:27.028182 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.786971 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 19:21:27.028182 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.786974 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 19:21:27.028671 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.786978 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 19:21:27.028671 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.786980 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 15 19:21:27.028671 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.786983 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 19:21:27.028671 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.786986 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 19:21:27.028671 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.786989 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 19:21:27.028671 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.786991 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 19:21:27.028671 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.786994 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 19:21:27.028671 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.786997 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 19:21:27.028671 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.786999 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 19:21:27.028671 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787002 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 19:21:27.028671 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787005 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 19:21:27.028671 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787007 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 19:21:27.028671 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787010 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 19:21:27.028671 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787012 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 19:21:27.028671 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787015 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 19:21:27.028671 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787019 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 19:21:27.028671 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787023 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 19:21:27.028671 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787026 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 19:21:27.028671 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787030 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 19:21:27.029257 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787034 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 19:21:27.029257 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787040 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 19:21:27.029257 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787043 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 19:21:27.029257 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787046 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 19:21:27.029257 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787049 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 19:21:27.029257 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787052 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 19:21:27.029257 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787054 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 19:21:27.029257 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787057 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 19:21:27.029257 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787059 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 19:21:27.029257 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787062 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 19:21:27.029257 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787065 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 19:21:27.029257 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787067 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 19:21:27.029257 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787070 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 19:21:27.029257 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787073 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 19:21:27.029257 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787075 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 19:21:27.029257 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787078 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 19:21:27.029257 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787080 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 19:21:27.029257 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787083 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 19:21:27.029257 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787085 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 19:21:27.029257 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787088 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 19:21:27.029876 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787091 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 19:21:27.029876 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787094 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 19:21:27.029876 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787097 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 19:21:27.029876 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787099 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 19:21:27.029876 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787102 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 19:21:27.029876 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787105 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 19:21:27.029876 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787107 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 19:21:27.029876 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787110 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 19:21:27.029876 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787112 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 19:21:27.029876 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787115 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 19:21:27.029876 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787117 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 19:21:27.029876 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787120 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 19:21:27.029876 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787123 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 19:21:27.029876 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787127 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 19:21:27.029876 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787130 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 19:21:27.029876 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787132 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 19:21:27.029876 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787135 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 19:21:27.029876 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787138 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 19:21:27.029876 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787140 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 19:21:27.029876 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787143 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 19:21:27.030503 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787145 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 19:21:27.030503 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787148 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 19:21:27.030503 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787150 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 19:21:27.030503 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787153 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 19:21:27.030503 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787155 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 19:21:27.030503 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787158 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 19:21:27.030503 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787161 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 19:21:27.030503 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787163 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 19:21:27.030503 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787167 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 19:21:27.030503 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787170 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 19:21:27.030503 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787173 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 19:21:27.030503 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787176 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 19:21:27.030503 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787180 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 19:21:27.030503 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787183 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 19:21:27.030503 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787186 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 19:21:27.030503 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787189 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 19:21:27.030503 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787192 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 19:21:27.030503 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787195 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 19:21:27.030503 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787198 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 19:21:27.030503 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787201 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 19:21:27.031065 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787204 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 19:21:27.031065 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787207 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 19:21:27.031065 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787225 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 19:21:27.031065 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787229 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 19:21:27.031065 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787232 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 19:21:27.031065 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787237 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 19:21:27.031065 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787240 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 19:21:27.031065 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787243 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 19:21:27.031065 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787245 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 19:21:27.031065 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787248 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 19:21:27.031065 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787251 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 19:21:27.031065 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787254 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 19:21:27.031065 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787256 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 19:21:27.031065 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787259 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 19:21:27.031065 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787261 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 19:21:27.031065 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787264 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 19:21:27.031065 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787267 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 19:21:27.031636 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.787271 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 19:21:27.031636 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787407 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 19:21:27.031636 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787413 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 19:21:27.031636 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787416 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 19:21:27.031636 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787419 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 19:21:27.031636 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787422 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 19:21:27.031636 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787426 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 19:21:27.031636 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787432 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 19:21:27.031636 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787434 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 19:21:27.031636 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787437 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 19:21:27.031636 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787439 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 19:21:27.031636 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787442 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 19:21:27.031636 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787445 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 19:21:27.031636 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787448 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 19:21:27.031636 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787451 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 19:21:27.032433 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787453 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 19:21:27.032433 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787456 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 19:21:27.032433 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787458 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 19:21:27.032433 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787461 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 19:21:27.032433 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787463 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 19:21:27.032433 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787466 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 19:21:27.032433 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787470 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 19:21:27.032433 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787473 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 19:21:27.032433 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787476 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 19:21:27.032433 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787478 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 19:21:27.032433 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787481 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 19:21:27.032433 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787483 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 19:21:27.032433 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787486 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 19:21:27.032433 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787488 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 19:21:27.032433 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787491 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 19:21:27.032433 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787493 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 15 19:21:27.032433 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787496 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 19:21:27.032433 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787498 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 19:21:27.032433 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787501 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 19:21:27.032433 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787503 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 19:21:27.033053 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787506 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 19:21:27.033053 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787508 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 19:21:27.033053 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787511 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 19:21:27.033053 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787513 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 19:21:27.033053 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787517 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 19:21:27.033053 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787519 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 19:21:27.033053 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787522 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 19:21:27.033053 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787524 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 19:21:27.033053 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787527 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 19:21:27.033053 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787529 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 19:21:27.033053 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787532 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 19:21:27.033053 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787535 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 19:21:27.033053 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787537 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 19:21:27.033053 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787540 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 19:21:27.033053 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787542 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 19:21:27.033053 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787545 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 19:21:27.033053 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787547 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 19:21:27.033053 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787549 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 19:21:27.033053 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787554 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 19:21:27.033053 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787556 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 19:21:27.176552 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787559 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 19:21:27.176552 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787561 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 19:21:27.176552 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787564 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 19:21:27.176552 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787566 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 19:21:27.176552 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787569 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 19:21:27.176552 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787572 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 19:21:27.176552 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787574 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 19:21:27.176552 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787577 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 19:21:27.176552 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787579 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 19:21:27.176552 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787582 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 19:21:27.176552 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787584 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 19:21:27.176552 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787588 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 19:21:27.176552 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787592 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 19:21:27.176552 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787595 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 19:21:27.176552 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787598 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 19:21:27.176552 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787600 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 19:21:27.176552 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787605 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 19:21:27.176552 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787608 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 19:21:27.176552 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787611 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 19:21:27.177649 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787613 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 19:21:27.177649 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787616 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 19:21:27.177649 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787618 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 19:21:27.177649 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787621 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 19:21:27.177649 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787624 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 19:21:27.177649 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787628 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 19:21:27.177649 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787630 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 19:21:27.177649 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787633 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 19:21:27.177649 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787635 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 19:21:27.177649 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787638 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 19:21:27.177649 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787640 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 19:21:27.177649 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787644 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 19:21:27.177649 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787647 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 19:21:27.177649 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787649 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 19:21:27.177649 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787652 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 19:21:27.177649 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787654 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 19:21:27.177649 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787657 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 19:21:27.177649 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787659 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 19:21:27.177649 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787662 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 19:21:27.177649 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787664 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 19:21:27.178320 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787667 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 19:21:27.178320 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787670 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 19:21:27.178320 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787672 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 19:21:27.178320 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787675 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 19:21:27.178320 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787679 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 19:21:27.178320 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:26.787682 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 19:21:27.178320 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.787687 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 19:21:27.178320 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.787776 2577 server.go:962] "Client rotation is on, will bootstrap in background" Jul 15 19:21:27.178320 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.791218 2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 15 19:21:27.178320 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.792988 2577 server.go:1019] "Starting client certificate rotation" Jul 15 19:21:27.178320 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.793423 2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 15 19:21:27.178320 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.793538 2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 15 19:21:27.178320 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.797057 2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 15 19:21:27.178997 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.798355 2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 15 19:21:27.178997 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.808381 2577 log.go:25] "Validated CRI v1 runtime API" Jul 15 19:21:27.178997 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.810865 2577 log.go:25] "Validated CRI v1 image API" Jul 15 19:21:27.178997 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.811769 2577 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 15 19:21:27.178997 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.814234 2577 fs.go:135] Filesystem UUIDs: map[5ef70f27-ec2c-4c20-9638-5fdd5c305b1d:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 eb05ca13-3782-4792-93a9-762bb235cf16:/dev/nvme0n1p3] Jul 15 19:21:27.178997 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.814247 2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 15 19:21:27.179687 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.819032 2577 manager.go:216] Machine: {Timestamp:2026-07-15 19:21:26.81854202 +0000 UTC m=+0.395543683 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100227 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec297dc28a829275d8e5627142c2644b SystemUUID:ec297dc2-8a82-9275-d8e5-627142c2644b BootID:051e0b5d-0dc9-48a2-88bb-8893610ab0ba Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:b8:38:b3:d5:89 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:b8:38:b3:d5:89 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:6a:80:0f:06:b1:01 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 15 19:21:27.179687 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.819147 2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 15 19:21:27.179687 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.819216 2577 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 15 19:21:27.179687 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.823117 2577 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 15 19:21:27.179687 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.823177 2577 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-62.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 15 19:21:27.179687 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.824592 2577 topology_manager.go:138] "Creating topology manager with none policy" Jul 15 19:21:27.179687 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.824608 2577 container_manager_linux.go:309] "Creating device plugin manager" Jul 15 19:21:27.179687 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.824618 2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 15 19:21:27.179687 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.825644 2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 15 19:21:27.179687 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.826959 2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 15 19:21:27.179687 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.827136 2577 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 15 19:21:27.179687 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.830757 2577 state_mem.go:36] "Initialized new in-memory state store" Jul 15 19:21:27.179687 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.830864 2577 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 15 19:21:27.179687 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.830890 2577 kubelet.go:486] "Attempting to sync node with API server" Jul 15 19:21:27.179687 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.830899 2577 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 15 19:21:27.179687 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.830916 2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 15 19:21:27.179687 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.830924 2577 kubelet.go:398] "Adding apiserver pod source" Jul 15 19:21:27.180351 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.830932 2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 15 19:21:27.180351 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.831207 2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 15 19:21:27.180351 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.831221 2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 15 19:21:27.180351 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.833530 2577 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 15 19:21:27.180351 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.835810 2577 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 15 19:21:27.180351 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.835836 2577 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 15 19:21:27.180351 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.835984 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 15 19:21:27.180351 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.835997 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 15 19:21:27.180351 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.836003 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 15 19:21:27.180351 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.836009 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 15 19:21:27.180351 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.836017 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 15 19:21:27.180351 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.836023 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 15 19:21:27.180351 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.836996 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 15 19:21:27.180351 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.837003 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 15 19:21:27.180351 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.837011 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 15 19:21:27.180351 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.837016 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 15 19:21:27.180351 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.837027 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 15 19:21:27.180351 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.837034 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 15 19:21:27.180351 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.837060 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 15 19:21:27.180351 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.837066 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 15 19:21:27.180351 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.839926 2577 server.go:1268] "Started kubelet" Jul 15 19:21:27.180351 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.839993 2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 15 19:21:27.180351 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.840042 2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 15 19:21:27.180351 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.840090 2577 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 15 19:21:27.180351 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.841341 2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-62.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 15 19:21:27.180351 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:26.841406 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-62.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 15 19:21:27.180351 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:26.841477 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 15 19:21:27.180351 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.841520 2577 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 15 19:21:27.180351 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.844069 2577 server.go:310] "Adding debug handlers to kubelet server" Jul 15 19:21:27.180351 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:26.845416 2577 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 15 19:21:27.180351 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.848393 2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 15 19:21:27.180351 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.848431 2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 15 19:21:27.180351 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.849205 2577 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 15 19:21:27.180351 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.849215 2577 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 15 19:21:27.180351 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:26.848124 2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-62.ec2.internal.18c28cddd87d8403 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-62.ec2.internal,UID:ip-10-0-141-62.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-62.ec2.internal,},FirstTimestamp:2026-07-15 19:21:26.839903235 +0000 UTC m=+0.416904896,LastTimestamp:2026-07-15 19:21:26.839903235 +0000 UTC m=+0.416904896,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-62.ec2.internal,}" Jul 15 19:21:27.180351 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.849304 2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 15 19:21:27.180351 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.849370 2577 reconstruct.go:97] "Volume reconstruction finished" Jul 15 19:21:27.180351 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.849376 2577 reconciler.go:29] "Reconciler: start to sync state" Jul 15 19:21:27.180351 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:26.849822 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-62.ec2.internal\" not found" Jul 15 19:21:27.180351 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.850929 2577 factory.go:55] Registering systemd factory Jul 15 19:21:27.180351 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.850940 2577 factory.go:223] Registration of the systemd container factory successfully Jul 15 19:21:27.181666 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.852135 2577 factory.go:153] Registering CRI-O factory Jul 15 19:21:27.181666 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.852148 2577 factory.go:223] Registration of the crio container factory successfully Jul 15 19:21:27.181666 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.852219 2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 15 19:21:27.181666 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.852242 2577 factory.go:103] Registering Raw factory Jul 15 19:21:27.181666 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.852257 2577 manager.go:1196] Started watching for new ooms in manager Jul 15 19:21:27.181666 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.853009 2577 manager.go:349] Starting recovery of all containers Jul 15 19:21:27.181666 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.856900 2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 15 19:21:27.181666 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:26.858189 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 15 19:21:27.181666 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:26.858397 2577 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-62.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 15 19:21:27.181666 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.863488 2577 manager.go:354] Recovery completed Jul 15 19:21:27.181666 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.874552 2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-pv58t" Jul 15 19:21:27.181666 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.875203 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 19:21:27.181666 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.877135 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-62.ec2.internal" event="NodeHasSufficientMemory" Jul 15 19:21:27.181666 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.877172 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-62.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 19:21:27.181666 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.877189 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-62.ec2.internal" event="NodeHasSufficientPID" Jul 15 19:21:27.181666 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.877828 2577 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 15 19:21:27.181666 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.877839 2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 15 19:21:27.181666 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.877855 2577 state_mem.go:36] "Initialized new in-memory state store" Jul 15 19:21:27.181666 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.877862 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 19:21:27.181666 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.878560 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-62.ec2.internal" event="NodeHasSufficientMemory" Jul 15 19:21:27.181666 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.878589 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-62.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 19:21:27.181666 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.878606 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-62.ec2.internal" event="NodeHasSufficientPID" Jul 15 19:21:27.181666 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.879231 2577 policy_none.go:49] "None policy: Start" Jul 15 19:21:27.181666 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.879247 2577 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 15 19:21:27.181666 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.879279 2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 15 19:21:27.181666 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:26.879433 2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-62.ec2.internal.18c28cdddab5e1bc default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-62.ec2.internal,UID:ip-10-0-141-62.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-62.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-62.ec2.internal,},FirstTimestamp:2026-07-15 19:21:26.877151676 +0000 UTC m=+0.454153342,LastTimestamp:2026-07-15 19:21:26.877151676 +0000 UTC m=+0.454153342,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-62.ec2.internal,}" Jul 15 19:21:27.181666 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.880873 2577 policy_none.go:47] "Start" Jul 15 19:21:27.181666 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.882653 2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-pv58t" Jul 15 19:21:27.181666 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.932793 2577 manager.go:337] "Starting Device Plugin manager" Jul 15 19:21:27.181666 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:26.937813 2577 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 15 19:21:27.181666 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.937826 2577 server.go:87] "Starting device plugin registration server" Jul 15 19:21:27.181666 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.938147 2577 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 15 19:21:27.181666 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.938157 2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 15 19:21:27.181666 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.938261 2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 15 19:21:27.181666 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.938340 2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 15 19:21:27.181666 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:26.938349 2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 15 19:21:27.181666 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:26.938840 2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 15 19:21:27.181666 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:26.938884 2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-62.ec2.internal\" not found" Jul 15 19:21:27.181666 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:27.041339 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 19:21:27.182789 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:27.042462 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-62.ec2.internal" event="NodeHasSufficientMemory" Jul 15 19:21:27.182789 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:27.042507 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-62.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 19:21:27.182789 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:27.042521 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-62.ec2.internal" event="NodeHasSufficientPID" Jul 15 19:21:27.182789 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:27.042548 2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-62.ec2.internal" Jul 15 19:21:27.182789 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:27.055679 2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-62.ec2.internal" Jul 15 19:21:27.182789 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:27.055698 2577 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-62.ec2.internal\": node \"ip-10-0-141-62.ec2.internal\" not found" Jul 15 19:21:27.182789 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:27.085818 2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 15 19:21:27.182789 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:27.085853 2577 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 15 19:21:27.182789 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:27.085872 2577 kubelet.go:2448] "Starting kubelet main sync loop" Jul 15 19:21:27.182789 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:27.085902 2577 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 15 19:21:27.182789 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:27.089715 2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 19:21:27.182789 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:27.095877 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-62.ec2.internal\" not found" Jul 15 19:21:27.321486 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:27.186325 2577 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-62.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-62.ec2.internal"] Jul 15 19:21:27.321486 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:27.186409 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 19:21:27.321486 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:27.188079 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-62.ec2.internal" event="NodeHasSufficientMemory" Jul 15 19:21:27.321486 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:27.188108 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-62.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 19:21:27.321486 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:27.188118 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-62.ec2.internal" event="NodeHasSufficientPID" Jul 15 19:21:27.321486 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:27.189675 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 19:21:27.321486 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:27.189796 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-62.ec2.internal" Jul 15 19:21:27.321486 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:27.189822 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 19:21:27.321486 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:27.190388 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-62.ec2.internal" event="NodeHasSufficientMemory" Jul 15 19:21:27.321486 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:27.190419 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-62.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 19:21:27.321486 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:27.190430 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-62.ec2.internal" event="NodeHasSufficientPID" Jul 15 19:21:27.321486 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:27.190387 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-62.ec2.internal" event="NodeHasSufficientMemory" Jul 15 19:21:27.321486 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:27.190491 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-62.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 19:21:27.321486 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:27.190503 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-62.ec2.internal" event="NodeHasSufficientPID" Jul 15 19:21:27.321486 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:27.192263 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-62.ec2.internal" Jul 15 19:21:27.321486 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:27.192294 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 19:21:27.321486 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:27.193780 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-62.ec2.internal" event="NodeHasSufficientMemory" Jul 15 19:21:27.321486 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:27.193805 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-62.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 19:21:27.321486 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:27.193818 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-62.ec2.internal" event="NodeHasSufficientPID" Jul 15 19:21:27.321486 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:27.195960 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-62.ec2.internal\" not found" Jul 15 19:21:27.321486 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:27.228090 2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-62.ec2.internal\" not found" node="ip-10-0-141-62.ec2.internal" Jul 15 19:21:27.321486 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:27.232594 2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-62.ec2.internal\" not found" node="ip-10-0-141-62.ec2.internal" Jul 15 19:21:27.321486 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:27.251172 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/64ae407aa9b62ec31a4fe3be45950ba9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-62.ec2.internal\" (UID: \"64ae407aa9b62ec31a4fe3be45950ba9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-62.ec2.internal" Jul 15 19:21:27.321486 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:27.251198 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/64ae407aa9b62ec31a4fe3be45950ba9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-62.ec2.internal\" (UID: \"64ae407aa9b62ec31a4fe3be45950ba9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-62.ec2.internal" Jul 15 19:21:27.321486 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:27.251222 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/db478e14aba317d5490063a0b6ec4a2a-config\") pod \"kube-apiserver-proxy-ip-10-0-141-62.ec2.internal\" (UID: \"db478e14aba317d5490063a0b6ec4a2a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-62.ec2.internal" Jul 15 19:21:27.321486 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:27.296117 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-62.ec2.internal\" not found" Jul 15 19:21:27.509926 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:27.351485 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/db478e14aba317d5490063a0b6ec4a2a-config\") pod \"kube-apiserver-proxy-ip-10-0-141-62.ec2.internal\" (UID: \"db478e14aba317d5490063a0b6ec4a2a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-62.ec2.internal" Jul 15 19:21:27.509926 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:27.351522 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/64ae407aa9b62ec31a4fe3be45950ba9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-62.ec2.internal\" (UID: \"64ae407aa9b62ec31a4fe3be45950ba9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-62.ec2.internal" Jul 15 19:21:27.509926 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:27.351582 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/db478e14aba317d5490063a0b6ec4a2a-config\") pod \"kube-apiserver-proxy-ip-10-0-141-62.ec2.internal\" (UID: \"db478e14aba317d5490063a0b6ec4a2a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-62.ec2.internal" Jul 15 19:21:27.509926 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:27.351603 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/64ae407aa9b62ec31a4fe3be45950ba9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-62.ec2.internal\" (UID: \"64ae407aa9b62ec31a4fe3be45950ba9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-62.ec2.internal" Jul 15 19:21:27.509926 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:27.351703 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/64ae407aa9b62ec31a4fe3be45950ba9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-62.ec2.internal\" (UID: \"64ae407aa9b62ec31a4fe3be45950ba9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-62.ec2.internal" Jul 15 19:21:27.509926 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:27.351538 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/64ae407aa9b62ec31a4fe3be45950ba9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-62.ec2.internal\" (UID: \"64ae407aa9b62ec31a4fe3be45950ba9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-62.ec2.internal" Jul 15 19:21:27.509926 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:27.396729 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-62.ec2.internal\" not found" Jul 15 19:21:27.509926 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:27.499804 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-62.ec2.internal\" not found" Jul 15 19:21:27.660615 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:27.529677 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-62.ec2.internal" Jul 15 19:21:27.660615 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:27.535949 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-62.ec2.internal" Jul 15 19:21:27.660615 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:27.601533 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-62.ec2.internal\" not found" Jul 15 19:21:27.788566 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:27.701684 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-62.ec2.internal\" not found" Jul 15 19:21:27.792927 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:27.791635 2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 15 19:21:27.792927 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:27.791801 2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 15 19:21:27.868601 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:27.802703 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-62.ec2.internal\" not found" Jul 15 19:21:27.868601 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:27.850248 2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 19:21:27.950652 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:27.886602 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-14 19:16:27 +0000 UTC" deadline="2028-01-06 09:09:32.391626384 +0000 UTC" Jul 15 19:21:27.950652 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:27.886625 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12949h48m4.505004839s" Jul 15 19:21:27.950652 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:27.918148 2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 19:21:27.950652 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:27.950643 2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-62.ec2.internal" Jul 15 19:21:28.115167 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:27.971628 2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 15 19:21:28.115167 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:27.973113 2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-62.ec2.internal" Jul 15 19:21:28.115167 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:27.984235 2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 15 19:21:28.115167 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:27.995733 2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 19:21:28.414868 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:28.316688 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb478e14aba317d5490063a0b6ec4a2a.slice/crio-8d1a7510f658fc6fae58e8da4b0bae14eb76c4a60efe4cf25a259250ff61dc9c WatchSource:0}: Error finding container 8d1a7510f658fc6fae58e8da4b0bae14eb76c4a60efe4cf25a259250ff61dc9c: Status 404 returned error can't find the container with id 8d1a7510f658fc6fae58e8da4b0bae14eb76c4a60efe4cf25a259250ff61dc9c Jul 15 19:21:28.414868 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:28.317081 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64ae407aa9b62ec31a4fe3be45950ba9.slice/crio-056dfffd79ad34da047db20f3beef9787e50e2b50f05a2493f4ddc702969fae4 WatchSource:0}: Error finding container 056dfffd79ad34da047db20f3beef9787e50e2b50f05a2493f4ddc702969fae4: Status 404 returned error can't find the container with id 056dfffd79ad34da047db20f3beef9787e50e2b50f05a2493f4ddc702969fae4 Jul 15 19:21:28.414868 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.327161 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 19:21:28.840171 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.840146 2577 apiserver.go:52] "Watching apiserver" Jul 15 19:21:28.849438 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.849413 2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 15 19:21:28.850589 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.850566 2577 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 15 19:21:28.851799 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.851777 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-c8jtk","openshift-network-operator/iptables-alerter-hbtrt","openshift-ovn-kubernetes/ovnkube-node-fw6vl","kube-system/global-pull-secret-syncer-pn46g","kube-system/konnectivity-agent-xx47b","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-62.ec2.internal","openshift-multus/multus-48pqd","openshift-multus/multus-additional-cni-plugins-f7pg5","openshift-multus/network-metrics-daemon-cmqc9","openshift-network-diagnostics/network-check-target-5zk6t","kube-system/kube-apiserver-proxy-ip-10-0-141-62.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j99qj","openshift-cluster-node-tuning-operator/tuned-7m9g6","openshift-dns/node-resolver-9vzln"] Jul 15 19:21:28.854375 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.854358 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-48pqd" Jul 15 19:21:28.855757 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.855741 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7m9g6" Jul 15 19:21:28.857046 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.857025 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:28.859147 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.859126 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 15 19:21:28.859414 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.859280 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pn46g" Jul 15 19:21:28.859880 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.859310 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 15 19:21:28.859986 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.859360 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-nwzbq\"" Jul 15 19:21:28.860065 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.859474 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-cr4j7\"" Jul 15 19:21:28.860252 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.859590 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xx47b" Jul 15 19:21:28.860744 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:28.860709 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pn46g" podUID="05d64bcb-1ac1-47aa-a557-63ba6c2011ab" Jul 15 19:21:28.860834 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.860763 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 15 19:21:28.860834 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.860831 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 15 19:21:28.860945 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.860924 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 15 19:21:28.861355 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.861102 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 15 19:21:28.861355 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.861149 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9vzln" Jul 15 19:21:28.861792 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.861771 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 15 19:21:28.861923 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.861905 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 15 19:21:28.862324 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.862307 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 15 19:21:28.862532 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.862516 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-bmb8h\"" Jul 15 19:21:28.862859 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.862842 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 15 19:21:28.863035 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.863021 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 15 19:21:28.863210 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.863197 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmqc9" Jul 15 19:21:28.863276 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:28.863251 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmqc9" podUID="237b9375-100a-4d62-83f0-cb271ab8e0bb" Jul 15 19:21:28.863550 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.863529 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 15 19:21:28.864268 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.864252 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-c8jtk" Jul 15 19:21:28.865868 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.865849 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hbtrt" Jul 15 19:21:28.866869 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.866852 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zk6t" Jul 15 19:21:28.866933 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:28.866913 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zk6t" podUID="c90bd526-03fe-4c0c-9d9d-5bd777b6bd87" Jul 15 19:21:28.867867 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.867849 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-5s6xf\"" Jul 15 19:21:28.871631 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.868340 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-f7pg5" Jul 15 19:21:28.871631 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.868590 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 15 19:21:28.871631 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.868824 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 15 19:21:28.871631 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.869235 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-hmd7q\"" Jul 15 19:21:28.871631 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.869318 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 15 19:21:28.871631 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.869706 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 15 19:21:28.872118 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.871789 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j99qj" Jul 15 19:21:28.872714 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.872698 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 15 19:21:28.872714 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.872706 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 15 19:21:28.873345 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.872843 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 15 19:21:28.873345 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.872987 2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 15 19:21:28.873345 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.873065 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 15 19:21:28.873345 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.873083 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-cbmzq\"" Jul 15 19:21:28.873345 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.873158 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 15 19:21:28.873345 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.873158 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 15 19:21:28.873345 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.873180 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-d8gdb\"" Jul 15 19:21:28.878249 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.878177 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 15 19:21:28.878249 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.878193 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 15 19:21:28.878249 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.878225 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 15 19:21:28.878547 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.878317 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 15 19:21:28.878610 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.878593 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-t7tdn\"" Jul 15 19:21:28.878655 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.878619 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 15 19:21:28.878718 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.878619 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-pf446\"" Jul 15 19:21:28.892514 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.892494 2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-vz8nj" Jul 15 19:21:28.901998 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.901978 2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-vz8nj" Jul 15 19:21:28.951267 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.951248 2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 15 19:21:28.982887 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.982853 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-host-slash\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:28.983013 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.982894 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzmbr\" (UniqueName: \"kubernetes.io/projected/b6d81326-425e-4a61-a7db-8790d035bd50-kube-api-access-xzmbr\") pod \"node-ca-c8jtk\" (UID: \"b6d81326-425e-4a61-a7db-8790d035bd50\") " pod="openshift-image-registry/node-ca-c8jtk" Jul 15 19:21:28.983013 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.982912 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/05d64bcb-1ac1-47aa-a557-63ba6c2011ab-original-pull-secret\") pod \"global-pull-secret-syncer-pn46g\" (UID: \"05d64bcb-1ac1-47aa-a557-63ba6c2011ab\") " pod="kube-system/global-pull-secret-syncer-pn46g" Jul 15 19:21:28.983013 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.982934 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ec13993c-0b5b-456f-81cb-542da64f3d23-device-dir\") pod \"aws-ebs-csi-driver-node-j99qj\" (UID: \"ec13993c-0b5b-456f-81cb-542da64f3d23\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j99qj" Jul 15 19:21:28.983013 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.982956 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwqlh\" (UniqueName: \"kubernetes.io/projected/ec13993c-0b5b-456f-81cb-542da64f3d23-kube-api-access-nwqlh\") pod \"aws-ebs-csi-driver-node-j99qj\" (UID: \"ec13993c-0b5b-456f-81cb-542da64f3d23\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j99qj" Jul 15 19:21:28.983013 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.983006 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/71aa4ce2-c8ac-4224-9226-ba125d49f534-multus-socket-dir-parent\") pod \"multus-48pqd\" (UID: \"71aa4ce2-c8ac-4224-9226-ba125d49f534\") " pod="openshift-multus/multus-48pqd" Jul 15 19:21:28.983265 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.983039 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/71aa4ce2-c8ac-4224-9226-ba125d49f534-host-run-k8s-cni-cncf-io\") pod \"multus-48pqd\" (UID: \"71aa4ce2-c8ac-4224-9226-ba125d49f534\") " pod="openshift-multus/multus-48pqd" Jul 15 19:21:28.983265 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.983067 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/61c3693e-12af-4a9e-81d9-f66cc6520c73-etc-tuned\") pod \"tuned-7m9g6\" (UID: \"61c3693e-12af-4a9e-81d9-f66cc6520c73\") " pod="openshift-cluster-node-tuning-operator/tuned-7m9g6" Jul 15 19:21:28.983265 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.983090 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ec13993c-0b5b-456f-81cb-542da64f3d23-socket-dir\") pod \"aws-ebs-csi-driver-node-j99qj\" (UID: \"ec13993c-0b5b-456f-81cb-542da64f3d23\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j99qj" Jul 15 19:21:28.983265 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.983113 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/71aa4ce2-c8ac-4224-9226-ba125d49f534-system-cni-dir\") pod \"multus-48pqd\" (UID: \"71aa4ce2-c8ac-4224-9226-ba125d49f534\") " pod="openshift-multus/multus-48pqd" Jul 15 19:21:28.983265 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.983135 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/71aa4ce2-c8ac-4224-9226-ba125d49f534-hostroot\") pod \"multus-48pqd\" (UID: \"71aa4ce2-c8ac-4224-9226-ba125d49f534\") " pod="openshift-multus/multus-48pqd" Jul 15 19:21:28.983265 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.983159 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/71aa4ce2-c8ac-4224-9226-ba125d49f534-multus-daemon-config\") pod \"multus-48pqd\" (UID: \"71aa4ce2-c8ac-4224-9226-ba125d49f534\") " pod="openshift-multus/multus-48pqd" Jul 15 19:21:28.983265 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.983183 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-etc-openvswitch\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:28.983265 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.983227 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/71aa4ce2-c8ac-4224-9226-ba125d49f534-etc-kubernetes\") pod \"multus-48pqd\" (UID: \"71aa4ce2-c8ac-4224-9226-ba125d49f534\") " pod="openshift-multus/multus-48pqd" Jul 15 19:21:28.983265 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.983249 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pdxz\" (UniqueName: \"kubernetes.io/projected/23786db4-0338-4d64-9ea3-be6ebc44d74e-kube-api-access-5pdxz\") pod \"multus-additional-cni-plugins-f7pg5\" (UID: \"23786db4-0338-4d64-9ea3-be6ebc44d74e\") " pod="openshift-multus/multus-additional-cni-plugins-f7pg5" Jul 15 19:21:28.983735 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.983292 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqqm5\" (UniqueName: \"kubernetes.io/projected/237b9375-100a-4d62-83f0-cb271ab8e0bb-kube-api-access-pqqm5\") pod \"network-metrics-daemon-cmqc9\" (UID: \"237b9375-100a-4d62-83f0-cb271ab8e0bb\") " pod="openshift-multus/network-metrics-daemon-cmqc9" Jul 15 19:21:28.983735 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.983359 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/61c3693e-12af-4a9e-81d9-f66cc6520c73-etc-kubernetes\") pod \"tuned-7m9g6\" (UID: \"61c3693e-12af-4a9e-81d9-f66cc6520c73\") " pod="openshift-cluster-node-tuning-operator/tuned-7m9g6" Jul 15 19:21:28.983735 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.983402 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-ovn-node-metrics-cert\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:28.983735 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.983431 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/23786db4-0338-4d64-9ea3-be6ebc44d74e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-f7pg5\" (UID: \"23786db4-0338-4d64-9ea3-be6ebc44d74e\") " pod="openshift-multus/multus-additional-cni-plugins-f7pg5" Jul 15 19:21:28.983735 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.983460 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/299d8c27-2943-40f5-8810-67bcfcbbb3a2-hosts-file\") pod \"node-resolver-9vzln\" (UID: \"299d8c27-2943-40f5-8810-67bcfcbbb3a2\") " pod="openshift-dns/node-resolver-9vzln" Jul 15 19:21:28.983735 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.983481 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/61c3693e-12af-4a9e-81d9-f66cc6520c73-etc-systemd\") pod \"tuned-7m9g6\" (UID: \"61c3693e-12af-4a9e-81d9-f66cc6520c73\") " pod="openshift-cluster-node-tuning-operator/tuned-7m9g6" Jul 15 19:21:28.983735 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.983497 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/71aa4ce2-c8ac-4224-9226-ba125d49f534-host-run-multus-certs\") pod \"multus-48pqd\" (UID: \"71aa4ce2-c8ac-4224-9226-ba125d49f534\") " pod="openshift-multus/multus-48pqd" Jul 15 19:21:28.983735 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.983514 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61c3693e-12af-4a9e-81d9-f66cc6520c73-tmp\") pod \"tuned-7m9g6\" (UID: \"61c3693e-12af-4a9e-81d9-f66cc6520c73\") " pod="openshift-cluster-node-tuning-operator/tuned-7m9g6" Jul 15 19:21:28.983735 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.983537 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-node-log\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:28.983735 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.983563 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/23786db4-0338-4d64-9ea3-be6ebc44d74e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-f7pg5\" (UID: \"23786db4-0338-4d64-9ea3-be6ebc44d74e\") " pod="openshift-multus/multus-additional-cni-plugins-f7pg5" Jul 15 19:21:28.983735 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.983607 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/237b9375-100a-4d62-83f0-cb271ab8e0bb-metrics-certs\") pod \"network-metrics-daemon-cmqc9\" (UID: \"237b9375-100a-4d62-83f0-cb271ab8e0bb\") " pod="openshift-multus/network-metrics-daemon-cmqc9" Jul 15 19:21:28.983735 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.983638 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdd5x\" (UniqueName: \"kubernetes.io/projected/71aa4ce2-c8ac-4224-9226-ba125d49f534-kube-api-access-tdd5x\") pod \"multus-48pqd\" (UID: \"71aa4ce2-c8ac-4224-9226-ba125d49f534\") " pod="openshift-multus/multus-48pqd" Jul 15 19:21:28.983735 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.983662 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/61c3693e-12af-4a9e-81d9-f66cc6520c73-etc-sysctl-conf\") pod \"tuned-7m9g6\" (UID: \"61c3693e-12af-4a9e-81d9-f66cc6520c73\") " pod="openshift-cluster-node-tuning-operator/tuned-7m9g6" Jul 15 19:21:28.983735 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.983700 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/61c3693e-12af-4a9e-81d9-f66cc6520c73-host\") pod \"tuned-7m9g6\" (UID: \"61c3693e-12af-4a9e-81d9-f66cc6520c73\") " pod="openshift-cluster-node-tuning-operator/tuned-7m9g6" Jul 15 19:21:28.983735 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.983728 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-run-systemd\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:28.984289 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.983752 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b6d81326-425e-4a61-a7db-8790d035bd50-host\") pod \"node-ca-c8jtk\" (UID: \"b6d81326-425e-4a61-a7db-8790d035bd50\") " pod="openshift-image-registry/node-ca-c8jtk" Jul 15 19:21:28.984289 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.983777 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ec13993c-0b5b-456f-81cb-542da64f3d23-kubelet-dir\") pod \"aws-ebs-csi-driver-node-j99qj\" (UID: \"ec13993c-0b5b-456f-81cb-542da64f3d23\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j99qj" Jul 15 19:21:28.984289 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.983803 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/61c3693e-12af-4a9e-81d9-f66cc6520c73-etc-sysctl-d\") pod \"tuned-7m9g6\" (UID: \"61c3693e-12af-4a9e-81d9-f66cc6520c73\") " pod="openshift-cluster-node-tuning-operator/tuned-7m9g6" Jul 15 19:21:28.984289 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.983820 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/61c3693e-12af-4a9e-81d9-f66cc6520c73-lib-modules\") pod \"tuned-7m9g6\" (UID: \"61c3693e-12af-4a9e-81d9-f66cc6520c73\") " pod="openshift-cluster-node-tuning-operator/tuned-7m9g6" Jul 15 19:21:28.984289 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.983834 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1820cfdf-910d-42da-bafb-fb076d565b78-agent-certs\") pod \"konnectivity-agent-xx47b\" (UID: \"1820cfdf-910d-42da-bafb-fb076d565b78\") " pod="kube-system/konnectivity-agent-xx47b" Jul 15 19:21:28.984289 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.983860 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1820cfdf-910d-42da-bafb-fb076d565b78-konnectivity-ca\") pod \"konnectivity-agent-xx47b\" (UID: \"1820cfdf-910d-42da-bafb-fb076d565b78\") " pod="kube-system/konnectivity-agent-xx47b" Jul 15 19:21:28.984289 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.983886 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/35abc39e-4584-4a00-b15e-5c25c1f8ce41-iptables-alerter-script\") pod \"iptables-alerter-hbtrt\" (UID: \"35abc39e-4584-4a00-b15e-5c25c1f8ce41\") " pod="openshift-network-operator/iptables-alerter-hbtrt" Jul 15 19:21:28.984289 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.983909 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/23786db4-0338-4d64-9ea3-be6ebc44d74e-cnibin\") pod \"multus-additional-cni-plugins-f7pg5\" (UID: \"23786db4-0338-4d64-9ea3-be6ebc44d74e\") " pod="openshift-multus/multus-additional-cni-plugins-f7pg5" Jul 15 19:21:28.984289 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.983950 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/23786db4-0338-4d64-9ea3-be6ebc44d74e-cni-binary-copy\") pod \"multus-additional-cni-plugins-f7pg5\" (UID: \"23786db4-0338-4d64-9ea3-be6ebc44d74e\") " pod="openshift-multus/multus-additional-cni-plugins-f7pg5" Jul 15 19:21:28.984289 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.983978 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/71aa4ce2-c8ac-4224-9226-ba125d49f534-multus-conf-dir\") pod \"multus-48pqd\" (UID: \"71aa4ce2-c8ac-4224-9226-ba125d49f534\") " pod="openshift-multus/multus-48pqd" Jul 15 19:21:28.984289 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.983999 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-host-cni-netd\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:28.984289 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.984026 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8b5qx\" (UniqueName: \"kubernetes.io/projected/35abc39e-4584-4a00-b15e-5c25c1f8ce41-kube-api-access-8b5qx\") pod \"iptables-alerter-hbtrt\" (UID: \"35abc39e-4584-4a00-b15e-5c25c1f8ce41\") " pod="openshift-network-operator/iptables-alerter-hbtrt" Jul 15 19:21:28.984289 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.984052 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/05d64bcb-1ac1-47aa-a557-63ba6c2011ab-dbus\") pod \"global-pull-secret-syncer-pn46g\" (UID: \"05d64bcb-1ac1-47aa-a557-63ba6c2011ab\") " pod="kube-system/global-pull-secret-syncer-pn46g" Jul 15 19:21:28.984289 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.984071 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfgwz\" (UniqueName: \"kubernetes.io/projected/299d8c27-2943-40f5-8810-67bcfcbbb3a2-kube-api-access-nfgwz\") pod \"node-resolver-9vzln\" (UID: \"299d8c27-2943-40f5-8810-67bcfcbbb3a2\") " pod="openshift-dns/node-resolver-9vzln" Jul 15 19:21:28.984289 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.984122 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8f8j\" (UniqueName: \"kubernetes.io/projected/c90bd526-03fe-4c0c-9d9d-5bd777b6bd87-kube-api-access-z8f8j\") pod \"network-check-target-5zk6t\" (UID: \"c90bd526-03fe-4c0c-9d9d-5bd777b6bd87\") " pod="openshift-network-diagnostics/network-check-target-5zk6t" Jul 15 19:21:28.984289 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.984162 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/61c3693e-12af-4a9e-81d9-f66cc6520c73-sys\") pod \"tuned-7m9g6\" (UID: \"61c3693e-12af-4a9e-81d9-f66cc6520c73\") " pod="openshift-cluster-node-tuning-operator/tuned-7m9g6" Jul 15 19:21:28.985093 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.984185 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/71aa4ce2-c8ac-4224-9226-ba125d49f534-host-var-lib-kubelet\") pod \"multus-48pqd\" (UID: \"71aa4ce2-c8ac-4224-9226-ba125d49f534\") " pod="openshift-multus/multus-48pqd" Jul 15 19:21:28.985093 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.984205 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/61c3693e-12af-4a9e-81d9-f66cc6520c73-var-lib-kubelet\") pod \"tuned-7m9g6\" (UID: \"61c3693e-12af-4a9e-81d9-f66cc6520c73\") " pod="openshift-cluster-node-tuning-operator/tuned-7m9g6" Jul 15 19:21:28.985093 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.984221 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-host-kubelet\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:28.985093 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.984238 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-run-ovn\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:28.985093 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.984256 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b6d81326-425e-4a61-a7db-8790d035bd50-serviceca\") pod \"node-ca-c8jtk\" (UID: \"b6d81326-425e-4a61-a7db-8790d035bd50\") " pod="openshift-image-registry/node-ca-c8jtk" Jul 15 19:21:28.985093 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.984272 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/23786db4-0338-4d64-9ea3-be6ebc44d74e-os-release\") pod \"multus-additional-cni-plugins-f7pg5\" (UID: \"23786db4-0338-4d64-9ea3-be6ebc44d74e\") " pod="openshift-multus/multus-additional-cni-plugins-f7pg5" Jul 15 19:21:28.985093 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.984295 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ec13993c-0b5b-456f-81cb-542da64f3d23-sys-fs\") pod \"aws-ebs-csi-driver-node-j99qj\" (UID: \"ec13993c-0b5b-456f-81cb-542da64f3d23\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j99qj" Jul 15 19:21:28.985093 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.984312 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/71aa4ce2-c8ac-4224-9226-ba125d49f534-cnibin\") pod \"multus-48pqd\" (UID: \"71aa4ce2-c8ac-4224-9226-ba125d49f534\") " pod="openshift-multus/multus-48pqd" Jul 15 19:21:28.985093 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.984349 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/71aa4ce2-c8ac-4224-9226-ba125d49f534-cni-binary-copy\") pod \"multus-48pqd\" (UID: \"71aa4ce2-c8ac-4224-9226-ba125d49f534\") " pod="openshift-multus/multus-48pqd" Jul 15 19:21:28.985093 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.984384 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-host-cni-bin\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:28.985093 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.984404 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-ovnkube-config\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:28.985093 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.984441 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/71aa4ce2-c8ac-4224-9226-ba125d49f534-multus-cni-dir\") pod \"multus-48pqd\" (UID: \"71aa4ce2-c8ac-4224-9226-ba125d49f534\") " pod="openshift-multus/multus-48pqd" Jul 15 19:21:28.985093 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.984465 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/23786db4-0338-4d64-9ea3-be6ebc44d74e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-f7pg5\" (UID: \"23786db4-0338-4d64-9ea3-be6ebc44d74e\") " pod="openshift-multus/multus-additional-cni-plugins-f7pg5" Jul 15 19:21:28.985093 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.984489 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/299d8c27-2943-40f5-8810-67bcfcbbb3a2-tmp-dir\") pod \"node-resolver-9vzln\" (UID: \"299d8c27-2943-40f5-8810-67bcfcbbb3a2\") " pod="openshift-dns/node-resolver-9vzln" Jul 15 19:21:28.985093 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.984511 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxgbr\" (UniqueName: \"kubernetes.io/projected/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-kube-api-access-sxgbr\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:28.985093 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.984534 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/61c3693e-12af-4a9e-81d9-f66cc6520c73-etc-modprobe-d\") pod \"tuned-7m9g6\" (UID: \"61c3693e-12af-4a9e-81d9-f66cc6520c73\") " pod="openshift-cluster-node-tuning-operator/tuned-7m9g6" Jul 15 19:21:28.985093 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.984560 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p55zx\" (UniqueName: \"kubernetes.io/projected/61c3693e-12af-4a9e-81d9-f66cc6520c73-kube-api-access-p55zx\") pod \"tuned-7m9g6\" (UID: \"61c3693e-12af-4a9e-81d9-f66cc6520c73\") " pod="openshift-cluster-node-tuning-operator/tuned-7m9g6" Jul 15 19:21:28.985620 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.984588 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-systemd-units\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:28.985620 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.984617 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-host-run-netns\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:28.985620 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.984640 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-host-run-ovn-kubernetes\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:28.985620 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.984675 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-ovnkube-script-lib\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:28.985620 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.984713 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/35abc39e-4584-4a00-b15e-5c25c1f8ce41-host-slash\") pod \"iptables-alerter-hbtrt\" (UID: \"35abc39e-4584-4a00-b15e-5c25c1f8ce41\") " pod="openshift-network-operator/iptables-alerter-hbtrt" Jul 15 19:21:28.985620 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.984742 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-env-overrides\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:28.985620 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.984769 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/61c3693e-12af-4a9e-81d9-f66cc6520c73-run\") pod \"tuned-7m9g6\" (UID: \"61c3693e-12af-4a9e-81d9-f66cc6520c73\") " pod="openshift-cluster-node-tuning-operator/tuned-7m9g6" Jul 15 19:21:28.985620 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.984783 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/71aa4ce2-c8ac-4224-9226-ba125d49f534-os-release\") pod \"multus-48pqd\" (UID: \"71aa4ce2-c8ac-4224-9226-ba125d49f534\") " pod="openshift-multus/multus-48pqd" Jul 15 19:21:28.985620 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.984803 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/71aa4ce2-c8ac-4224-9226-ba125d49f534-host-run-netns\") pod \"multus-48pqd\" (UID: \"71aa4ce2-c8ac-4224-9226-ba125d49f534\") " pod="openshift-multus/multus-48pqd" Jul 15 19:21:28.985620 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.984827 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/71aa4ce2-c8ac-4224-9226-ba125d49f534-host-var-lib-cni-multus\") pod \"multus-48pqd\" (UID: \"71aa4ce2-c8ac-4224-9226-ba125d49f534\") " pod="openshift-multus/multus-48pqd" Jul 15 19:21:28.985620 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.984850 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-var-lib-openvswitch\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:28.985620 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.984868 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-run-openvswitch\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:28.985620 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.984887 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-log-socket\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:28.985620 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.984914 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/71aa4ce2-c8ac-4224-9226-ba125d49f534-host-var-lib-cni-bin\") pod \"multus-48pqd\" (UID: \"71aa4ce2-c8ac-4224-9226-ba125d49f534\") " pod="openshift-multus/multus-48pqd" Jul 15 19:21:28.985620 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.984935 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/05d64bcb-1ac1-47aa-a557-63ba6c2011ab-kubelet-config\") pod \"global-pull-secret-syncer-pn46g\" (UID: \"05d64bcb-1ac1-47aa-a557-63ba6c2011ab\") " pod="kube-system/global-pull-secret-syncer-pn46g" Jul 15 19:21:28.985620 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.984962 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ec13993c-0b5b-456f-81cb-542da64f3d23-etc-selinux\") pod \"aws-ebs-csi-driver-node-j99qj\" (UID: \"ec13993c-0b5b-456f-81cb-542da64f3d23\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j99qj" Jul 15 19:21:28.986175 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.984991 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/61c3693e-12af-4a9e-81d9-f66cc6520c73-etc-sysconfig\") pod \"tuned-7m9g6\" (UID: \"61c3693e-12af-4a9e-81d9-f66cc6520c73\") " pod="openshift-cluster-node-tuning-operator/tuned-7m9g6" Jul 15 19:21:28.986175 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.985013 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:28.986175 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.985035 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/23786db4-0338-4d64-9ea3-be6ebc44d74e-system-cni-dir\") pod \"multus-additional-cni-plugins-f7pg5\" (UID: \"23786db4-0338-4d64-9ea3-be6ebc44d74e\") " pod="openshift-multus/multus-additional-cni-plugins-f7pg5" Jul 15 19:21:28.986175 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:28.985057 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ec13993c-0b5b-456f-81cb-542da64f3d23-registration-dir\") pod \"aws-ebs-csi-driver-node-j99qj\" (UID: \"ec13993c-0b5b-456f-81cb-542da64f3d23\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j99qj" Jul 15 19:21:29.085982 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.085896 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/71aa4ce2-c8ac-4224-9226-ba125d49f534-system-cni-dir\") pod \"multus-48pqd\" (UID: \"71aa4ce2-c8ac-4224-9226-ba125d49f534\") " pod="openshift-multus/multus-48pqd" Jul 15 19:21:29.085982 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.085939 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/71aa4ce2-c8ac-4224-9226-ba125d49f534-hostroot\") pod \"multus-48pqd\" (UID: \"71aa4ce2-c8ac-4224-9226-ba125d49f534\") " pod="openshift-multus/multus-48pqd" Jul 15 19:21:29.085982 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.085967 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/71aa4ce2-c8ac-4224-9226-ba125d49f534-multus-daemon-config\") pod \"multus-48pqd\" (UID: \"71aa4ce2-c8ac-4224-9226-ba125d49f534\") " pod="openshift-multus/multus-48pqd" Jul 15 19:21:29.086257 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.085990 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-etc-openvswitch\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:29.086257 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.086011 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/71aa4ce2-c8ac-4224-9226-ba125d49f534-etc-kubernetes\") pod \"multus-48pqd\" (UID: \"71aa4ce2-c8ac-4224-9226-ba125d49f534\") " pod="openshift-multus/multus-48pqd" Jul 15 19:21:29.086257 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.086034 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pdxz\" (UniqueName: \"kubernetes.io/projected/23786db4-0338-4d64-9ea3-be6ebc44d74e-kube-api-access-5pdxz\") pod \"multus-additional-cni-plugins-f7pg5\" (UID: \"23786db4-0338-4d64-9ea3-be6ebc44d74e\") " pod="openshift-multus/multus-additional-cni-plugins-f7pg5" Jul 15 19:21:29.086257 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.086056 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqqm5\" (UniqueName: \"kubernetes.io/projected/237b9375-100a-4d62-83f0-cb271ab8e0bb-kube-api-access-pqqm5\") pod \"network-metrics-daemon-cmqc9\" (UID: \"237b9375-100a-4d62-83f0-cb271ab8e0bb\") " pod="openshift-multus/network-metrics-daemon-cmqc9" Jul 15 19:21:29.086257 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.086077 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/61c3693e-12af-4a9e-81d9-f66cc6520c73-etc-kubernetes\") pod \"tuned-7m9g6\" (UID: \"61c3693e-12af-4a9e-81d9-f66cc6520c73\") " pod="openshift-cluster-node-tuning-operator/tuned-7m9g6" Jul 15 19:21:29.086257 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.086099 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-ovn-node-metrics-cert\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:29.086257 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.086122 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/23786db4-0338-4d64-9ea3-be6ebc44d74e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-f7pg5\" (UID: \"23786db4-0338-4d64-9ea3-be6ebc44d74e\") " pod="openshift-multus/multus-additional-cni-plugins-f7pg5" Jul 15 19:21:29.086257 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.086143 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/299d8c27-2943-40f5-8810-67bcfcbbb3a2-hosts-file\") pod \"node-resolver-9vzln\" (UID: \"299d8c27-2943-40f5-8810-67bcfcbbb3a2\") " pod="openshift-dns/node-resolver-9vzln" Jul 15 19:21:29.086257 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.086166 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/61c3693e-12af-4a9e-81d9-f66cc6520c73-etc-systemd\") pod \"tuned-7m9g6\" (UID: \"61c3693e-12af-4a9e-81d9-f66cc6520c73\") " pod="openshift-cluster-node-tuning-operator/tuned-7m9g6" Jul 15 19:21:29.086257 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.086191 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/71aa4ce2-c8ac-4224-9226-ba125d49f534-host-run-multus-certs\") pod \"multus-48pqd\" (UID: \"71aa4ce2-c8ac-4224-9226-ba125d49f534\") " pod="openshift-multus/multus-48pqd" Jul 15 19:21:29.086257 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.086216 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61c3693e-12af-4a9e-81d9-f66cc6520c73-tmp\") pod \"tuned-7m9g6\" (UID: \"61c3693e-12af-4a9e-81d9-f66cc6520c73\") " pod="openshift-cluster-node-tuning-operator/tuned-7m9g6" Jul 15 19:21:29.086257 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.086239 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-node-log\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:29.086862 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.086263 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/23786db4-0338-4d64-9ea3-be6ebc44d74e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-f7pg5\" (UID: \"23786db4-0338-4d64-9ea3-be6ebc44d74e\") " pod="openshift-multus/multus-additional-cni-plugins-f7pg5" Jul 15 19:21:29.086862 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.086287 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/237b9375-100a-4d62-83f0-cb271ab8e0bb-metrics-certs\") pod \"network-metrics-daemon-cmqc9\" (UID: \"237b9375-100a-4d62-83f0-cb271ab8e0bb\") " pod="openshift-multus/network-metrics-daemon-cmqc9" Jul 15 19:21:29.086862 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.086320 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdd5x\" (UniqueName: \"kubernetes.io/projected/71aa4ce2-c8ac-4224-9226-ba125d49f534-kube-api-access-tdd5x\") pod \"multus-48pqd\" (UID: \"71aa4ce2-c8ac-4224-9226-ba125d49f534\") " pod="openshift-multus/multus-48pqd" Jul 15 19:21:29.086862 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.086364 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/61c3693e-12af-4a9e-81d9-f66cc6520c73-etc-sysctl-conf\") pod \"tuned-7m9g6\" (UID: \"61c3693e-12af-4a9e-81d9-f66cc6520c73\") " pod="openshift-cluster-node-tuning-operator/tuned-7m9g6" Jul 15 19:21:29.086862 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.086387 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/61c3693e-12af-4a9e-81d9-f66cc6520c73-host\") pod \"tuned-7m9g6\" (UID: \"61c3693e-12af-4a9e-81d9-f66cc6520c73\") " pod="openshift-cluster-node-tuning-operator/tuned-7m9g6" Jul 15 19:21:29.086862 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.086409 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-run-systemd\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:29.086862 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.086433 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b6d81326-425e-4a61-a7db-8790d035bd50-host\") pod \"node-ca-c8jtk\" (UID: \"b6d81326-425e-4a61-a7db-8790d035bd50\") " pod="openshift-image-registry/node-ca-c8jtk" Jul 15 19:21:29.086862 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.086456 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ec13993c-0b5b-456f-81cb-542da64f3d23-kubelet-dir\") pod \"aws-ebs-csi-driver-node-j99qj\" (UID: \"ec13993c-0b5b-456f-81cb-542da64f3d23\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j99qj" Jul 15 19:21:29.086862 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.086481 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/61c3693e-12af-4a9e-81d9-f66cc6520c73-etc-sysctl-d\") pod \"tuned-7m9g6\" (UID: \"61c3693e-12af-4a9e-81d9-f66cc6520c73\") " pod="openshift-cluster-node-tuning-operator/tuned-7m9g6" Jul 15 19:21:29.086862 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.086506 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/61c3693e-12af-4a9e-81d9-f66cc6520c73-lib-modules\") pod \"tuned-7m9g6\" (UID: \"61c3693e-12af-4a9e-81d9-f66cc6520c73\") " pod="openshift-cluster-node-tuning-operator/tuned-7m9g6" Jul 15 19:21:29.086862 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.086530 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1820cfdf-910d-42da-bafb-fb076d565b78-agent-certs\") pod \"konnectivity-agent-xx47b\" (UID: \"1820cfdf-910d-42da-bafb-fb076d565b78\") " pod="kube-system/konnectivity-agent-xx47b" Jul 15 19:21:29.086862 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.086552 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1820cfdf-910d-42da-bafb-fb076d565b78-konnectivity-ca\") pod \"konnectivity-agent-xx47b\" (UID: \"1820cfdf-910d-42da-bafb-fb076d565b78\") " pod="kube-system/konnectivity-agent-xx47b" Jul 15 19:21:29.086862 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.086577 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/35abc39e-4584-4a00-b15e-5c25c1f8ce41-iptables-alerter-script\") pod \"iptables-alerter-hbtrt\" (UID: \"35abc39e-4584-4a00-b15e-5c25c1f8ce41\") " pod="openshift-network-operator/iptables-alerter-hbtrt" Jul 15 19:21:29.086862 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.086602 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/23786db4-0338-4d64-9ea3-be6ebc44d74e-cnibin\") pod \"multus-additional-cni-plugins-f7pg5\" (UID: \"23786db4-0338-4d64-9ea3-be6ebc44d74e\") " pod="openshift-multus/multus-additional-cni-plugins-f7pg5" Jul 15 19:21:29.086862 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.086629 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/23786db4-0338-4d64-9ea3-be6ebc44d74e-cni-binary-copy\") pod \"multus-additional-cni-plugins-f7pg5\" (UID: \"23786db4-0338-4d64-9ea3-be6ebc44d74e\") " pod="openshift-multus/multus-additional-cni-plugins-f7pg5" Jul 15 19:21:29.086862 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.086653 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/71aa4ce2-c8ac-4224-9226-ba125d49f534-multus-conf-dir\") pod \"multus-48pqd\" (UID: \"71aa4ce2-c8ac-4224-9226-ba125d49f534\") " pod="openshift-multus/multus-48pqd" Jul 15 19:21:29.086862 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.086677 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-host-cni-netd\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:29.087686 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.086701 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8b5qx\" (UniqueName: \"kubernetes.io/projected/35abc39e-4584-4a00-b15e-5c25c1f8ce41-kube-api-access-8b5qx\") pod \"iptables-alerter-hbtrt\" (UID: \"35abc39e-4584-4a00-b15e-5c25c1f8ce41\") " pod="openshift-network-operator/iptables-alerter-hbtrt" Jul 15 19:21:29.087686 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.086735 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/05d64bcb-1ac1-47aa-a557-63ba6c2011ab-dbus\") pod \"global-pull-secret-syncer-pn46g\" (UID: \"05d64bcb-1ac1-47aa-a557-63ba6c2011ab\") " pod="kube-system/global-pull-secret-syncer-pn46g" Jul 15 19:21:29.087686 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.086764 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfgwz\" (UniqueName: \"kubernetes.io/projected/299d8c27-2943-40f5-8810-67bcfcbbb3a2-kube-api-access-nfgwz\") pod \"node-resolver-9vzln\" (UID: \"299d8c27-2943-40f5-8810-67bcfcbbb3a2\") " pod="openshift-dns/node-resolver-9vzln" Jul 15 19:21:29.087686 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.086792 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8f8j\" (UniqueName: \"kubernetes.io/projected/c90bd526-03fe-4c0c-9d9d-5bd777b6bd87-kube-api-access-z8f8j\") pod \"network-check-target-5zk6t\" (UID: \"c90bd526-03fe-4c0c-9d9d-5bd777b6bd87\") " pod="openshift-network-diagnostics/network-check-target-5zk6t" Jul 15 19:21:29.087686 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.086816 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/61c3693e-12af-4a9e-81d9-f66cc6520c73-sys\") pod \"tuned-7m9g6\" (UID: \"61c3693e-12af-4a9e-81d9-f66cc6520c73\") " pod="openshift-cluster-node-tuning-operator/tuned-7m9g6" Jul 15 19:21:29.087686 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.086843 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/71aa4ce2-c8ac-4224-9226-ba125d49f534-host-var-lib-kubelet\") pod \"multus-48pqd\" (UID: \"71aa4ce2-c8ac-4224-9226-ba125d49f534\") " pod="openshift-multus/multus-48pqd" Jul 15 19:21:29.087686 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.086867 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/61c3693e-12af-4a9e-81d9-f66cc6520c73-var-lib-kubelet\") pod \"tuned-7m9g6\" (UID: \"61c3693e-12af-4a9e-81d9-f66cc6520c73\") " pod="openshift-cluster-node-tuning-operator/tuned-7m9g6" Jul 15 19:21:29.087686 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.086897 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-host-kubelet\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:29.087686 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.086923 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-run-ovn\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:29.087686 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.086948 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b6d81326-425e-4a61-a7db-8790d035bd50-serviceca\") pod \"node-ca-c8jtk\" (UID: \"b6d81326-425e-4a61-a7db-8790d035bd50\") " pod="openshift-image-registry/node-ca-c8jtk" Jul 15 19:21:29.087686 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.086973 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/23786db4-0338-4d64-9ea3-be6ebc44d74e-os-release\") pod \"multus-additional-cni-plugins-f7pg5\" (UID: \"23786db4-0338-4d64-9ea3-be6ebc44d74e\") " pod="openshift-multus/multus-additional-cni-plugins-f7pg5" Jul 15 19:21:29.087686 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.086997 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ec13993c-0b5b-456f-81cb-542da64f3d23-sys-fs\") pod \"aws-ebs-csi-driver-node-j99qj\" (UID: \"ec13993c-0b5b-456f-81cb-542da64f3d23\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j99qj" Jul 15 19:21:29.087686 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.087022 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/71aa4ce2-c8ac-4224-9226-ba125d49f534-cnibin\") pod \"multus-48pqd\" (UID: \"71aa4ce2-c8ac-4224-9226-ba125d49f534\") " pod="openshift-multus/multus-48pqd" Jul 15 19:21:29.087686 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.087053 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/71aa4ce2-c8ac-4224-9226-ba125d49f534-cni-binary-copy\") pod \"multus-48pqd\" (UID: \"71aa4ce2-c8ac-4224-9226-ba125d49f534\") " pod="openshift-multus/multus-48pqd" Jul 15 19:21:29.087686 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.087078 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-host-cni-bin\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:29.087686 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.087103 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-ovnkube-config\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:29.087686 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.087126 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/71aa4ce2-c8ac-4224-9226-ba125d49f534-multus-cni-dir\") pod \"multus-48pqd\" (UID: \"71aa4ce2-c8ac-4224-9226-ba125d49f534\") " pod="openshift-multus/multus-48pqd" Jul 15 19:21:29.088548 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.087151 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/23786db4-0338-4d64-9ea3-be6ebc44d74e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-f7pg5\" (UID: \"23786db4-0338-4d64-9ea3-be6ebc44d74e\") " pod="openshift-multus/multus-additional-cni-plugins-f7pg5" Jul 15 19:21:29.088548 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.087176 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/299d8c27-2943-40f5-8810-67bcfcbbb3a2-tmp-dir\") pod \"node-resolver-9vzln\" (UID: \"299d8c27-2943-40f5-8810-67bcfcbbb3a2\") " pod="openshift-dns/node-resolver-9vzln" Jul 15 19:21:29.088548 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.087202 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxgbr\" (UniqueName: \"kubernetes.io/projected/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-kube-api-access-sxgbr\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:29.088548 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.087226 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/61c3693e-12af-4a9e-81d9-f66cc6520c73-etc-modprobe-d\") pod \"tuned-7m9g6\" (UID: \"61c3693e-12af-4a9e-81d9-f66cc6520c73\") " pod="openshift-cluster-node-tuning-operator/tuned-7m9g6" Jul 15 19:21:29.088548 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.087252 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p55zx\" (UniqueName: \"kubernetes.io/projected/61c3693e-12af-4a9e-81d9-f66cc6520c73-kube-api-access-p55zx\") pod \"tuned-7m9g6\" (UID: \"61c3693e-12af-4a9e-81d9-f66cc6520c73\") " pod="openshift-cluster-node-tuning-operator/tuned-7m9g6" Jul 15 19:21:29.088548 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.087278 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-systemd-units\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:29.088548 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.087305 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-host-run-netns\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:29.088548 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.087331 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-host-run-ovn-kubernetes\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:29.088548 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.087375 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-ovnkube-script-lib\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:29.088548 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.087401 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/35abc39e-4584-4a00-b15e-5c25c1f8ce41-host-slash\") pod \"iptables-alerter-hbtrt\" (UID: \"35abc39e-4584-4a00-b15e-5c25c1f8ce41\") " pod="openshift-network-operator/iptables-alerter-hbtrt" Jul 15 19:21:29.088548 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.087427 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-env-overrides\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:29.088548 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.087452 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/61c3693e-12af-4a9e-81d9-f66cc6520c73-run\") pod \"tuned-7m9g6\" (UID: \"61c3693e-12af-4a9e-81d9-f66cc6520c73\") " pod="openshift-cluster-node-tuning-operator/tuned-7m9g6" Jul 15 19:21:29.088548 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.087478 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/71aa4ce2-c8ac-4224-9226-ba125d49f534-os-release\") pod \"multus-48pqd\" (UID: \"71aa4ce2-c8ac-4224-9226-ba125d49f534\") " pod="openshift-multus/multus-48pqd" Jul 15 19:21:29.088548 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.087504 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/71aa4ce2-c8ac-4224-9226-ba125d49f534-host-run-netns\") pod \"multus-48pqd\" (UID: \"71aa4ce2-c8ac-4224-9226-ba125d49f534\") " pod="openshift-multus/multus-48pqd" Jul 15 19:21:29.088548 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.087530 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/71aa4ce2-c8ac-4224-9226-ba125d49f534-host-var-lib-cni-multus\") pod \"multus-48pqd\" (UID: \"71aa4ce2-c8ac-4224-9226-ba125d49f534\") " pod="openshift-multus/multus-48pqd" Jul 15 19:21:29.088548 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.087555 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-var-lib-openvswitch\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:29.088548 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.087582 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-run-openvswitch\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:29.089329 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.087608 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-log-socket\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:29.089329 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.087635 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/71aa4ce2-c8ac-4224-9226-ba125d49f534-host-var-lib-cni-bin\") pod \"multus-48pqd\" (UID: \"71aa4ce2-c8ac-4224-9226-ba125d49f534\") " pod="openshift-multus/multus-48pqd" Jul 15 19:21:29.089329 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.087698 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/71aa4ce2-c8ac-4224-9226-ba125d49f534-host-var-lib-cni-bin\") pod \"multus-48pqd\" (UID: \"71aa4ce2-c8ac-4224-9226-ba125d49f534\") " pod="openshift-multus/multus-48pqd" Jul 15 19:21:29.089329 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.087858 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/05d64bcb-1ac1-47aa-a557-63ba6c2011ab-dbus\") pod \"global-pull-secret-syncer-pn46g\" (UID: \"05d64bcb-1ac1-47aa-a557-63ba6c2011ab\") " pod="kube-system/global-pull-secret-syncer-pn46g" Jul 15 19:21:29.089329 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:29.087901 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:21:29.089329 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.087903 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/23786db4-0338-4d64-9ea3-be6ebc44d74e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-f7pg5\" (UID: \"23786db4-0338-4d64-9ea3-be6ebc44d74e\") " pod="openshift-multus/multus-additional-cni-plugins-f7pg5" Jul 15 19:21:29.089329 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:29.087965 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/237b9375-100a-4d62-83f0-cb271ab8e0bb-metrics-certs podName:237b9375-100a-4d62-83f0-cb271ab8e0bb nodeName:}" failed. No retries permitted until 2026-07-15 19:21:29.587949127 +0000 UTC m=+3.164950777 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/237b9375-100a-4d62-83f0-cb271ab8e0bb-metrics-certs") pod "network-metrics-daemon-cmqc9" (UID: "237b9375-100a-4d62-83f0-cb271ab8e0bb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:21:29.089329 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.087972 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/71aa4ce2-c8ac-4224-9226-ba125d49f534-system-cni-dir\") pod \"multus-48pqd\" (UID: \"71aa4ce2-c8ac-4224-9226-ba125d49f534\") " pod="openshift-multus/multus-48pqd" Jul 15 19:21:29.089329 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.088015 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/71aa4ce2-c8ac-4224-9226-ba125d49f534-hostroot\") pod \"multus-48pqd\" (UID: \"71aa4ce2-c8ac-4224-9226-ba125d49f534\") " pod="openshift-multus/multus-48pqd" Jul 15 19:21:29.089329 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.088259 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/61c3693e-12af-4a9e-81d9-f66cc6520c73-etc-systemd\") pod \"tuned-7m9g6\" (UID: \"61c3693e-12af-4a9e-81d9-f66cc6520c73\") " pod="openshift-cluster-node-tuning-operator/tuned-7m9g6" Jul 15 19:21:29.089329 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.088332 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/61c3693e-12af-4a9e-81d9-f66cc6520c73-sys\") pod \"tuned-7m9g6\" (UID: \"61c3693e-12af-4a9e-81d9-f66cc6520c73\") " pod="openshift-cluster-node-tuning-operator/tuned-7m9g6" Jul 15 19:21:29.089329 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.088404 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/71aa4ce2-c8ac-4224-9226-ba125d49f534-multus-conf-dir\") pod \"multus-48pqd\" (UID: \"71aa4ce2-c8ac-4224-9226-ba125d49f534\") " pod="openshift-multus/multus-48pqd" Jul 15 19:21:29.089329 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.088426 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/71aa4ce2-c8ac-4224-9226-ba125d49f534-host-var-lib-kubelet\") pod \"multus-48pqd\" (UID: \"71aa4ce2-c8ac-4224-9226-ba125d49f534\") " pod="openshift-multus/multus-48pqd" Jul 15 19:21:29.089329 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.088485 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/61c3693e-12af-4a9e-81d9-f66cc6520c73-var-lib-kubelet\") pod \"tuned-7m9g6\" (UID: \"61c3693e-12af-4a9e-81d9-f66cc6520c73\") " pod="openshift-cluster-node-tuning-operator/tuned-7m9g6" Jul 15 19:21:29.089329 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.088527 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-host-kubelet\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:29.089329 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.088565 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-run-ovn\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:29.089329 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.088609 2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 15 19:21:29.089329 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.088639 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/71aa4ce2-c8ac-4224-9226-ba125d49f534-multus-daemon-config\") pod \"multus-48pqd\" (UID: \"71aa4ce2-c8ac-4224-9226-ba125d49f534\") " pod="openshift-multus/multus-48pqd" Jul 15 19:21:29.090300 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.088717 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/71aa4ce2-c8ac-4224-9226-ba125d49f534-host-run-multus-certs\") pod \"multus-48pqd\" (UID: \"71aa4ce2-c8ac-4224-9226-ba125d49f534\") " pod="openshift-multus/multus-48pqd" Jul 15 19:21:29.090300 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.088893 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1820cfdf-910d-42da-bafb-fb076d565b78-konnectivity-ca\") pod \"konnectivity-agent-xx47b\" (UID: \"1820cfdf-910d-42da-bafb-fb076d565b78\") " pod="kube-system/konnectivity-agent-xx47b" Jul 15 19:21:29.090300 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.088937 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/23786db4-0338-4d64-9ea3-be6ebc44d74e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-f7pg5\" (UID: \"23786db4-0338-4d64-9ea3-be6ebc44d74e\") " pod="openshift-multus/multus-additional-cni-plugins-f7pg5" Jul 15 19:21:29.090300 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.088957 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-host-cni-netd\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:29.090300 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.089008 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/299d8c27-2943-40f5-8810-67bcfcbbb3a2-hosts-file\") pod \"node-resolver-9vzln\" (UID: \"299d8c27-2943-40f5-8810-67bcfcbbb3a2\") " pod="openshift-dns/node-resolver-9vzln" Jul 15 19:21:29.090300 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.089010 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-systemd-units\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:29.090300 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.089047 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/35abc39e-4584-4a00-b15e-5c25c1f8ce41-iptables-alerter-script\") pod \"iptables-alerter-hbtrt\" (UID: \"35abc39e-4584-4a00-b15e-5c25c1f8ce41\") " pod="openshift-network-operator/iptables-alerter-hbtrt" Jul 15 19:21:29.090300 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.089056 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/05d64bcb-1ac1-47aa-a557-63ba6c2011ab-kubelet-config\") pod \"global-pull-secret-syncer-pn46g\" (UID: \"05d64bcb-1ac1-47aa-a557-63ba6c2011ab\") " pod="kube-system/global-pull-secret-syncer-pn46g" Jul 15 19:21:29.090300 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.089063 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/23786db4-0338-4d64-9ea3-be6ebc44d74e-os-release\") pod \"multus-additional-cni-plugins-f7pg5\" (UID: \"23786db4-0338-4d64-9ea3-be6ebc44d74e\") " pod="openshift-multus/multus-additional-cni-plugins-f7pg5" Jul 15 19:21:29.090300 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.089090 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ec13993c-0b5b-456f-81cb-542da64f3d23-etc-selinux\") pod \"aws-ebs-csi-driver-node-j99qj\" (UID: \"ec13993c-0b5b-456f-81cb-542da64f3d23\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j99qj" Jul 15 19:21:29.090300 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.089104 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/05d64bcb-1ac1-47aa-a557-63ba6c2011ab-kubelet-config\") pod \"global-pull-secret-syncer-pn46g\" (UID: \"05d64bcb-1ac1-47aa-a557-63ba6c2011ab\") " pod="kube-system/global-pull-secret-syncer-pn46g" Jul 15 19:21:29.090300 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.089110 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/61c3693e-12af-4a9e-81d9-f66cc6520c73-etc-sysconfig\") pod \"tuned-7m9g6\" (UID: \"61c3693e-12af-4a9e-81d9-f66cc6520c73\") " pod="openshift-cluster-node-tuning-operator/tuned-7m9g6" Jul 15 19:21:29.090300 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.089117 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ec13993c-0b5b-456f-81cb-542da64f3d23-sys-fs\") pod \"aws-ebs-csi-driver-node-j99qj\" (UID: \"ec13993c-0b5b-456f-81cb-542da64f3d23\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j99qj" Jul 15 19:21:29.090300 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.089131 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:29.090300 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.089150 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/23786db4-0338-4d64-9ea3-be6ebc44d74e-system-cni-dir\") pod \"multus-additional-cni-plugins-f7pg5\" (UID: \"23786db4-0338-4d64-9ea3-be6ebc44d74e\") " pod="openshift-multus/multus-additional-cni-plugins-f7pg5" Jul 15 19:21:29.090300 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.089155 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/61c3693e-12af-4a9e-81d9-f66cc6520c73-run\") pod \"tuned-7m9g6\" (UID: \"61c3693e-12af-4a9e-81d9-f66cc6520c73\") " pod="openshift-cluster-node-tuning-operator/tuned-7m9g6" Jul 15 19:21:29.090300 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.089167 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ec13993c-0b5b-456f-81cb-542da64f3d23-registration-dir\") pod \"aws-ebs-csi-driver-node-j99qj\" (UID: \"ec13993c-0b5b-456f-81cb-542da64f3d23\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j99qj" Jul 15 19:21:29.091134 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.089168 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/71aa4ce2-c8ac-4224-9226-ba125d49f534-cnibin\") pod \"multus-48pqd\" (UID: \"71aa4ce2-c8ac-4224-9226-ba125d49f534\") " pod="openshift-multus/multus-48pqd" Jul 15 19:21:29.091134 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.089179 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b6d81326-425e-4a61-a7db-8790d035bd50-serviceca\") pod \"node-ca-c8jtk\" (UID: \"b6d81326-425e-4a61-a7db-8790d035bd50\") " pod="openshift-image-registry/node-ca-c8jtk" Jul 15 19:21:29.091134 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.089198 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-host-slash\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:29.091134 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.089220 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/23786db4-0338-4d64-9ea3-be6ebc44d74e-system-cni-dir\") pod \"multus-additional-cni-plugins-f7pg5\" (UID: \"23786db4-0338-4d64-9ea3-be6ebc44d74e\") " pod="openshift-multus/multus-additional-cni-plugins-f7pg5" Jul 15 19:21:29.091134 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.089224 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-host-run-ovn-kubernetes\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:29.091134 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.089230 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-host-slash\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:29.091134 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.089234 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzmbr\" (UniqueName: \"kubernetes.io/projected/b6d81326-425e-4a61-a7db-8790d035bd50-kube-api-access-xzmbr\") pod \"node-ca-c8jtk\" (UID: \"b6d81326-425e-4a61-a7db-8790d035bd50\") " pod="openshift-image-registry/node-ca-c8jtk" Jul 15 19:21:29.091134 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.089252 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b6d81326-425e-4a61-a7db-8790d035bd50-host\") pod \"node-ca-c8jtk\" (UID: \"b6d81326-425e-4a61-a7db-8790d035bd50\") " pod="openshift-image-registry/node-ca-c8jtk" Jul 15 19:21:29.091134 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.089266 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-host-run-netns\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:29.091134 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.089266 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/05d64bcb-1ac1-47aa-a557-63ba6c2011ab-original-pull-secret\") pod \"global-pull-secret-syncer-pn46g\" (UID: \"05d64bcb-1ac1-47aa-a557-63ba6c2011ab\") " pod="kube-system/global-pull-secret-syncer-pn46g" Jul 15 19:21:29.091134 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.089324 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ec13993c-0b5b-456f-81cb-542da64f3d23-device-dir\") pod \"aws-ebs-csi-driver-node-j99qj\" (UID: \"ec13993c-0b5b-456f-81cb-542da64f3d23\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j99qj" Jul 15 19:21:29.091134 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:29.089364 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 19:21:29.091134 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.089376 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwqlh\" (UniqueName: \"kubernetes.io/projected/ec13993c-0b5b-456f-81cb-542da64f3d23-kube-api-access-nwqlh\") pod \"aws-ebs-csi-driver-node-j99qj\" (UID: \"ec13993c-0b5b-456f-81cb-542da64f3d23\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j99qj" Jul 15 19:21:29.091134 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.089380 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/61c3693e-12af-4a9e-81d9-f66cc6520c73-etc-sysctl-conf\") pod \"tuned-7m9g6\" (UID: \"61c3693e-12af-4a9e-81d9-f66cc6520c73\") " pod="openshift-cluster-node-tuning-operator/tuned-7m9g6" Jul 15 19:21:29.091134 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.089408 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/71aa4ce2-c8ac-4224-9226-ba125d49f534-multus-socket-dir-parent\") pod \"multus-48pqd\" (UID: \"71aa4ce2-c8ac-4224-9226-ba125d49f534\") " pod="openshift-multus/multus-48pqd" Jul 15 19:21:29.091134 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:29.089432 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05d64bcb-1ac1-47aa-a557-63ba6c2011ab-original-pull-secret podName:05d64bcb-1ac1-47aa-a557-63ba6c2011ab nodeName:}" failed. No retries permitted until 2026-07-15 19:21:29.589415972 +0000 UTC m=+3.166417638 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/05d64bcb-1ac1-47aa-a557-63ba6c2011ab-original-pull-secret") pod "global-pull-secret-syncer-pn46g" (UID: "05d64bcb-1ac1-47aa-a557-63ba6c2011ab") : object "kube-system"/"original-pull-secret" not registered Jul 15 19:21:29.091134 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.089438 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/61c3693e-12af-4a9e-81d9-f66cc6520c73-host\") pod \"tuned-7m9g6\" (UID: \"61c3693e-12af-4a9e-81d9-f66cc6520c73\") " pod="openshift-cluster-node-tuning-operator/tuned-7m9g6" Jul 15 19:21:29.091996 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.089480 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-run-systemd\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:29.091996 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.089498 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/71aa4ce2-c8ac-4224-9226-ba125d49f534-os-release\") pod \"multus-48pqd\" (UID: \"71aa4ce2-c8ac-4224-9226-ba125d49f534\") " pod="openshift-multus/multus-48pqd" Jul 15 19:21:29.091996 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.089529 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-node-log\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:29.091996 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.089604 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/71aa4ce2-c8ac-4224-9226-ba125d49f534-host-run-netns\") pod \"multus-48pqd\" (UID: \"71aa4ce2-c8ac-4224-9226-ba125d49f534\") " pod="openshift-multus/multus-48pqd" Jul 15 19:21:29.091996 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.089652 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/71aa4ce2-c8ac-4224-9226-ba125d49f534-host-var-lib-cni-multus\") pod \"multus-48pqd\" (UID: \"71aa4ce2-c8ac-4224-9226-ba125d49f534\") " pod="openshift-multus/multus-48pqd" Jul 15 19:21:29.091996 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.089661 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-env-overrides\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:29.091996 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.089699 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-var-lib-openvswitch\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:29.091996 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.089701 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-run-openvswitch\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:29.091996 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.089751 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/61c3693e-12af-4a9e-81d9-f66cc6520c73-etc-sysconfig\") pod \"tuned-7m9g6\" (UID: \"61c3693e-12af-4a9e-81d9-f66cc6520c73\") " pod="openshift-cluster-node-tuning-operator/tuned-7m9g6" Jul 15 19:21:29.091996 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.089791 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:29.091996 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.089831 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/23786db4-0338-4d64-9ea3-be6ebc44d74e-cnibin\") pod \"multus-additional-cni-plugins-f7pg5\" (UID: \"23786db4-0338-4d64-9ea3-be6ebc44d74e\") " pod="openshift-multus/multus-additional-cni-plugins-f7pg5" Jul 15 19:21:29.091996 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.089870 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/35abc39e-4584-4a00-b15e-5c25c1f8ce41-host-slash\") pod \"iptables-alerter-hbtrt\" (UID: \"35abc39e-4584-4a00-b15e-5c25c1f8ce41\") " pod="openshift-network-operator/iptables-alerter-hbtrt" Jul 15 19:21:29.091996 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.089936 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/71aa4ce2-c8ac-4224-9226-ba125d49f534-host-run-k8s-cni-cncf-io\") pod \"multus-48pqd\" (UID: \"71aa4ce2-c8ac-4224-9226-ba125d49f534\") " pod="openshift-multus/multus-48pqd" Jul 15 19:21:29.091996 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.089956 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/61c3693e-12af-4a9e-81d9-f66cc6520c73-etc-tuned\") pod \"tuned-7m9g6\" (UID: \"61c3693e-12af-4a9e-81d9-f66cc6520c73\") " pod="openshift-cluster-node-tuning-operator/tuned-7m9g6" Jul 15 19:21:29.091996 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.089978 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ec13993c-0b5b-456f-81cb-542da64f3d23-socket-dir\") pod \"aws-ebs-csi-driver-node-j99qj\" (UID: \"ec13993c-0b5b-456f-81cb-542da64f3d23\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j99qj" Jul 15 19:21:29.091996 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.090014 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-log-socket\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:29.091996 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.090078 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ec13993c-0b5b-456f-81cb-542da64f3d23-socket-dir\") pod \"aws-ebs-csi-driver-node-j99qj\" (UID: \"ec13993c-0b5b-456f-81cb-542da64f3d23\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j99qj" Jul 15 19:21:29.091996 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.090137 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/71aa4ce2-c8ac-4224-9226-ba125d49f534-host-run-k8s-cni-cncf-io\") pod \"multus-48pqd\" (UID: \"71aa4ce2-c8ac-4224-9226-ba125d49f534\") " pod="openshift-multus/multus-48pqd" Jul 15 19:21:29.092862 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.090426 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/23786db4-0338-4d64-9ea3-be6ebc44d74e-cni-binary-copy\") pod \"multus-additional-cni-plugins-f7pg5\" (UID: \"23786db4-0338-4d64-9ea3-be6ebc44d74e\") " pod="openshift-multus/multus-additional-cni-plugins-f7pg5" Jul 15 19:21:29.092862 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.090444 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/71aa4ce2-c8ac-4224-9226-ba125d49f534-cni-binary-copy\") pod \"multus-48pqd\" (UID: \"71aa4ce2-c8ac-4224-9226-ba125d49f534\") " pod="openshift-multus/multus-48pqd" Jul 15 19:21:29.092862 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.090443 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-ovnkube-script-lib\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:29.092862 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.090575 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/61c3693e-12af-4a9e-81d9-f66cc6520c73-etc-modprobe-d\") pod \"tuned-7m9g6\" (UID: \"61c3693e-12af-4a9e-81d9-f66cc6520c73\") " pod="openshift-cluster-node-tuning-operator/tuned-7m9g6" Jul 15 19:21:29.092862 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.090635 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-host-cni-bin\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:29.092862 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.090661 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ec13993c-0b5b-456f-81cb-542da64f3d23-registration-dir\") pod \"aws-ebs-csi-driver-node-j99qj\" (UID: \"ec13993c-0b5b-456f-81cb-542da64f3d23\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j99qj" Jul 15 19:21:29.092862 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.090714 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/61c3693e-12af-4a9e-81d9-f66cc6520c73-etc-sysctl-d\") pod \"tuned-7m9g6\" (UID: \"61c3693e-12af-4a9e-81d9-f66cc6520c73\") " pod="openshift-cluster-node-tuning-operator/tuned-7m9g6" Jul 15 19:21:29.092862 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.090721 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ec13993c-0b5b-456f-81cb-542da64f3d23-etc-selinux\") pod \"aws-ebs-csi-driver-node-j99qj\" (UID: \"ec13993c-0b5b-456f-81cb-542da64f3d23\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j99qj" Jul 15 19:21:29.092862 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.090744 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ec13993c-0b5b-456f-81cb-542da64f3d23-device-dir\") pod \"aws-ebs-csi-driver-node-j99qj\" (UID: \"ec13993c-0b5b-456f-81cb-542da64f3d23\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j99qj" Jul 15 19:21:29.092862 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.090794 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ec13993c-0b5b-456f-81cb-542da64f3d23-kubelet-dir\") pod \"aws-ebs-csi-driver-node-j99qj\" (UID: \"ec13993c-0b5b-456f-81cb-542da64f3d23\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j99qj" Jul 15 19:21:29.092862 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.090858 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/71aa4ce2-c8ac-4224-9226-ba125d49f534-multus-socket-dir-parent\") pod \"multus-48pqd\" (UID: \"71aa4ce2-c8ac-4224-9226-ba125d49f534\") " pod="openshift-multus/multus-48pqd" Jul 15 19:21:29.092862 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.090937 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/61c3693e-12af-4a9e-81d9-f66cc6520c73-lib-modules\") pod \"tuned-7m9g6\" (UID: \"61c3693e-12af-4a9e-81d9-f66cc6520c73\") " pod="openshift-cluster-node-tuning-operator/tuned-7m9g6" Jul 15 19:21:29.092862 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.090974 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/299d8c27-2943-40f5-8810-67bcfcbbb3a2-tmp-dir\") pod \"node-resolver-9vzln\" (UID: \"299d8c27-2943-40f5-8810-67bcfcbbb3a2\") " pod="openshift-dns/node-resolver-9vzln" Jul 15 19:21:29.092862 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.091031 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/23786db4-0338-4d64-9ea3-be6ebc44d74e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-f7pg5\" (UID: \"23786db4-0338-4d64-9ea3-be6ebc44d74e\") " pod="openshift-multus/multus-additional-cni-plugins-f7pg5" Jul 15 19:21:29.092862 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.091093 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/71aa4ce2-c8ac-4224-9226-ba125d49f534-multus-cni-dir\") pod \"multus-48pqd\" (UID: \"71aa4ce2-c8ac-4224-9226-ba125d49f534\") " pod="openshift-multus/multus-48pqd" Jul 15 19:21:29.092862 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.091262 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-ovnkube-config\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:29.092862 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.091298 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/71aa4ce2-c8ac-4224-9226-ba125d49f534-etc-kubernetes\") pod \"multus-48pqd\" (UID: \"71aa4ce2-c8ac-4224-9226-ba125d49f534\") " pod="openshift-multus/multus-48pqd" Jul 15 19:21:29.092862 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.091305 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-etc-openvswitch\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:29.093756 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.091391 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/61c3693e-12af-4a9e-81d9-f66cc6520c73-etc-kubernetes\") pod \"tuned-7m9g6\" (UID: \"61c3693e-12af-4a9e-81d9-f66cc6520c73\") " pod="openshift-cluster-node-tuning-operator/tuned-7m9g6" Jul 15 19:21:29.094122 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.094067 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-62.ec2.internal" event={"ID":"db478e14aba317d5490063a0b6ec4a2a","Type":"ContainerStarted","Data":"8d1a7510f658fc6fae58e8da4b0bae14eb76c4a60efe4cf25a259250ff61dc9c"} Jul 15 19:21:29.094718 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.094557 2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 19:21:29.095495 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.095434 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-ovn-node-metrics-cert\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:29.096175 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.096135 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61c3693e-12af-4a9e-81d9-f66cc6520c73-tmp\") pod \"tuned-7m9g6\" (UID: \"61c3693e-12af-4a9e-81d9-f66cc6520c73\") " pod="openshift-cluster-node-tuning-operator/tuned-7m9g6" Jul 15 19:21:29.096614 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:29.096497 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 19:21:29.096614 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:29.096520 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 19:21:29.096614 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:29.096535 2577 projected.go:196] Error preparing data for projected volume kube-api-access-z8f8j for pod openshift-network-diagnostics/network-check-target-5zk6t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:21:29.096614 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:29.096595 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c90bd526-03fe-4c0c-9d9d-5bd777b6bd87-kube-api-access-z8f8j podName:c90bd526-03fe-4c0c-9d9d-5bd777b6bd87 nodeName:}" failed. No retries permitted until 2026-07-15 19:21:29.596577922 +0000 UTC m=+3.173579588 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-z8f8j" (UniqueName: "kubernetes.io/projected/c90bd526-03fe-4c0c-9d9d-5bd777b6bd87-kube-api-access-z8f8j") pod "network-check-target-5zk6t" (UID: "c90bd526-03fe-4c0c-9d9d-5bd777b6bd87") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:21:29.096894 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.096783 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-62.ec2.internal" event={"ID":"64ae407aa9b62ec31a4fe3be45950ba9","Type":"ContainerStarted","Data":"056dfffd79ad34da047db20f3beef9787e50e2b50f05a2493f4ddc702969fae4"} Jul 15 19:21:29.096894 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.096823 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/61c3693e-12af-4a9e-81d9-f66cc6520c73-etc-tuned\") pod \"tuned-7m9g6\" (UID: \"61c3693e-12af-4a9e-81d9-f66cc6520c73\") " pod="openshift-cluster-node-tuning-operator/tuned-7m9g6" Jul 15 19:21:29.098458 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.098402 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdd5x\" (UniqueName: \"kubernetes.io/projected/71aa4ce2-c8ac-4224-9226-ba125d49f534-kube-api-access-tdd5x\") pod \"multus-48pqd\" (UID: \"71aa4ce2-c8ac-4224-9226-ba125d49f534\") " pod="openshift-multus/multus-48pqd" Jul 15 19:21:29.099447 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.099422 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1820cfdf-910d-42da-bafb-fb076d565b78-agent-certs\") pod \"konnectivity-agent-xx47b\" (UID: \"1820cfdf-910d-42da-bafb-fb076d565b78\") " pod="kube-system/konnectivity-agent-xx47b" Jul 15 19:21:29.099866 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.099819 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfgwz\" (UniqueName: \"kubernetes.io/projected/299d8c27-2943-40f5-8810-67bcfcbbb3a2-kube-api-access-nfgwz\") pod \"node-resolver-9vzln\" (UID: \"299d8c27-2943-40f5-8810-67bcfcbbb3a2\") " pod="openshift-dns/node-resolver-9vzln" Jul 15 19:21:29.100916 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.100892 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8b5qx\" (UniqueName: \"kubernetes.io/projected/35abc39e-4584-4a00-b15e-5c25c1f8ce41-kube-api-access-8b5qx\") pod \"iptables-alerter-hbtrt\" (UID: \"35abc39e-4584-4a00-b15e-5c25c1f8ce41\") " pod="openshift-network-operator/iptables-alerter-hbtrt" Jul 15 19:21:29.103465 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.103418 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzmbr\" (UniqueName: \"kubernetes.io/projected/b6d81326-425e-4a61-a7db-8790d035bd50-kube-api-access-xzmbr\") pod \"node-ca-c8jtk\" (UID: \"b6d81326-425e-4a61-a7db-8790d035bd50\") " pod="openshift-image-registry/node-ca-c8jtk" Jul 15 19:21:29.103465 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.103439 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwqlh\" (UniqueName: \"kubernetes.io/projected/ec13993c-0b5b-456f-81cb-542da64f3d23-kube-api-access-nwqlh\") pod \"aws-ebs-csi-driver-node-j99qj\" (UID: \"ec13993c-0b5b-456f-81cb-542da64f3d23\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j99qj" Jul 15 19:21:29.106782 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.106758 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqqm5\" (UniqueName: \"kubernetes.io/projected/237b9375-100a-4d62-83f0-cb271ab8e0bb-kube-api-access-pqqm5\") pod \"network-metrics-daemon-cmqc9\" (UID: \"237b9375-100a-4d62-83f0-cb271ab8e0bb\") " pod="openshift-multus/network-metrics-daemon-cmqc9" Jul 15 19:21:29.108179 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.108158 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p55zx\" (UniqueName: \"kubernetes.io/projected/61c3693e-12af-4a9e-81d9-f66cc6520c73-kube-api-access-p55zx\") pod \"tuned-7m9g6\" (UID: \"61c3693e-12af-4a9e-81d9-f66cc6520c73\") " pod="openshift-cluster-node-tuning-operator/tuned-7m9g6" Jul 15 19:21:29.108517 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.108496 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pdxz\" (UniqueName: \"kubernetes.io/projected/23786db4-0338-4d64-9ea3-be6ebc44d74e-kube-api-access-5pdxz\") pod \"multus-additional-cni-plugins-f7pg5\" (UID: \"23786db4-0338-4d64-9ea3-be6ebc44d74e\") " pod="openshift-multus/multus-additional-cni-plugins-f7pg5" Jul 15 19:21:29.110293 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.110269 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxgbr\" (UniqueName: \"kubernetes.io/projected/89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21-kube-api-access-sxgbr\") pod \"ovnkube-node-fw6vl\" (UID: \"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:29.165434 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.165405 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-48pqd" Jul 15 19:21:29.174565 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.174374 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7m9g6" Jul 15 19:21:29.182603 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.182303 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:29.186823 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.186674 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xx47b" Jul 15 19:21:29.194702 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.194505 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9vzln" Jul 15 19:21:29.201244 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.201092 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-c8jtk" Jul 15 19:21:29.201428 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:29.201404 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1820cfdf_910d_42da_bafb_fb076d565b78.slice/crio-c15babc0e547945f59493bfbb0dcf9397e06be4fcdd54a5e037ae51cbb52f380 WatchSource:0}: Error finding container c15babc0e547945f59493bfbb0dcf9397e06be4fcdd54a5e037ae51cbb52f380: Status 404 returned error can't find the container with id c15babc0e547945f59493bfbb0dcf9397e06be4fcdd54a5e037ae51cbb52f380 Jul 15 19:21:29.208313 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.208177 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hbtrt" Jul 15 19:21:29.213905 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.213879 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-f7pg5" Jul 15 19:21:29.220207 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.220180 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j99qj" Jul 15 19:21:29.236957 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:21:29.236932 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec13993c_0b5b_456f_81cb_542da64f3d23.slice/crio-f007c8fdf327d7407c950081df21e139d5f3bfc4e019312aa9a67e7f3991a21e WatchSource:0}: Error finding container f007c8fdf327d7407c950081df21e139d5f3bfc4e019312aa9a67e7f3991a21e: Status 404 returned error can't find the container with id f007c8fdf327d7407c950081df21e139d5f3bfc4e019312aa9a67e7f3991a21e Jul 15 19:21:29.594852 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.594819 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/237b9375-100a-4d62-83f0-cb271ab8e0bb-metrics-certs\") pod \"network-metrics-daemon-cmqc9\" (UID: \"237b9375-100a-4d62-83f0-cb271ab8e0bb\") " pod="openshift-multus/network-metrics-daemon-cmqc9" Jul 15 19:21:29.595026 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.594900 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/05d64bcb-1ac1-47aa-a557-63ba6c2011ab-original-pull-secret\") pod \"global-pull-secret-syncer-pn46g\" (UID: \"05d64bcb-1ac1-47aa-a557-63ba6c2011ab\") " pod="kube-system/global-pull-secret-syncer-pn46g" Jul 15 19:21:29.595026 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:29.594994 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 19:21:29.595026 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:29.595006 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:21:29.595152 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:29.595055 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05d64bcb-1ac1-47aa-a557-63ba6c2011ab-original-pull-secret podName:05d64bcb-1ac1-47aa-a557-63ba6c2011ab nodeName:}" failed. No retries permitted until 2026-07-15 19:21:30.595035832 +0000 UTC m=+4.172037500 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/05d64bcb-1ac1-47aa-a557-63ba6c2011ab-original-pull-secret") pod "global-pull-secret-syncer-pn46g" (UID: "05d64bcb-1ac1-47aa-a557-63ba6c2011ab") : object "kube-system"/"original-pull-secret" not registered Jul 15 19:21:29.595152 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:29.595073 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/237b9375-100a-4d62-83f0-cb271ab8e0bb-metrics-certs podName:237b9375-100a-4d62-83f0-cb271ab8e0bb nodeName:}" failed. No retries permitted until 2026-07-15 19:21:30.595063764 +0000 UTC m=+4.172065418 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/237b9375-100a-4d62-83f0-cb271ab8e0bb-metrics-certs") pod "network-metrics-daemon-cmqc9" (UID: "237b9375-100a-4d62-83f0-cb271ab8e0bb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:21:29.695460 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.695427 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8f8j\" (UniqueName: \"kubernetes.io/projected/c90bd526-03fe-4c0c-9d9d-5bd777b6bd87-kube-api-access-z8f8j\") pod \"network-check-target-5zk6t\" (UID: \"c90bd526-03fe-4c0c-9d9d-5bd777b6bd87\") " pod="openshift-network-diagnostics/network-check-target-5zk6t" Jul 15 19:21:29.695637 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:29.695593 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 19:21:29.695637 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:29.695615 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 19:21:29.695637 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:29.695625 2577 projected.go:196] Error preparing data for projected volume kube-api-access-z8f8j for pod openshift-network-diagnostics/network-check-target-5zk6t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:21:29.695741 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:29.695673 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c90bd526-03fe-4c0c-9d9d-5bd777b6bd87-kube-api-access-z8f8j podName:c90bd526-03fe-4c0c-9d9d-5bd777b6bd87 nodeName:}" failed. No retries permitted until 2026-07-15 19:21:30.69565937 +0000 UTC m=+4.272661020 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-z8f8j" (UniqueName: "kubernetes.io/projected/c90bd526-03fe-4c0c-9d9d-5bd777b6bd87-kube-api-access-z8f8j") pod "network-check-target-5zk6t" (UID: "c90bd526-03fe-4c0c-9d9d-5bd777b6bd87") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:21:29.905078 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.904961 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 19:16:28 +0000 UTC" deadline="2028-01-11 23:34:53.976074319 +0000 UTC" Jul 15 19:21:29.905078 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:29.905029 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13084h13m24.071049262s" Jul 15 19:21:30.086329 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:30.086284 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmqc9" Jul 15 19:21:30.086498 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:30.086435 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmqc9" podUID="237b9375-100a-4d62-83f0-cb271ab8e0bb" Jul 15 19:21:30.086498 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:30.086479 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pn46g" Jul 15 19:21:30.086613 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:30.086582 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pn46g" podUID="05d64bcb-1ac1-47aa-a557-63ba6c2011ab" Jul 15 19:21:30.100424 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:30.100392 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hbtrt" event={"ID":"35abc39e-4584-4a00-b15e-5c25c1f8ce41","Type":"ContainerStarted","Data":"4753b999b4b2513d3365d025594154065613569e7b08923d14937cde98bce727"} Jul 15 19:21:30.101626 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:30.101593 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9vzln" event={"ID":"299d8c27-2943-40f5-8810-67bcfcbbb3a2","Type":"ContainerStarted","Data":"4aadd54110abf79dca4bd163bacaa1b474286393eddbdb5c5111300a4ccda71d"} Jul 15 19:21:30.102918 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:30.102890 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xx47b" event={"ID":"1820cfdf-910d-42da-bafb-fb076d565b78","Type":"ContainerStarted","Data":"c15babc0e547945f59493bfbb0dcf9397e06be4fcdd54a5e037ae51cbb52f380"} Jul 15 19:21:30.104098 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:30.104073 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j99qj" event={"ID":"ec13993c-0b5b-456f-81cb-542da64f3d23","Type":"ContainerStarted","Data":"f007c8fdf327d7407c950081df21e139d5f3bfc4e019312aa9a67e7f3991a21e"} Jul 15 19:21:30.105364 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:30.105343 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f7pg5" event={"ID":"23786db4-0338-4d64-9ea3-be6ebc44d74e","Type":"ContainerStarted","Data":"86f277dcb7eec244914b82acffe17bb2e851eed7e3bc515f7613fe7209fcb65d"} Jul 15 19:21:30.106402 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:30.106383 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-c8jtk" event={"ID":"b6d81326-425e-4a61-a7db-8790d035bd50","Type":"ContainerStarted","Data":"18a6fe9509dd22cd8bb6caa13161ed981cd0ef4052f9e8d9fcd5eea62edbea1c"} Jul 15 19:21:30.107536 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:30.107507 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" event={"ID":"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21","Type":"ContainerStarted","Data":"05ed1e64356a40ceaaa42ebc93d2c26b87de75f0c21fd478a70e85fd7a523be1"} Jul 15 19:21:30.108634 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:30.108614 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7m9g6" event={"ID":"61c3693e-12af-4a9e-81d9-f66cc6520c73","Type":"ContainerStarted","Data":"f6e76a5227e303262809a0182c716c702f3870694726895ad1806a7072e13b13"} Jul 15 19:21:30.109585 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:30.109565 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-48pqd" event={"ID":"71aa4ce2-c8ac-4224-9226-ba125d49f534","Type":"ContainerStarted","Data":"3e10849e34c60224e8d94bd69707920498fb75dd1e534c67dcad9e7f552b400a"} Jul 15 19:21:30.602853 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:30.602815 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/05d64bcb-1ac1-47aa-a557-63ba6c2011ab-original-pull-secret\") pod \"global-pull-secret-syncer-pn46g\" (UID: \"05d64bcb-1ac1-47aa-a557-63ba6c2011ab\") " pod="kube-system/global-pull-secret-syncer-pn46g" Jul 15 19:21:30.603032 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:30.602889 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/237b9375-100a-4d62-83f0-cb271ab8e0bb-metrics-certs\") pod \"network-metrics-daemon-cmqc9\" (UID: \"237b9375-100a-4d62-83f0-cb271ab8e0bb\") " pod="openshift-multus/network-metrics-daemon-cmqc9" Jul 15 19:21:30.603032 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:30.602972 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 19:21:30.603032 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:30.603018 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:21:30.603208 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:30.603051 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05d64bcb-1ac1-47aa-a557-63ba6c2011ab-original-pull-secret podName:05d64bcb-1ac1-47aa-a557-63ba6c2011ab nodeName:}" failed. No retries permitted until 2026-07-15 19:21:32.60303137 +0000 UTC m=+6.180033023 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/05d64bcb-1ac1-47aa-a557-63ba6c2011ab-original-pull-secret") pod "global-pull-secret-syncer-pn46g" (UID: "05d64bcb-1ac1-47aa-a557-63ba6c2011ab") : object "kube-system"/"original-pull-secret" not registered Jul 15 19:21:30.603208 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:30.603071 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/237b9375-100a-4d62-83f0-cb271ab8e0bb-metrics-certs podName:237b9375-100a-4d62-83f0-cb271ab8e0bb nodeName:}" failed. No retries permitted until 2026-07-15 19:21:32.603061951 +0000 UTC m=+6.180063605 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/237b9375-100a-4d62-83f0-cb271ab8e0bb-metrics-certs") pod "network-metrics-daemon-cmqc9" (UID: "237b9375-100a-4d62-83f0-cb271ab8e0bb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:21:30.704210 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:30.704175 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8f8j\" (UniqueName: \"kubernetes.io/projected/c90bd526-03fe-4c0c-9d9d-5bd777b6bd87-kube-api-access-z8f8j\") pod \"network-check-target-5zk6t\" (UID: \"c90bd526-03fe-4c0c-9d9d-5bd777b6bd87\") " pod="openshift-network-diagnostics/network-check-target-5zk6t" Jul 15 19:21:30.704416 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:30.704384 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 19:21:30.704416 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:30.704409 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 19:21:30.704527 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:30.704423 2577 projected.go:196] Error preparing data for projected volume kube-api-access-z8f8j for pod openshift-network-diagnostics/network-check-target-5zk6t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:21:30.704527 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:30.704480 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c90bd526-03fe-4c0c-9d9d-5bd777b6bd87-kube-api-access-z8f8j podName:c90bd526-03fe-4c0c-9d9d-5bd777b6bd87 nodeName:}" failed. No retries permitted until 2026-07-15 19:21:32.704461752 +0000 UTC m=+6.281463417 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-z8f8j" (UniqueName: "kubernetes.io/projected/c90bd526-03fe-4c0c-9d9d-5bd777b6bd87-kube-api-access-z8f8j") pod "network-check-target-5zk6t" (UID: "c90bd526-03fe-4c0c-9d9d-5bd777b6bd87") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:21:30.905935 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:30.905663 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 19:16:28 +0000 UTC" deadline="2028-02-08 17:20:45.169495185 +0000 UTC" Jul 15 19:21:30.905935 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:30.905739 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13749h59m14.263761933s" Jul 15 19:21:31.096934 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:31.096708 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zk6t" Jul 15 19:21:31.097145 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:31.097031 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zk6t" podUID="c90bd526-03fe-4c0c-9d9d-5bd777b6bd87" Jul 15 19:21:31.125955 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:31.125918 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-62.ec2.internal" event={"ID":"db478e14aba317d5490063a0b6ec4a2a","Type":"ContainerStarted","Data":"0c4cec86785d5bd1d78f18d4fb03dcd9bc1c2aca2fb10b711935331b2b4f79f0"} Jul 15 19:21:32.086457 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:32.086421 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmqc9" Jul 15 19:21:32.087031 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:32.086570 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmqc9" podUID="237b9375-100a-4d62-83f0-cb271ab8e0bb" Jul 15 19:21:32.087031 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:32.086996 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pn46g" Jul 15 19:21:32.087164 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:32.087087 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pn46g" podUID="05d64bcb-1ac1-47aa-a557-63ba6c2011ab" Jul 15 19:21:32.138813 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:32.138779 2577 generic.go:356] "Generic (PLEG): container finished" podID="64ae407aa9b62ec31a4fe3be45950ba9" containerID="d3a69ae9d3bf55aefa70dc86154d1eb103a9ad0da1cfa9c0e938009db1a6be8a" exitCode=0 Jul 15 19:21:32.139758 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:32.139732 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-62.ec2.internal" event={"ID":"64ae407aa9b62ec31a4fe3be45950ba9","Type":"ContainerDied","Data":"d3a69ae9d3bf55aefa70dc86154d1eb103a9ad0da1cfa9c0e938009db1a6be8a"} Jul 15 19:21:32.156430 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:32.156374 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-62.ec2.internal" podStartSLOduration=4.156357468 podStartE2EDuration="4.156357468s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:21:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:21:31.144430871 +0000 UTC m=+4.721432543" watchObservedRunningTime="2026-07-15 19:21:32.156357468 +0000 UTC m=+5.733359143" Jul 15 19:21:32.623681 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:32.623644 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/05d64bcb-1ac1-47aa-a557-63ba6c2011ab-original-pull-secret\") pod \"global-pull-secret-syncer-pn46g\" (UID: \"05d64bcb-1ac1-47aa-a557-63ba6c2011ab\") " pod="kube-system/global-pull-secret-syncer-pn46g" Jul 15 19:21:32.623860 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:32.623707 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/237b9375-100a-4d62-83f0-cb271ab8e0bb-metrics-certs\") pod \"network-metrics-daemon-cmqc9\" (UID: \"237b9375-100a-4d62-83f0-cb271ab8e0bb\") " pod="openshift-multus/network-metrics-daemon-cmqc9" Jul 15 19:21:32.623919 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:32.623865 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:21:32.623977 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:32.623924 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/237b9375-100a-4d62-83f0-cb271ab8e0bb-metrics-certs podName:237b9375-100a-4d62-83f0-cb271ab8e0bb nodeName:}" failed. No retries permitted until 2026-07-15 19:21:36.623906624 +0000 UTC m=+10.200908278 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/237b9375-100a-4d62-83f0-cb271ab8e0bb-metrics-certs") pod "network-metrics-daemon-cmqc9" (UID: "237b9375-100a-4d62-83f0-cb271ab8e0bb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:21:32.624380 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:32.624330 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 19:21:32.624380 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:32.624377 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05d64bcb-1ac1-47aa-a557-63ba6c2011ab-original-pull-secret podName:05d64bcb-1ac1-47aa-a557-63ba6c2011ab nodeName:}" failed. No retries permitted until 2026-07-15 19:21:36.624363432 +0000 UTC m=+10.201365085 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/05d64bcb-1ac1-47aa-a557-63ba6c2011ab-original-pull-secret") pod "global-pull-secret-syncer-pn46g" (UID: "05d64bcb-1ac1-47aa-a557-63ba6c2011ab") : object "kube-system"/"original-pull-secret" not registered Jul 15 19:21:32.725374 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:32.724654 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8f8j\" (UniqueName: \"kubernetes.io/projected/c90bd526-03fe-4c0c-9d9d-5bd777b6bd87-kube-api-access-z8f8j\") pod \"network-check-target-5zk6t\" (UID: \"c90bd526-03fe-4c0c-9d9d-5bd777b6bd87\") " pod="openshift-network-diagnostics/network-check-target-5zk6t" Jul 15 19:21:32.725374 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:32.724841 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 19:21:32.725374 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:32.724859 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 19:21:32.725374 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:32.724872 2577 projected.go:196] Error preparing data for projected volume kube-api-access-z8f8j for pod openshift-network-diagnostics/network-check-target-5zk6t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:21:32.725374 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:32.724928 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c90bd526-03fe-4c0c-9d9d-5bd777b6bd87-kube-api-access-z8f8j podName:c90bd526-03fe-4c0c-9d9d-5bd777b6bd87 nodeName:}" failed. No retries permitted until 2026-07-15 19:21:36.724910379 +0000 UTC m=+10.301912046 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-z8f8j" (UniqueName: "kubernetes.io/projected/c90bd526-03fe-4c0c-9d9d-5bd777b6bd87-kube-api-access-z8f8j") pod "network-check-target-5zk6t" (UID: "c90bd526-03fe-4c0c-9d9d-5bd777b6bd87") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:21:33.086472 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:33.086400 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zk6t" Jul 15 19:21:33.086928 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:33.086515 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zk6t" podUID="c90bd526-03fe-4c0c-9d9d-5bd777b6bd87" Jul 15 19:21:34.086857 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:34.086229 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmqc9" Jul 15 19:21:34.086857 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:34.086398 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmqc9" podUID="237b9375-100a-4d62-83f0-cb271ab8e0bb" Jul 15 19:21:34.086857 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:34.086462 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pn46g" Jul 15 19:21:34.086857 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:34.086532 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pn46g" podUID="05d64bcb-1ac1-47aa-a557-63ba6c2011ab" Jul 15 19:21:35.088729 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:35.087943 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zk6t" Jul 15 19:21:35.088729 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:35.088068 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zk6t" podUID="c90bd526-03fe-4c0c-9d9d-5bd777b6bd87" Jul 15 19:21:36.086126 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:36.086092 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pn46g" Jul 15 19:21:36.086126 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:36.086114 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmqc9" Jul 15 19:21:36.086380 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:36.086225 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pn46g" podUID="05d64bcb-1ac1-47aa-a557-63ba6c2011ab" Jul 15 19:21:36.086380 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:36.086344 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmqc9" podUID="237b9375-100a-4d62-83f0-cb271ab8e0bb" Jul 15 19:21:36.655652 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:36.655015 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/05d64bcb-1ac1-47aa-a557-63ba6c2011ab-original-pull-secret\") pod \"global-pull-secret-syncer-pn46g\" (UID: \"05d64bcb-1ac1-47aa-a557-63ba6c2011ab\") " pod="kube-system/global-pull-secret-syncer-pn46g" Jul 15 19:21:36.655652 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:36.655080 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/237b9375-100a-4d62-83f0-cb271ab8e0bb-metrics-certs\") pod \"network-metrics-daemon-cmqc9\" (UID: \"237b9375-100a-4d62-83f0-cb271ab8e0bb\") " pod="openshift-multus/network-metrics-daemon-cmqc9" Jul 15 19:21:36.655652 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:36.655252 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:21:36.655652 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:36.655287 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 19:21:36.655652 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:36.655325 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/237b9375-100a-4d62-83f0-cb271ab8e0bb-metrics-certs podName:237b9375-100a-4d62-83f0-cb271ab8e0bb nodeName:}" failed. No retries permitted until 2026-07-15 19:21:44.655292769 +0000 UTC m=+18.232294424 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/237b9375-100a-4d62-83f0-cb271ab8e0bb-metrics-certs") pod "network-metrics-daemon-cmqc9" (UID: "237b9375-100a-4d62-83f0-cb271ab8e0bb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:21:36.655652 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:36.655365 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05d64bcb-1ac1-47aa-a557-63ba6c2011ab-original-pull-secret podName:05d64bcb-1ac1-47aa-a557-63ba6c2011ab nodeName:}" failed. No retries permitted until 2026-07-15 19:21:44.655346784 +0000 UTC m=+18.232348452 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/05d64bcb-1ac1-47aa-a557-63ba6c2011ab-original-pull-secret") pod "global-pull-secret-syncer-pn46g" (UID: "05d64bcb-1ac1-47aa-a557-63ba6c2011ab") : object "kube-system"/"original-pull-secret" not registered Jul 15 19:21:36.755920 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:36.755884 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8f8j\" (UniqueName: \"kubernetes.io/projected/c90bd526-03fe-4c0c-9d9d-5bd777b6bd87-kube-api-access-z8f8j\") pod \"network-check-target-5zk6t\" (UID: \"c90bd526-03fe-4c0c-9d9d-5bd777b6bd87\") " pod="openshift-network-diagnostics/network-check-target-5zk6t" Jul 15 19:21:36.756095 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:36.756066 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 19:21:36.756095 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:36.756085 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 19:21:36.756095 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:36.756097 2577 projected.go:196] Error preparing data for projected volume kube-api-access-z8f8j for pod openshift-network-diagnostics/network-check-target-5zk6t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:21:36.756257 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:36.756151 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c90bd526-03fe-4c0c-9d9d-5bd777b6bd87-kube-api-access-z8f8j podName:c90bd526-03fe-4c0c-9d9d-5bd777b6bd87 nodeName:}" failed. No retries permitted until 2026-07-15 19:21:44.756133713 +0000 UTC m=+18.333135374 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-z8f8j" (UniqueName: "kubernetes.io/projected/c90bd526-03fe-4c0c-9d9d-5bd777b6bd87-kube-api-access-z8f8j") pod "network-check-target-5zk6t" (UID: "c90bd526-03fe-4c0c-9d9d-5bd777b6bd87") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:21:37.087655 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:37.087172 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zk6t" Jul 15 19:21:37.087655 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:37.087283 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zk6t" podUID="c90bd526-03fe-4c0c-9d9d-5bd777b6bd87" Jul 15 19:21:38.086980 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:38.086950 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmqc9" Jul 15 19:21:38.087452 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:38.087087 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmqc9" podUID="237b9375-100a-4d62-83f0-cb271ab8e0bb" Jul 15 19:21:38.087540 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:38.087475 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pn46g" Jul 15 19:21:38.087597 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:38.087563 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pn46g" podUID="05d64bcb-1ac1-47aa-a557-63ba6c2011ab" Jul 15 19:21:39.086282 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:39.086245 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zk6t" Jul 15 19:21:39.086450 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:39.086394 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zk6t" podUID="c90bd526-03fe-4c0c-9d9d-5bd777b6bd87" Jul 15 19:21:40.086731 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:40.086696 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmqc9" Jul 15 19:21:40.086731 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:40.086728 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pn46g" Jul 15 19:21:40.087157 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:40.086819 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmqc9" podUID="237b9375-100a-4d62-83f0-cb271ab8e0bb" Jul 15 19:21:40.087157 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:40.086934 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pn46g" podUID="05d64bcb-1ac1-47aa-a557-63ba6c2011ab" Jul 15 19:21:41.089279 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:41.089252 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zk6t" Jul 15 19:21:41.089714 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:41.089368 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zk6t" podUID="c90bd526-03fe-4c0c-9d9d-5bd777b6bd87" Jul 15 19:21:42.087005 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:42.086974 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmqc9" Jul 15 19:21:42.087188 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:42.087099 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmqc9" podUID="237b9375-100a-4d62-83f0-cb271ab8e0bb" Jul 15 19:21:42.087188 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:42.087136 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pn46g" Jul 15 19:21:42.087305 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:42.087215 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pn46g" podUID="05d64bcb-1ac1-47aa-a557-63ba6c2011ab" Jul 15 19:21:43.086274 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:43.086238 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zk6t" Jul 15 19:21:43.086682 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:43.086375 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zk6t" podUID="c90bd526-03fe-4c0c-9d9d-5bd777b6bd87" Jul 15 19:21:44.086595 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:44.086564 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pn46g" Jul 15 19:21:44.087087 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:44.086558 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmqc9" Jul 15 19:21:44.087087 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:44.086694 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pn46g" podUID="05d64bcb-1ac1-47aa-a557-63ba6c2011ab" Jul 15 19:21:44.087087 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:44.086744 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmqc9" podUID="237b9375-100a-4d62-83f0-cb271ab8e0bb" Jul 15 19:21:44.715282 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:44.715255 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/237b9375-100a-4d62-83f0-cb271ab8e0bb-metrics-certs\") pod \"network-metrics-daemon-cmqc9\" (UID: \"237b9375-100a-4d62-83f0-cb271ab8e0bb\") " pod="openshift-multus/network-metrics-daemon-cmqc9" Jul 15 19:21:44.715392 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:44.715331 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/05d64bcb-1ac1-47aa-a557-63ba6c2011ab-original-pull-secret\") pod \"global-pull-secret-syncer-pn46g\" (UID: \"05d64bcb-1ac1-47aa-a557-63ba6c2011ab\") " pod="kube-system/global-pull-secret-syncer-pn46g" Jul 15 19:21:44.715462 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:44.715439 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 19:21:44.715462 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:44.715442 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:21:44.715552 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:44.715491 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05d64bcb-1ac1-47aa-a557-63ba6c2011ab-original-pull-secret podName:05d64bcb-1ac1-47aa-a557-63ba6c2011ab nodeName:}" failed. No retries permitted until 2026-07-15 19:22:00.715476528 +0000 UTC m=+34.292478177 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/05d64bcb-1ac1-47aa-a557-63ba6c2011ab-original-pull-secret") pod "global-pull-secret-syncer-pn46g" (UID: "05d64bcb-1ac1-47aa-a557-63ba6c2011ab") : object "kube-system"/"original-pull-secret" not registered Jul 15 19:21:44.715552 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:44.715506 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/237b9375-100a-4d62-83f0-cb271ab8e0bb-metrics-certs podName:237b9375-100a-4d62-83f0-cb271ab8e0bb nodeName:}" failed. No retries permitted until 2026-07-15 19:22:00.715498677 +0000 UTC m=+34.292500326 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/237b9375-100a-4d62-83f0-cb271ab8e0bb-metrics-certs") pod "network-metrics-daemon-cmqc9" (UID: "237b9375-100a-4d62-83f0-cb271ab8e0bb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:21:44.815803 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:44.815773 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8f8j\" (UniqueName: \"kubernetes.io/projected/c90bd526-03fe-4c0c-9d9d-5bd777b6bd87-kube-api-access-z8f8j\") pod \"network-check-target-5zk6t\" (UID: \"c90bd526-03fe-4c0c-9d9d-5bd777b6bd87\") " pod="openshift-network-diagnostics/network-check-target-5zk6t" Jul 15 19:21:44.815979 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:44.815950 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 19:21:44.815979 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:44.815979 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 19:21:44.816141 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:44.815992 2577 projected.go:196] Error preparing data for projected volume kube-api-access-z8f8j for pod openshift-network-diagnostics/network-check-target-5zk6t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:21:44.816141 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:44.816049 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c90bd526-03fe-4c0c-9d9d-5bd777b6bd87-kube-api-access-z8f8j podName:c90bd526-03fe-4c0c-9d9d-5bd777b6bd87 nodeName:}" failed. No retries permitted until 2026-07-15 19:22:00.81603441 +0000 UTC m=+34.393036065 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-z8f8j" (UniqueName: "kubernetes.io/projected/c90bd526-03fe-4c0c-9d9d-5bd777b6bd87-kube-api-access-z8f8j") pod "network-check-target-5zk6t" (UID: "c90bd526-03fe-4c0c-9d9d-5bd777b6bd87") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:21:45.086853 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:45.086813 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zk6t" Jul 15 19:21:45.087293 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:45.086961 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zk6t" podUID="c90bd526-03fe-4c0c-9d9d-5bd777b6bd87" Jul 15 19:21:46.086428 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:46.086396 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmqc9" Jul 15 19:21:46.086605 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:46.086531 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmqc9" podUID="237b9375-100a-4d62-83f0-cb271ab8e0bb" Jul 15 19:21:46.086605 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:46.086578 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pn46g" Jul 15 19:21:46.086737 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:46.086679 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pn46g" podUID="05d64bcb-1ac1-47aa-a557-63ba6c2011ab" Jul 15 19:21:47.086705 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:47.086669 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zk6t" Jul 15 19:21:47.087101 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:47.086762 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zk6t" podUID="c90bd526-03fe-4c0c-9d9d-5bd777b6bd87" Jul 15 19:21:48.086438 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:48.086405 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmqc9" Jul 15 19:21:48.086438 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:48.086432 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pn46g" Jul 15 19:21:48.086652 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:48.086528 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmqc9" podUID="237b9375-100a-4d62-83f0-cb271ab8e0bb" Jul 15 19:21:48.086702 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:48.086651 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pn46g" podUID="05d64bcb-1ac1-47aa-a557-63ba6c2011ab" Jul 15 19:21:49.086959 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:49.086804 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zk6t" Jul 15 19:21:49.087419 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:49.087036 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zk6t" podUID="c90bd526-03fe-4c0c-9d9d-5bd777b6bd87" Jul 15 19:21:49.172330 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:49.172280 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xx47b" event={"ID":"1820cfdf-910d-42da-bafb-fb076d565b78","Type":"ContainerStarted","Data":"d24051726ed2d174c417b2d0fb3a67d06df30d3064f3a5516fc8d67b93ebb2e9"} Jul 15 19:21:49.173748 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:49.173717 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j99qj" event={"ID":"ec13993c-0b5b-456f-81cb-542da64f3d23","Type":"ContainerStarted","Data":"73451aa34328f8d19a0cb5349d1e6adba827defbb6cdcc4d3a0ccdaf97f75c06"} Jul 15 19:21:49.175347 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:49.175304 2577 generic.go:356] "Generic (PLEG): container finished" podID="23786db4-0338-4d64-9ea3-be6ebc44d74e" containerID="445644bef3f656c03342a273df2af48763cf296541892e69310d079e740ab865" exitCode=0 Jul 15 19:21:49.175347 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:49.175337 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f7pg5" event={"ID":"23786db4-0338-4d64-9ea3-be6ebc44d74e","Type":"ContainerDied","Data":"445644bef3f656c03342a273df2af48763cf296541892e69310d079e740ab865"} Jul 15 19:21:49.180144 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:49.180117 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-c8jtk" event={"ID":"b6d81326-425e-4a61-a7db-8790d035bd50","Type":"ContainerStarted","Data":"5343862ff021f23a57ee81316cb49b88e3ac511b3010190d893a3b699b640923"} Jul 15 19:21:49.182665 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:49.182633 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fw6vl_89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21/ovn-acl-logging/0.log" Jul 15 19:21:49.183051 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:49.183022 2577 generic.go:356] "Generic (PLEG): container finished" podID="89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21" containerID="a918668921155231b8eb275098285ead88e1bdacf10a04d7c51b0bf73a176a44" exitCode=1 Jul 15 19:21:49.183149 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:49.183088 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" event={"ID":"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21","Type":"ContainerStarted","Data":"f6f6843da50b61398f968edf96bf5c7713310f5465b189f9fcff71fc8f5fe62d"} Jul 15 19:21:49.183212 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:49.183150 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" event={"ID":"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21","Type":"ContainerStarted","Data":"eebc8d8e96fc06b8fb2f897b22ff82f2abe826062142f1715829f3307629ac97"} Jul 15 19:21:49.183212 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:49.183180 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" event={"ID":"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21","Type":"ContainerDied","Data":"a918668921155231b8eb275098285ead88e1bdacf10a04d7c51b0bf73a176a44"} Jul 15 19:21:49.183379 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:49.183213 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" event={"ID":"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21","Type":"ContainerStarted","Data":"fe1156aeb1e631e26eaddf6eba3215558104351615ca30ddc5d6ad1b41e25d57"} Jul 15 19:21:49.185289 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:49.185242 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7m9g6" event={"ID":"61c3693e-12af-4a9e-81d9-f66cc6520c73","Type":"ContainerStarted","Data":"454fa073d255e4960f9cddd7972d1c0eeb23ee59bd8a637955c87209ff6288f8"} Jul 15 19:21:49.186609 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:49.186565 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-xx47b" podStartSLOduration=6.744922657 podStartE2EDuration="22.186550646s" totalImagesPullingTime="15.441627989s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:21:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:21:49.186469997 +0000 UTC m=+22.763471670" watchObservedRunningTime="2026-07-15 19:21:49.186550646 +0000 UTC m=+22.763552317" Jul 15 19:21:49.187747 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:49.187679 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-62.ec2.internal" event={"ID":"64ae407aa9b62ec31a4fe3be45950ba9","Type":"ContainerStarted","Data":"0a2719dc3d248c2fa43f11a8b83c748580a824725e7ea886aa1ff5483eb3a4ae"} Jul 15 19:21:49.189184 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:49.189164 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9vzln" event={"ID":"299d8c27-2943-40f5-8810-67bcfcbbb3a2","Type":"ContainerStarted","Data":"4329b33dc2fac06481c7f3b632f68ce34a6acbdafff9fba6f69c328884d2c438"} Jul 15 19:21:49.244556 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:49.244403 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-c8jtk" podStartSLOduration=2.821990516 podStartE2EDuration="22.244387529s" totalImagesPullingTime="19.422397013s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:21:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:21:49.205923332 +0000 UTC m=+22.782925002" watchObservedRunningTime="2026-07-15 19:21:49.244387529 +0000 UTC m=+22.821389199" Jul 15 19:21:49.272291 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:49.272227 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-7m9g6" podStartSLOduration=2.805604546 podStartE2EDuration="22.272206224s" totalImagesPullingTime="19.466601678s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:21:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:21:49.271760535 +0000 UTC m=+22.848762205" watchObservedRunningTime="2026-07-15 19:21:49.272206224 +0000 UTC m=+22.849207920" Jul 15 19:21:49.300450 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:49.300405 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-9vzln" podStartSLOduration=2.86349853 podStartE2EDuration="22.30039096s" totalImagesPullingTime="19.43689243s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:21:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:21:49.300039396 +0000 UTC m=+22.877041068" watchObservedRunningTime="2026-07-15 19:21:49.30039096 +0000 UTC m=+22.877392631" Jul 15 19:21:49.324273 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:49.324220 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-62.ec2.internal" podStartSLOduration=21.324197289 podStartE2EDuration="21.324197289s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:21:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:21:49.323501348 +0000 UTC m=+22.900503019" watchObservedRunningTime="2026-07-15 19:21:49.324197289 +0000 UTC m=+22.901198961" Jul 15 19:21:50.086481 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:50.086450 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pn46g" Jul 15 19:21:50.086625 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:50.086454 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmqc9" Jul 15 19:21:50.086625 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:50.086598 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pn46g" podUID="05d64bcb-1ac1-47aa-a557-63ba6c2011ab" Jul 15 19:21:50.086730 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:50.086669 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmqc9" podUID="237b9375-100a-4d62-83f0-cb271ab8e0bb" Jul 15 19:21:50.193276 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:50.193233 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-48pqd" event={"ID":"71aa4ce2-c8ac-4224-9226-ba125d49f534","Type":"ContainerStarted","Data":"d4449ded111e4a6865ed5eea1f3c1079f24a03cd8880e8063d28520f13705aeb"} Jul 15 19:21:50.198876 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:50.198848 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hbtrt" event={"ID":"35abc39e-4584-4a00-b15e-5c25c1f8ce41","Type":"ContainerStarted","Data":"8db6f7f8e63c9521303504d17e5aab1c95aa91156c8d892491edf347c0ea7563"} Jul 15 19:21:50.201890 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:50.201870 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fw6vl_89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21/ovn-acl-logging/0.log" Jul 15 19:21:50.202235 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:50.202210 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" event={"ID":"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21","Type":"ContainerStarted","Data":"2e6198d4f108652af33d75900eb3a98262a636f6616de78c494afe8853498635"} Jul 15 19:21:50.202369 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:50.202284 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" event={"ID":"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21","Type":"ContainerStarted","Data":"524d66094c035739277fc24ba76d2aaa57b7ed52c64a669b7bdaf95833c4b7d8"} Jul 15 19:21:50.217162 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:50.217122 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-48pqd" podStartSLOduration=3.2699693 podStartE2EDuration="23.217110414s" totalImagesPullingTime="19.947141114s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:21:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:21:50.216688106 +0000 UTC m=+23.793689779" watchObservedRunningTime="2026-07-15 19:21:50.217110414 +0000 UTC m=+23.794112085" Jul 15 19:21:50.231986 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:50.231947 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-hbtrt" podStartSLOduration=3.808801405 podStartE2EDuration="23.231934142s" totalImagesPullingTime="19.423132737s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:21:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:21:50.231607865 +0000 UTC m=+23.808609551" watchObservedRunningTime="2026-07-15 19:21:50.231934142 +0000 UTC m=+23.808935813" Jul 15 19:21:50.529399 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:50.529300 2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 15 19:21:50.952619 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:50.952511 2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-15T19:21:50.529394733Z","UUID":"f1bea031-b810-4789-8d60-c9ae159d7e15","Handler":null,"Name":"","Endpoint":""} Jul 15 19:21:50.954016 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:50.953983 2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 15 19:21:50.954143 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:50.954022 2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 15 19:21:51.086898 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:51.086869 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zk6t" Jul 15 19:21:51.087056 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:51.086993 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zk6t" podUID="c90bd526-03fe-4c0c-9d9d-5bd777b6bd87" Jul 15 19:21:51.206364 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:51.206279 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j99qj" event={"ID":"ec13993c-0b5b-456f-81cb-542da64f3d23","Type":"ContainerStarted","Data":"526a96087b49208a3053a01dcd9229bc733770e1d30dcd9adbd7163295700414"} Jul 15 19:21:52.087303 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:52.087091 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmqc9" Jul 15 19:21:52.087524 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:52.087163 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pn46g" Jul 15 19:21:52.087524 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:52.087405 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmqc9" podUID="237b9375-100a-4d62-83f0-cb271ab8e0bb" Jul 15 19:21:52.087524 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:52.087468 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pn46g" podUID="05d64bcb-1ac1-47aa-a557-63ba6c2011ab" Jul 15 19:21:52.211085 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:52.210843 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j99qj" event={"ID":"ec13993c-0b5b-456f-81cb-542da64f3d23","Type":"ContainerStarted","Data":"1da3a95766968669448dbd0fe0705d7884fa50d9a3b0067d643883846cea57bd"} Jul 15 19:21:52.214089 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:52.214069 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fw6vl_89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21/ovn-acl-logging/0.log" Jul 15 19:21:52.214494 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:52.214473 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" event={"ID":"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21","Type":"ContainerStarted","Data":"76218ce7777806ca3b053b6c520f1006a51a576c5d69b9848be6ec74451edf70"} Jul 15 19:21:52.232384 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:52.232331 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j99qj" podStartSLOduration=3.067104401 podStartE2EDuration="25.23229764s" totalImagesPullingTime="22.165193239s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:21:27 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:21:52.231694582 +0000 UTC m=+25.808696255" watchObservedRunningTime="2026-07-15 19:21:52.23229764 +0000 UTC m=+25.809299344" Jul 15 19:21:52.299181 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:52.299149 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-xx47b" Jul 15 19:21:52.299763 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:52.299741 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-xx47b" Jul 15 19:21:53.087167 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:53.087131 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zk6t" Jul 15 19:21:53.087472 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:53.087272 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zk6t" podUID="c90bd526-03fe-4c0c-9d9d-5bd777b6bd87" Jul 15 19:21:53.217230 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:53.217203 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-xx47b" Jul 15 19:21:53.217782 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:53.217738 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-xx47b" Jul 15 19:21:54.086667 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:54.086497 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmqc9" Jul 15 19:21:54.086841 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:54.086555 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pn46g" Jul 15 19:21:54.086841 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:54.086748 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmqc9" podUID="237b9375-100a-4d62-83f0-cb271ab8e0bb" Jul 15 19:21:54.086934 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:54.086847 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pn46g" podUID="05d64bcb-1ac1-47aa-a557-63ba6c2011ab" Jul 15 19:21:54.220125 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:54.220099 2577 generic.go:356] "Generic (PLEG): container finished" podID="23786db4-0338-4d64-9ea3-be6ebc44d74e" containerID="70438ae73d09f0653bfb1714fbd945a4e37225340baa7b46e234e222a82d94a7" exitCode=0 Jul 15 19:21:54.220721 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:54.220179 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f7pg5" event={"ID":"23786db4-0338-4d64-9ea3-be6ebc44d74e","Type":"ContainerDied","Data":"70438ae73d09f0653bfb1714fbd945a4e37225340baa7b46e234e222a82d94a7"} Jul 15 19:21:54.223709 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:54.223692 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fw6vl_89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21/ovn-acl-logging/0.log" Jul 15 19:21:54.224110 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:54.224083 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" event={"ID":"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21","Type":"ContainerStarted","Data":"2352a40bc0058eeb8a50d1dfc6980e0561ec4ad256b7648b2214cf721fdc3f7b"} Jul 15 19:21:54.224345 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:54.224329 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:54.224426 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:54.224394 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:54.224476 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:54.224440 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:54.224476 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:54.224453 2577 scope.go:117] "RemoveContainer" containerID="a918668921155231b8eb275098285ead88e1bdacf10a04d7c51b0bf73a176a44" Jul 15 19:21:54.239750 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:54.239728 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:54.241751 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:54.241731 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:21:55.086270 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:55.086246 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zk6t" Jul 15 19:21:55.086450 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:55.086353 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zk6t" podUID="c90bd526-03fe-4c0c-9d9d-5bd777b6bd87" Jul 15 19:21:55.230734 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:55.230703 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fw6vl_89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21/ovn-acl-logging/0.log" Jul 15 19:21:55.231500 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:55.231262 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" event={"ID":"89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21","Type":"ContainerStarted","Data":"f6624caae821f095469e62d2b7b964847af226b44ccbfeaa1f244ec1d6c806d0"} Jul 15 19:21:55.264081 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:55.264027 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" podStartSLOduration=8.754878022 podStartE2EDuration="28.264007795s" totalImagesPullingTime="19.509129773s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:21:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:21:55.263400159 +0000 UTC m=+28.840401854" watchObservedRunningTime="2026-07-15 19:21:55.264007795 +0000 UTC m=+28.841009468" Jul 15 19:21:55.888982 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:55.888918 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-pn46g"] Jul 15 19:21:55.889105 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:55.889024 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pn46g" Jul 15 19:21:55.889155 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:55.889111 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pn46g" podUID="05d64bcb-1ac1-47aa-a557-63ba6c2011ab" Jul 15 19:21:55.896726 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:55.896697 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cmqc9"] Jul 15 19:21:55.896870 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:55.896809 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmqc9" Jul 15 19:21:55.896936 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:55.896916 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmqc9" podUID="237b9375-100a-4d62-83f0-cb271ab8e0bb" Jul 15 19:21:55.900438 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:55.900418 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5zk6t"] Jul 15 19:21:55.900529 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:55.900481 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zk6t" Jul 15 19:21:55.900569 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:55.900542 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zk6t" podUID="c90bd526-03fe-4c0c-9d9d-5bd777b6bd87" Jul 15 19:21:56.235135 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:56.234913 2577 generic.go:356] "Generic (PLEG): container finished" podID="23786db4-0338-4d64-9ea3-be6ebc44d74e" containerID="706ec8f67c658177e84704690b47ddf679d047da458efd52cd09a0cc088c964e" exitCode=0 Jul 15 19:21:56.235486 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:56.234986 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f7pg5" event={"ID":"23786db4-0338-4d64-9ea3-be6ebc44d74e","Type":"ContainerDied","Data":"706ec8f67c658177e84704690b47ddf679d047da458efd52cd09a0cc088c964e"} Jul 15 19:21:58.086289 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:58.086261 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zk6t" Jul 15 19:21:58.086764 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:58.086389 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pn46g" Jul 15 19:21:58.086764 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:58.086402 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zk6t" podUID="c90bd526-03fe-4c0c-9d9d-5bd777b6bd87" Jul 15 19:21:58.086764 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:58.086419 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmqc9" Jul 15 19:21:58.086764 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:58.086493 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pn46g" podUID="05d64bcb-1ac1-47aa-a557-63ba6c2011ab" Jul 15 19:21:58.086764 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:21:58.086589 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmqc9" podUID="237b9375-100a-4d62-83f0-cb271ab8e0bb" Jul 15 19:21:58.242107 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:58.242076 2577 generic.go:356] "Generic (PLEG): container finished" podID="23786db4-0338-4d64-9ea3-be6ebc44d74e" containerID="774ea4febaf81c8a92ccae7a9ae9b42c3cc4db82d270b163b8bec3a5477e61da" exitCode=0 Jul 15 19:21:58.242271 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:21:58.242135 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f7pg5" event={"ID":"23786db4-0338-4d64-9ea3-be6ebc44d74e","Type":"ContainerDied","Data":"774ea4febaf81c8a92ccae7a9ae9b42c3cc4db82d270b163b8bec3a5477e61da"} Jul 15 19:22:00.086871 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:00.086835 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zk6t" Jul 15 19:22:00.087608 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:00.086871 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pn46g" Jul 15 19:22:00.087608 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:00.086871 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmqc9" Jul 15 19:22:00.087608 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:00.086965 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zk6t" podUID="c90bd526-03fe-4c0c-9d9d-5bd777b6bd87" Jul 15 19:22:00.087608 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:00.087083 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pn46g" podUID="05d64bcb-1ac1-47aa-a557-63ba6c2011ab" Jul 15 19:22:00.087608 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:00.087175 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cmqc9" podUID="237b9375-100a-4d62-83f0-cb271ab8e0bb" Jul 15 19:22:00.734639 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:00.734604 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/05d64bcb-1ac1-47aa-a557-63ba6c2011ab-original-pull-secret\") pod \"global-pull-secret-syncer-pn46g\" (UID: \"05d64bcb-1ac1-47aa-a557-63ba6c2011ab\") " pod="kube-system/global-pull-secret-syncer-pn46g" Jul 15 19:22:00.734836 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:00.734662 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/237b9375-100a-4d62-83f0-cb271ab8e0bb-metrics-certs\") pod \"network-metrics-daemon-cmqc9\" (UID: \"237b9375-100a-4d62-83f0-cb271ab8e0bb\") " pod="openshift-multus/network-metrics-daemon-cmqc9" Jul 15 19:22:00.734836 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:00.734818 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:22:00.734962 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:00.734844 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 19:22:00.734962 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:00.734871 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/237b9375-100a-4d62-83f0-cb271ab8e0bb-metrics-certs podName:237b9375-100a-4d62-83f0-cb271ab8e0bb nodeName:}" failed. No retries permitted until 2026-07-15 19:22:32.734856705 +0000 UTC m=+66.311858354 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/237b9375-100a-4d62-83f0-cb271ab8e0bb-metrics-certs") pod "network-metrics-daemon-cmqc9" (UID: "237b9375-100a-4d62-83f0-cb271ab8e0bb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:22:00.734962 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:00.734903 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05d64bcb-1ac1-47aa-a557-63ba6c2011ab-original-pull-secret podName:05d64bcb-1ac1-47aa-a557-63ba6c2011ab nodeName:}" failed. No retries permitted until 2026-07-15 19:22:32.734884914 +0000 UTC m=+66.311886563 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/05d64bcb-1ac1-47aa-a557-63ba6c2011ab-original-pull-secret") pod "global-pull-secret-syncer-pn46g" (UID: "05d64bcb-1ac1-47aa-a557-63ba6c2011ab") : object "kube-system"/"original-pull-secret" not registered Jul 15 19:22:00.835281 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:00.835246 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8f8j\" (UniqueName: \"kubernetes.io/projected/c90bd526-03fe-4c0c-9d9d-5bd777b6bd87-kube-api-access-z8f8j\") pod \"network-check-target-5zk6t\" (UID: \"c90bd526-03fe-4c0c-9d9d-5bd777b6bd87\") " pod="openshift-network-diagnostics/network-check-target-5zk6t" Jul 15 19:22:00.835479 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:00.835460 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 19:22:00.835529 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:00.835517 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 19:22:00.835569 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:00.835531 2577 projected.go:196] Error preparing data for projected volume kube-api-access-z8f8j for pod openshift-network-diagnostics/network-check-target-5zk6t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:22:00.835601 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:00.835591 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c90bd526-03fe-4c0c-9d9d-5bd777b6bd87-kube-api-access-z8f8j podName:c90bd526-03fe-4c0c-9d9d-5bd777b6bd87 nodeName:}" failed. No retries permitted until 2026-07-15 19:22:32.835572629 +0000 UTC m=+66.412574300 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-z8f8j" (UniqueName: "kubernetes.io/projected/c90bd526-03fe-4c0c-9d9d-5bd777b6bd87-kube-api-access-z8f8j") pod "network-check-target-5zk6t" (UID: "c90bd526-03fe-4c0c-9d9d-5bd777b6bd87") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:22:01.735812 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:01.735556 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-62.ec2.internal" event="NodeReady" Jul 15 19:22:01.736190 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:01.735852 2577 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 15 19:22:01.770194 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:01.770164 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-w4b96"] Jul 15 19:22:01.799148 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:01.799079 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-9db745d49-xxf8k"] Jul 15 19:22:01.799331 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:01.799203 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-w4b96" Jul 15 19:22:01.801680 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:01.801641 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-j42p2\"" Jul 15 19:22:01.801925 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:01.801908 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 15 19:22:01.802152 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:01.802135 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 15 19:22:01.820961 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:01.820933 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-w4b96"] Jul 15 19:22:01.821062 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:01.820977 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-5zrgr"] Jul 15 19:22:01.821062 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:01.821038 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9db745d49-xxf8k" Jul 15 19:22:01.823466 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:01.823425 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 15 19:22:01.823580 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:01.823565 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-r4g6n\"" Jul 15 19:22:01.823648 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:01.823591 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 15 19:22:01.823648 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:01.823598 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 15 19:22:01.829060 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:01.829039 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 15 19:22:01.843551 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:01.843530 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-rdqmw"] Jul 15 19:22:01.843643 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:01.843633 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5zrgr" Jul 15 19:22:01.849009 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:01.848982 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 15 19:22:01.849009 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:01.848992 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bw2gg\"" Jul 15 19:22:01.849220 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:01.849196 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 15 19:22:01.870790 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:01.870773 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-9db745d49-xxf8k"] Jul 15 19:22:01.870902 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:01.870817 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rdqmw"] Jul 15 19:22:01.870902 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:01.870847 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5zrgr"] Jul 15 19:22:01.870902 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:01.870816 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rdqmw" Jul 15 19:22:01.873132 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:01.873114 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 15 19:22:01.873238 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:01.873149 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 15 19:22:01.873352 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:01.873336 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 15 19:22:01.873452 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:01.873439 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-dghvh\"" Jul 15 19:22:01.943984 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:01.943957 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-trusted-ca\") pod \"image-registry-9db745d49-xxf8k\" (UID: \"1235737d-3c8a-4c3a-af1a-dfd3351d72fb\") " pod="openshift-image-registry/image-registry-9db745d49-xxf8k" Jul 15 19:22:01.943984 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:01.943988 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56c7bcac-8a82-4c32-8658-1db0bad8d228-tmp-dir\") pod \"dns-default-5zrgr\" (UID: \"56c7bcac-8a82-4c32-8658-1db0bad8d228\") " pod="openshift-dns/dns-default-5zrgr" Jul 15 19:22:01.944213 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:01.944008 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkftt\" (UniqueName: \"kubernetes.io/projected/56c7bcac-8a82-4c32-8658-1db0bad8d228-kube-api-access-lkftt\") pod \"dns-default-5zrgr\" (UID: \"56c7bcac-8a82-4c32-8658-1db0bad8d228\") " pod="openshift-dns/dns-default-5zrgr" Jul 15 19:22:01.944213 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:01.944110 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kb7v2\" (UniqueName: \"kubernetes.io/projected/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-kube-api-access-kb7v2\") pod \"image-registry-9db745d49-xxf8k\" (UID: \"1235737d-3c8a-4c3a-af1a-dfd3351d72fb\") " pod="openshift-image-registry/image-registry-9db745d49-xxf8k" Jul 15 19:22:01.944213 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:01.944147 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/56c7bcac-8a82-4c32-8658-1db0bad8d228-config-volume\") pod \"dns-default-5zrgr\" (UID: \"56c7bcac-8a82-4c32-8658-1db0bad8d228\") " pod="openshift-dns/dns-default-5zrgr" Jul 15 19:22:01.944213 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:01.944178 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/152d1d6b-5ee1-40c2-80a8-388091a57c58-nginx-conf\") pod \"networking-console-plugin-6cf856758c-w4b96\" (UID: \"152d1d6b-5ee1-40c2-80a8-388091a57c58\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-w4b96" Jul 15 19:22:01.944456 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:01.944232 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-registry-certificates\") pod \"image-registry-9db745d49-xxf8k\" (UID: \"1235737d-3c8a-4c3a-af1a-dfd3351d72fb\") " pod="openshift-image-registry/image-registry-9db745d49-xxf8k" Jul 15 19:22:01.944456 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:01.944264 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-bound-sa-token\") pod \"image-registry-9db745d49-xxf8k\" (UID: \"1235737d-3c8a-4c3a-af1a-dfd3351d72fb\") " pod="openshift-image-registry/image-registry-9db745d49-xxf8k" Jul 15 19:22:01.944456 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:01.944331 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-ca-trust-extracted\") pod \"image-registry-9db745d49-xxf8k\" (UID: \"1235737d-3c8a-4c3a-af1a-dfd3351d72fb\") " pod="openshift-image-registry/image-registry-9db745d49-xxf8k" Jul 15 19:22:01.944456 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:01.944381 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/56c7bcac-8a82-4c32-8658-1db0bad8d228-metrics-tls\") pod \"dns-default-5zrgr\" (UID: \"56c7bcac-8a82-4c32-8658-1db0bad8d228\") " pod="openshift-dns/dns-default-5zrgr" Jul 15 19:22:01.944456 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:01.944417 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-registry-tls\") pod \"image-registry-9db745d49-xxf8k\" (UID: \"1235737d-3c8a-4c3a-af1a-dfd3351d72fb\") " pod="openshift-image-registry/image-registry-9db745d49-xxf8k" Jul 15 19:22:01.944643 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:01.944458 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/152d1d6b-5ee1-40c2-80a8-388091a57c58-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-w4b96\" (UID: \"152d1d6b-5ee1-40c2-80a8-388091a57c58\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-w4b96" Jul 15 19:22:01.944643 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:01.944503 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-image-registry-private-configuration\") pod \"image-registry-9db745d49-xxf8k\" (UID: \"1235737d-3c8a-4c3a-af1a-dfd3351d72fb\") " pod="openshift-image-registry/image-registry-9db745d49-xxf8k" Jul 15 19:22:01.944643 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:01.944531 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-installation-pull-secrets\") pod \"image-registry-9db745d49-xxf8k\" (UID: \"1235737d-3c8a-4c3a-af1a-dfd3351d72fb\") " pod="openshift-image-registry/image-registry-9db745d49-xxf8k" Jul 15 19:22:02.045097 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:02.045025 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/56c7bcac-8a82-4c32-8658-1db0bad8d228-metrics-tls\") pod \"dns-default-5zrgr\" (UID: \"56c7bcac-8a82-4c32-8658-1db0bad8d228\") " pod="openshift-dns/dns-default-5zrgr" Jul 15 19:22:02.045097 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:02.045069 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-registry-tls\") pod \"image-registry-9db745d49-xxf8k\" (UID: \"1235737d-3c8a-4c3a-af1a-dfd3351d72fb\") " pod="openshift-image-registry/image-registry-9db745d49-xxf8k" Jul 15 19:22:02.045097 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:02.045094 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/152d1d6b-5ee1-40c2-80a8-388091a57c58-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-w4b96\" (UID: \"152d1d6b-5ee1-40c2-80a8-388091a57c58\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-w4b96" Jul 15 19:22:02.045376 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:02.045198 2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 15 19:22:02.045376 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:02.045227 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 19:22:02.045376 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:02.045196 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 19:22:02.045376 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:02.045272 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9db745d49-xxf8k: secret "image-registry-tls" not found Jul 15 19:22:02.045376 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:02.045275 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/152d1d6b-5ee1-40c2-80a8-388091a57c58-networking-console-plugin-cert podName:152d1d6b-5ee1-40c2-80a8-388091a57c58 nodeName:}" failed. No retries permitted until 2026-07-15 19:22:02.54525692 +0000 UTC m=+36.122258572 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/152d1d6b-5ee1-40c2-80a8-388091a57c58-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-w4b96" (UID: "152d1d6b-5ee1-40c2-80a8-388091a57c58") : secret "networking-console-plugin-cert" not found Jul 15 19:22:02.045376 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:02.045329 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56c7bcac-8a82-4c32-8658-1db0bad8d228-metrics-tls podName:56c7bcac-8a82-4c32-8658-1db0bad8d228 nodeName:}" failed. No retries permitted until 2026-07-15 19:22:02.545297359 +0000 UTC m=+36.122299013 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/56c7bcac-8a82-4c32-8658-1db0bad8d228-metrics-tls") pod "dns-default-5zrgr" (UID: "56c7bcac-8a82-4c32-8658-1db0bad8d228") : secret "dns-default-metrics-tls" not found Jul 15 19:22:02.045376 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:02.045349 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-image-registry-private-configuration\") pod \"image-registry-9db745d49-xxf8k\" (UID: \"1235737d-3c8a-4c3a-af1a-dfd3351d72fb\") " pod="openshift-image-registry/image-registry-9db745d49-xxf8k" Jul 15 19:22:02.045762 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:02.045382 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-installation-pull-secrets\") pod \"image-registry-9db745d49-xxf8k\" (UID: \"1235737d-3c8a-4c3a-af1a-dfd3351d72fb\") " pod="openshift-image-registry/image-registry-9db745d49-xxf8k" Jul 15 19:22:02.045762 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:02.045407 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cf8428ad-0fb7-4108-a41f-8fd01ec35c5c-cert\") pod \"ingress-canary-rdqmw\" (UID: \"cf8428ad-0fb7-4108-a41f-8fd01ec35c5c\") " pod="openshift-ingress-canary/ingress-canary-rdqmw" Jul 15 19:22:02.045762 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:02.045420 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-registry-tls podName:1235737d-3c8a-4c3a-af1a-dfd3351d72fb nodeName:}" failed. No retries permitted until 2026-07-15 19:22:02.545404617 +0000 UTC m=+36.122406280 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-registry-tls") pod "image-registry-9db745d49-xxf8k" (UID: "1235737d-3c8a-4c3a-af1a-dfd3351d72fb") : secret "image-registry-tls" not found Jul 15 19:22:02.045762 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:02.045459 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-trusted-ca\") pod \"image-registry-9db745d49-xxf8k\" (UID: \"1235737d-3c8a-4c3a-af1a-dfd3351d72fb\") " pod="openshift-image-registry/image-registry-9db745d49-xxf8k" Jul 15 19:22:02.045762 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:02.045482 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56c7bcac-8a82-4c32-8658-1db0bad8d228-tmp-dir\") pod \"dns-default-5zrgr\" (UID: \"56c7bcac-8a82-4c32-8658-1db0bad8d228\") " pod="openshift-dns/dns-default-5zrgr" Jul 15 19:22:02.045762 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:02.045502 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkftt\" (UniqueName: \"kubernetes.io/projected/56c7bcac-8a82-4c32-8658-1db0bad8d228-kube-api-access-lkftt\") pod \"dns-default-5zrgr\" (UID: \"56c7bcac-8a82-4c32-8658-1db0bad8d228\") " pod="openshift-dns/dns-default-5zrgr" Jul 15 19:22:02.045762 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:02.045525 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvcmw\" (UniqueName: \"kubernetes.io/projected/cf8428ad-0fb7-4108-a41f-8fd01ec35c5c-kube-api-access-wvcmw\") pod \"ingress-canary-rdqmw\" (UID: \"cf8428ad-0fb7-4108-a41f-8fd01ec35c5c\") " pod="openshift-ingress-canary/ingress-canary-rdqmw" Jul 15 19:22:02.045762 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:02.045567 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kb7v2\" (UniqueName: \"kubernetes.io/projected/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-kube-api-access-kb7v2\") pod \"image-registry-9db745d49-xxf8k\" (UID: \"1235737d-3c8a-4c3a-af1a-dfd3351d72fb\") " pod="openshift-image-registry/image-registry-9db745d49-xxf8k" Jul 15 19:22:02.045762 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:02.045591 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/56c7bcac-8a82-4c32-8658-1db0bad8d228-config-volume\") pod \"dns-default-5zrgr\" (UID: \"56c7bcac-8a82-4c32-8658-1db0bad8d228\") " pod="openshift-dns/dns-default-5zrgr" Jul 15 19:22:02.045762 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:02.045618 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/152d1d6b-5ee1-40c2-80a8-388091a57c58-nginx-conf\") pod \"networking-console-plugin-6cf856758c-w4b96\" (UID: \"152d1d6b-5ee1-40c2-80a8-388091a57c58\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-w4b96" Jul 15 19:22:02.045762 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:02.045662 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-registry-certificates\") pod \"image-registry-9db745d49-xxf8k\" (UID: \"1235737d-3c8a-4c3a-af1a-dfd3351d72fb\") " pod="openshift-image-registry/image-registry-9db745d49-xxf8k" Jul 15 19:22:02.045762 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:02.045715 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-bound-sa-token\") pod \"image-registry-9db745d49-xxf8k\" (UID: \"1235737d-3c8a-4c3a-af1a-dfd3351d72fb\") " pod="openshift-image-registry/image-registry-9db745d49-xxf8k" Jul 15 19:22:02.045762 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:02.045746 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-ca-trust-extracted\") pod \"image-registry-9db745d49-xxf8k\" (UID: \"1235737d-3c8a-4c3a-af1a-dfd3351d72fb\") " pod="openshift-image-registry/image-registry-9db745d49-xxf8k" Jul 15 19:22:02.046340 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:02.045821 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56c7bcac-8a82-4c32-8658-1db0bad8d228-tmp-dir\") pod \"dns-default-5zrgr\" (UID: \"56c7bcac-8a82-4c32-8658-1db0bad8d228\") " pod="openshift-dns/dns-default-5zrgr" Jul 15 19:22:02.046340 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:02.046115 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-ca-trust-extracted\") pod \"image-registry-9db745d49-xxf8k\" (UID: \"1235737d-3c8a-4c3a-af1a-dfd3351d72fb\") " pod="openshift-image-registry/image-registry-9db745d49-xxf8k" Jul 15 19:22:02.046972 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:02.046560 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/56c7bcac-8a82-4c32-8658-1db0bad8d228-config-volume\") pod \"dns-default-5zrgr\" (UID: \"56c7bcac-8a82-4c32-8658-1db0bad8d228\") " pod="openshift-dns/dns-default-5zrgr" Jul 15 19:22:02.046972 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:02.046787 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-trusted-ca\") pod \"image-registry-9db745d49-xxf8k\" (UID: \"1235737d-3c8a-4c3a-af1a-dfd3351d72fb\") " pod="openshift-image-registry/image-registry-9db745d49-xxf8k" Jul 15 19:22:02.046972 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:02.046824 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/152d1d6b-5ee1-40c2-80a8-388091a57c58-nginx-conf\") pod \"networking-console-plugin-6cf856758c-w4b96\" (UID: \"152d1d6b-5ee1-40c2-80a8-388091a57c58\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-w4b96" Jul 15 19:22:02.046972 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:02.046928 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-registry-certificates\") pod \"image-registry-9db745d49-xxf8k\" (UID: \"1235737d-3c8a-4c3a-af1a-dfd3351d72fb\") " pod="openshift-image-registry/image-registry-9db745d49-xxf8k" Jul 15 19:22:02.049624 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:02.049606 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-installation-pull-secrets\") pod \"image-registry-9db745d49-xxf8k\" (UID: \"1235737d-3c8a-4c3a-af1a-dfd3351d72fb\") " pod="openshift-image-registry/image-registry-9db745d49-xxf8k" Jul 15 19:22:02.049709 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:02.049641 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-image-registry-private-configuration\") pod \"image-registry-9db745d49-xxf8k\" (UID: \"1235737d-3c8a-4c3a-af1a-dfd3351d72fb\") " pod="openshift-image-registry/image-registry-9db745d49-xxf8k" Jul 15 19:22:02.055002 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:02.054978 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkftt\" (UniqueName: \"kubernetes.io/projected/56c7bcac-8a82-4c32-8658-1db0bad8d228-kube-api-access-lkftt\") pod \"dns-default-5zrgr\" (UID: \"56c7bcac-8a82-4c32-8658-1db0bad8d228\") " pod="openshift-dns/dns-default-5zrgr" Jul 15 19:22:02.055256 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:02.055232 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-bound-sa-token\") pod \"image-registry-9db745d49-xxf8k\" (UID: \"1235737d-3c8a-4c3a-af1a-dfd3351d72fb\") " pod="openshift-image-registry/image-registry-9db745d49-xxf8k" Jul 15 19:22:02.055602 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:02.055580 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kb7v2\" (UniqueName: \"kubernetes.io/projected/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-kube-api-access-kb7v2\") pod \"image-registry-9db745d49-xxf8k\" (UID: \"1235737d-3c8a-4c3a-af1a-dfd3351d72fb\") " pod="openshift-image-registry/image-registry-9db745d49-xxf8k" Jul 15 19:22:02.086327 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:02.086293 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zk6t" Jul 15 19:22:02.086423 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:02.086335 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pn46g" Jul 15 19:22:02.086423 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:02.086340 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmqc9" Jul 15 19:22:02.088667 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:02.088638 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jwgm9\"" Jul 15 19:22:02.088760 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:02.088737 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 15 19:22:02.088862 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:02.088839 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 15 19:22:02.088963 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:02.088883 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-dmtm7\"" Jul 15 19:22:02.088963 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:02.088944 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 15 19:22:02.089070 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:02.088976 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 15 19:22:02.147029 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:02.147009 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cf8428ad-0fb7-4108-a41f-8fd01ec35c5c-cert\") pod \"ingress-canary-rdqmw\" (UID: \"cf8428ad-0fb7-4108-a41f-8fd01ec35c5c\") " pod="openshift-ingress-canary/ingress-canary-rdqmw" Jul 15 19:22:02.147148 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:02.147039 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvcmw\" (UniqueName: \"kubernetes.io/projected/cf8428ad-0fb7-4108-a41f-8fd01ec35c5c-kube-api-access-wvcmw\") pod \"ingress-canary-rdqmw\" (UID: \"cf8428ad-0fb7-4108-a41f-8fd01ec35c5c\") " pod="openshift-ingress-canary/ingress-canary-rdqmw" Jul 15 19:22:02.149460 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:02.149441 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cf8428ad-0fb7-4108-a41f-8fd01ec35c5c-cert\") pod \"ingress-canary-rdqmw\" (UID: \"cf8428ad-0fb7-4108-a41f-8fd01ec35c5c\") " pod="openshift-ingress-canary/ingress-canary-rdqmw" Jul 15 19:22:02.155656 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:02.155636 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvcmw\" (UniqueName: \"kubernetes.io/projected/cf8428ad-0fb7-4108-a41f-8fd01ec35c5c-kube-api-access-wvcmw\") pod \"ingress-canary-rdqmw\" (UID: \"cf8428ad-0fb7-4108-a41f-8fd01ec35c5c\") " pod="openshift-ingress-canary/ingress-canary-rdqmw" Jul 15 19:22:02.181541 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:02.181517 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rdqmw" Jul 15 19:22:02.549713 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:02.549671 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/56c7bcac-8a82-4c32-8658-1db0bad8d228-metrics-tls\") pod \"dns-default-5zrgr\" (UID: \"56c7bcac-8a82-4c32-8658-1db0bad8d228\") " pod="openshift-dns/dns-default-5zrgr" Jul 15 19:22:02.549909 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:02.549834 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-registry-tls\") pod \"image-registry-9db745d49-xxf8k\" (UID: \"1235737d-3c8a-4c3a-af1a-dfd3351d72fb\") " pod="openshift-image-registry/image-registry-9db745d49-xxf8k" Jul 15 19:22:02.549909 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:02.549878 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/152d1d6b-5ee1-40c2-80a8-388091a57c58-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-w4b96\" (UID: \"152d1d6b-5ee1-40c2-80a8-388091a57c58\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-w4b96" Jul 15 19:22:02.550011 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:02.549840 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 19:22:02.550011 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:02.549982 2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 15 19:22:02.550011 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:02.549986 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 19:22:02.550011 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:02.550002 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9db745d49-xxf8k: secret "image-registry-tls" not found Jul 15 19:22:02.550235 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:02.549991 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56c7bcac-8a82-4c32-8658-1db0bad8d228-metrics-tls podName:56c7bcac-8a82-4c32-8658-1db0bad8d228 nodeName:}" failed. No retries permitted until 2026-07-15 19:22:03.549976015 +0000 UTC m=+37.126977665 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/56c7bcac-8a82-4c32-8658-1db0bad8d228-metrics-tls") pod "dns-default-5zrgr" (UID: "56c7bcac-8a82-4c32-8658-1db0bad8d228") : secret "dns-default-metrics-tls" not found Jul 15 19:22:02.550235 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:02.550077 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/152d1d6b-5ee1-40c2-80a8-388091a57c58-networking-console-plugin-cert podName:152d1d6b-5ee1-40c2-80a8-388091a57c58 nodeName:}" failed. No retries permitted until 2026-07-15 19:22:03.550053827 +0000 UTC m=+37.127055491 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/152d1d6b-5ee1-40c2-80a8-388091a57c58-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-w4b96" (UID: "152d1d6b-5ee1-40c2-80a8-388091a57c58") : secret "networking-console-plugin-cert" not found Jul 15 19:22:02.550235 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:02.550097 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-registry-tls podName:1235737d-3c8a-4c3a-af1a-dfd3351d72fb nodeName:}" failed. No retries permitted until 2026-07-15 19:22:03.550085841 +0000 UTC m=+37.127087502 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-registry-tls") pod "image-registry-9db745d49-xxf8k" (UID: "1235737d-3c8a-4c3a-af1a-dfd3351d72fb") : secret "image-registry-tls" not found Jul 15 19:22:02.830687 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:02.830660 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rdqmw"] Jul 15 19:22:02.892417 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:22:02.892389 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf8428ad_0fb7_4108_a41f_8fd01ec35c5c.slice/crio-1c2f3a66129fd27cf8e92af86ed86132b3b15bc9a0ca33a98796a1354b28dfd7 WatchSource:0}: Error finding container 1c2f3a66129fd27cf8e92af86ed86132b3b15bc9a0ca33a98796a1354b28dfd7: Status 404 returned error can't find the container with id 1c2f3a66129fd27cf8e92af86ed86132b3b15bc9a0ca33a98796a1354b28dfd7 Jul 15 19:22:03.256096 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:03.256062 2577 generic.go:356] "Generic (PLEG): container finished" podID="23786db4-0338-4d64-9ea3-be6ebc44d74e" containerID="299f37acc9b5997a248320889be691c1ecff1e60674515143a41156952da8123" exitCode=0 Jul 15 19:22:03.256230 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:03.256143 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f7pg5" event={"ID":"23786db4-0338-4d64-9ea3-be6ebc44d74e","Type":"ContainerDied","Data":"299f37acc9b5997a248320889be691c1ecff1e60674515143a41156952da8123"} Jul 15 19:22:03.257296 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:03.257271 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rdqmw" event={"ID":"cf8428ad-0fb7-4108-a41f-8fd01ec35c5c","Type":"ContainerStarted","Data":"1c2f3a66129fd27cf8e92af86ed86132b3b15bc9a0ca33a98796a1354b28dfd7"} Jul 15 19:22:03.558008 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:03.557808 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/152d1d6b-5ee1-40c2-80a8-388091a57c58-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-w4b96\" (UID: \"152d1d6b-5ee1-40c2-80a8-388091a57c58\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-w4b96" Jul 15 19:22:03.558180 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:03.557941 2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 15 19:22:03.558180 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:03.558083 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/56c7bcac-8a82-4c32-8658-1db0bad8d228-metrics-tls\") pod \"dns-default-5zrgr\" (UID: \"56c7bcac-8a82-4c32-8658-1db0bad8d228\") " pod="openshift-dns/dns-default-5zrgr" Jul 15 19:22:03.558180 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:03.558141 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/152d1d6b-5ee1-40c2-80a8-388091a57c58-networking-console-plugin-cert podName:152d1d6b-5ee1-40c2-80a8-388091a57c58 nodeName:}" failed. No retries permitted until 2026-07-15 19:22:05.55812154 +0000 UTC m=+39.135123189 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/152d1d6b-5ee1-40c2-80a8-388091a57c58-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-w4b96" (UID: "152d1d6b-5ee1-40c2-80a8-388091a57c58") : secret "networking-console-plugin-cert" not found Jul 15 19:22:03.558180 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:03.558161 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-registry-tls\") pod \"image-registry-9db745d49-xxf8k\" (UID: \"1235737d-3c8a-4c3a-af1a-dfd3351d72fb\") " pod="openshift-image-registry/image-registry-9db745d49-xxf8k" Jul 15 19:22:03.558379 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:03.558190 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 19:22:03.558379 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:03.558244 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56c7bcac-8a82-4c32-8658-1db0bad8d228-metrics-tls podName:56c7bcac-8a82-4c32-8658-1db0bad8d228 nodeName:}" failed. No retries permitted until 2026-07-15 19:22:05.558228662 +0000 UTC m=+39.135230329 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/56c7bcac-8a82-4c32-8658-1db0bad8d228-metrics-tls") pod "dns-default-5zrgr" (UID: "56c7bcac-8a82-4c32-8658-1db0bad8d228") : secret "dns-default-metrics-tls" not found Jul 15 19:22:03.558379 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:03.558258 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 19:22:03.558379 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:03.558274 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9db745d49-xxf8k: secret "image-registry-tls" not found Jul 15 19:22:03.558379 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:03.558339 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-registry-tls podName:1235737d-3c8a-4c3a-af1a-dfd3351d72fb nodeName:}" failed. No retries permitted until 2026-07-15 19:22:05.55830765 +0000 UTC m=+39.135309304 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-registry-tls") pod "image-registry-9db745d49-xxf8k" (UID: "1235737d-3c8a-4c3a-af1a-dfd3351d72fb") : secret "image-registry-tls" not found Jul 15 19:22:04.263988 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:04.263943 2577 generic.go:356] "Generic (PLEG): container finished" podID="23786db4-0338-4d64-9ea3-be6ebc44d74e" containerID="4069a1397304cb7f19066f676e893b2f5ccddb04b13e983d2260e07635e0d627" exitCode=0 Jul 15 19:22:04.264445 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:04.264006 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f7pg5" event={"ID":"23786db4-0338-4d64-9ea3-be6ebc44d74e","Type":"ContainerDied","Data":"4069a1397304cb7f19066f676e893b2f5ccddb04b13e983d2260e07635e0d627"} Jul 15 19:22:05.270480 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:05.270442 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f7pg5" event={"ID":"23786db4-0338-4d64-9ea3-be6ebc44d74e","Type":"ContainerStarted","Data":"c0452ec78b8cdf6562e918a191d852a84d13f04baf13bafcd1ab866bdde68da3"} Jul 15 19:22:05.293567 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:05.293517 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-f7pg5" podStartSLOduration=7.48308082 podStartE2EDuration="38.293501873s" totalImagesPullingTime="30.810421053s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:21:27 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:22:05.292344103 +0000 UTC m=+38.869345772" watchObservedRunningTime="2026-07-15 19:22:05.293501873 +0000 UTC m=+38.870503545" Jul 15 19:22:05.575385 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:05.575346 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/56c7bcac-8a82-4c32-8658-1db0bad8d228-metrics-tls\") pod \"dns-default-5zrgr\" (UID: \"56c7bcac-8a82-4c32-8658-1db0bad8d228\") " pod="openshift-dns/dns-default-5zrgr" Jul 15 19:22:05.575572 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:05.575416 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-registry-tls\") pod \"image-registry-9db745d49-xxf8k\" (UID: \"1235737d-3c8a-4c3a-af1a-dfd3351d72fb\") " pod="openshift-image-registry/image-registry-9db745d49-xxf8k" Jul 15 19:22:05.575572 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:05.575459 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/152d1d6b-5ee1-40c2-80a8-388091a57c58-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-w4b96\" (UID: \"152d1d6b-5ee1-40c2-80a8-388091a57c58\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-w4b96" Jul 15 19:22:05.575572 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:05.575520 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 19:22:05.575572 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:05.575529 2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 15 19:22:05.575572 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:05.575539 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9db745d49-xxf8k: secret "image-registry-tls" not found Jul 15 19:22:05.575791 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:05.575519 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 19:22:05.575791 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:05.575595 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/152d1d6b-5ee1-40c2-80a8-388091a57c58-networking-console-plugin-cert podName:152d1d6b-5ee1-40c2-80a8-388091a57c58 nodeName:}" failed. No retries permitted until 2026-07-15 19:22:09.575580063 +0000 UTC m=+43.152581716 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/152d1d6b-5ee1-40c2-80a8-388091a57c58-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-w4b96" (UID: "152d1d6b-5ee1-40c2-80a8-388091a57c58") : secret "networking-console-plugin-cert" not found Jul 15 19:22:05.575791 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:05.575621 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56c7bcac-8a82-4c32-8658-1db0bad8d228-metrics-tls podName:56c7bcac-8a82-4c32-8658-1db0bad8d228 nodeName:}" failed. No retries permitted until 2026-07-15 19:22:09.575603539 +0000 UTC m=+43.152605208 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/56c7bcac-8a82-4c32-8658-1db0bad8d228-metrics-tls") pod "dns-default-5zrgr" (UID: "56c7bcac-8a82-4c32-8658-1db0bad8d228") : secret "dns-default-metrics-tls" not found Jul 15 19:22:05.575791 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:05.575640 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-registry-tls podName:1235737d-3c8a-4c3a-af1a-dfd3351d72fb nodeName:}" failed. No retries permitted until 2026-07-15 19:22:09.575630969 +0000 UTC m=+43.152632622 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-registry-tls") pod "image-registry-9db745d49-xxf8k" (UID: "1235737d-3c8a-4c3a-af1a-dfd3351d72fb") : secret "image-registry-tls" not found Jul 15 19:22:06.274739 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:06.274664 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rdqmw" event={"ID":"cf8428ad-0fb7-4108-a41f-8fd01ec35c5c","Type":"ContainerStarted","Data":"d06c45c88c59d223a1c31319f9a1aea605d7a5f6a367e37044e09889a550ea37"} Jul 15 19:22:06.291106 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:06.291065 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-rdqmw" podStartSLOduration=2.31424293 podStartE2EDuration="5.291050303s" totalImagesPullingTime="2.976807373s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:22:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:22:06.289926254 +0000 UTC m=+39.866927936" watchObservedRunningTime="2026-07-15 19:22:06.291050303 +0000 UTC m=+39.868051973" Jul 15 19:22:09.603690 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:09.603649 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/56c7bcac-8a82-4c32-8658-1db0bad8d228-metrics-tls\") pod \"dns-default-5zrgr\" (UID: \"56c7bcac-8a82-4c32-8658-1db0bad8d228\") " pod="openshift-dns/dns-default-5zrgr" Jul 15 19:22:09.604118 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:09.603700 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-registry-tls\") pod \"image-registry-9db745d49-xxf8k\" (UID: \"1235737d-3c8a-4c3a-af1a-dfd3351d72fb\") " pod="openshift-image-registry/image-registry-9db745d49-xxf8k" Jul 15 19:22:09.604118 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:09.603722 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/152d1d6b-5ee1-40c2-80a8-388091a57c58-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-w4b96\" (UID: \"152d1d6b-5ee1-40c2-80a8-388091a57c58\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-w4b96" Jul 15 19:22:09.604118 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:09.603803 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 19:22:09.604118 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:09.603825 2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 15 19:22:09.604118 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:09.603825 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 19:22:09.604118 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:09.603850 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9db745d49-xxf8k: secret "image-registry-tls" not found Jul 15 19:22:09.604118 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:09.603868 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56c7bcac-8a82-4c32-8658-1db0bad8d228-metrics-tls podName:56c7bcac-8a82-4c32-8658-1db0bad8d228 nodeName:}" failed. No retries permitted until 2026-07-15 19:22:17.603853045 +0000 UTC m=+51.180854695 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/56c7bcac-8a82-4c32-8658-1db0bad8d228-metrics-tls") pod "dns-default-5zrgr" (UID: "56c7bcac-8a82-4c32-8658-1db0bad8d228") : secret "dns-default-metrics-tls" not found Jul 15 19:22:09.604118 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:09.603883 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/152d1d6b-5ee1-40c2-80a8-388091a57c58-networking-console-plugin-cert podName:152d1d6b-5ee1-40c2-80a8-388091a57c58 nodeName:}" failed. No retries permitted until 2026-07-15 19:22:17.603876599 +0000 UTC m=+51.180878248 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/152d1d6b-5ee1-40c2-80a8-388091a57c58-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-w4b96" (UID: "152d1d6b-5ee1-40c2-80a8-388091a57c58") : secret "networking-console-plugin-cert" not found Jul 15 19:22:09.604118 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:09.603893 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-registry-tls podName:1235737d-3c8a-4c3a-af1a-dfd3351d72fb nodeName:}" failed. No retries permitted until 2026-07-15 19:22:17.603887995 +0000 UTC m=+51.180889643 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-registry-tls") pod "image-registry-9db745d49-xxf8k" (UID: "1235737d-3c8a-4c3a-af1a-dfd3351d72fb") : secret "image-registry-tls" not found Jul 15 19:22:17.654906 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:17.654860 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/56c7bcac-8a82-4c32-8658-1db0bad8d228-metrics-tls\") pod \"dns-default-5zrgr\" (UID: \"56c7bcac-8a82-4c32-8658-1db0bad8d228\") " pod="openshift-dns/dns-default-5zrgr" Jul 15 19:22:17.654906 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:17.654914 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-registry-tls\") pod \"image-registry-9db745d49-xxf8k\" (UID: \"1235737d-3c8a-4c3a-af1a-dfd3351d72fb\") " pod="openshift-image-registry/image-registry-9db745d49-xxf8k" Jul 15 19:22:17.655427 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:17.654939 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/152d1d6b-5ee1-40c2-80a8-388091a57c58-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-w4b96\" (UID: \"152d1d6b-5ee1-40c2-80a8-388091a57c58\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-w4b96" Jul 15 19:22:17.655427 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:17.655019 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 19:22:17.655427 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:17.655042 2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 15 19:22:17.655427 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:17.655057 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 19:22:17.655427 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:17.655073 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9db745d49-xxf8k: secret "image-registry-tls" not found Jul 15 19:22:17.655427 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:17.655087 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56c7bcac-8a82-4c32-8658-1db0bad8d228-metrics-tls podName:56c7bcac-8a82-4c32-8658-1db0bad8d228 nodeName:}" failed. No retries permitted until 2026-07-15 19:22:33.655068498 +0000 UTC m=+67.232070166 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/56c7bcac-8a82-4c32-8658-1db0bad8d228-metrics-tls") pod "dns-default-5zrgr" (UID: "56c7bcac-8a82-4c32-8658-1db0bad8d228") : secret "dns-default-metrics-tls" not found Jul 15 19:22:17.655427 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:17.655108 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-registry-tls podName:1235737d-3c8a-4c3a-af1a-dfd3351d72fb nodeName:}" failed. No retries permitted until 2026-07-15 19:22:33.655096494 +0000 UTC m=+67.232098148 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-registry-tls") pod "image-registry-9db745d49-xxf8k" (UID: "1235737d-3c8a-4c3a-af1a-dfd3351d72fb") : secret "image-registry-tls" not found Jul 15 19:22:17.655427 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:17.655129 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/152d1d6b-5ee1-40c2-80a8-388091a57c58-networking-console-plugin-cert podName:152d1d6b-5ee1-40c2-80a8-388091a57c58 nodeName:}" failed. No retries permitted until 2026-07-15 19:22:33.655122705 +0000 UTC m=+67.232124354 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/152d1d6b-5ee1-40c2-80a8-388091a57c58-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-w4b96" (UID: "152d1d6b-5ee1-40c2-80a8-388091a57c58") : secret "networking-console-plugin-cert" not found Jul 15 19:22:26.245357 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:26.245333 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fw6vl" Jul 15 19:22:32.761137 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:32.761083 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/237b9375-100a-4d62-83f0-cb271ab8e0bb-metrics-certs\") pod \"network-metrics-daemon-cmqc9\" (UID: \"237b9375-100a-4d62-83f0-cb271ab8e0bb\") " pod="openshift-multus/network-metrics-daemon-cmqc9" Jul 15 19:22:32.761666 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:32.761184 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/05d64bcb-1ac1-47aa-a557-63ba6c2011ab-original-pull-secret\") pod \"global-pull-secret-syncer-pn46g\" (UID: \"05d64bcb-1ac1-47aa-a557-63ba6c2011ab\") " pod="kube-system/global-pull-secret-syncer-pn46g" Jul 15 19:22:32.763860 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:32.763841 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 15 19:22:32.763983 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:32.763968 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 15 19:22:32.771518 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:32.771504 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 15 19:22:32.771569 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:32.771559 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/237b9375-100a-4d62-83f0-cb271ab8e0bb-metrics-certs podName:237b9375-100a-4d62-83f0-cb271ab8e0bb nodeName:}" failed. No retries permitted until 2026-07-15 19:23:36.771542989 +0000 UTC m=+130.348544638 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/237b9375-100a-4d62-83f0-cb271ab8e0bb-metrics-certs") pod "network-metrics-daemon-cmqc9" (UID: "237b9375-100a-4d62-83f0-cb271ab8e0bb") : secret "metrics-daemon-secret" not found Jul 15 19:22:32.774703 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:32.774685 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/05d64bcb-1ac1-47aa-a557-63ba6c2011ab-original-pull-secret\") pod \"global-pull-secret-syncer-pn46g\" (UID: \"05d64bcb-1ac1-47aa-a557-63ba6c2011ab\") " pod="kube-system/global-pull-secret-syncer-pn46g" Jul 15 19:22:32.861810 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:32.861786 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8f8j\" (UniqueName: \"kubernetes.io/projected/c90bd526-03fe-4c0c-9d9d-5bd777b6bd87-kube-api-access-z8f8j\") pod \"network-check-target-5zk6t\" (UID: \"c90bd526-03fe-4c0c-9d9d-5bd777b6bd87\") " pod="openshift-network-diagnostics/network-check-target-5zk6t" Jul 15 19:22:32.864477 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:32.864458 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 15 19:22:32.874071 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:32.874053 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 15 19:22:32.884692 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:32.884673 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8f8j\" (UniqueName: \"kubernetes.io/projected/c90bd526-03fe-4c0c-9d9d-5bd777b6bd87-kube-api-access-z8f8j\") pod \"network-check-target-5zk6t\" (UID: \"c90bd526-03fe-4c0c-9d9d-5bd777b6bd87\") " pod="openshift-network-diagnostics/network-check-target-5zk6t" Jul 15 19:22:33.000840 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:33.000818 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jwgm9\"" Jul 15 19:22:33.008778 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:33.008758 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zk6t" Jul 15 19:22:33.011055 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:33.011039 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pn46g" Jul 15 19:22:33.152015 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:33.151985 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5zk6t"] Jul 15 19:22:33.156349 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:22:33.156321 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc90bd526_03fe_4c0c_9d9d_5bd777b6bd87.slice/crio-0cd198c130cb7405527db929110f4d4cf677b2ed57533c8967bb0fd322a56dc8 WatchSource:0}: Error finding container 0cd198c130cb7405527db929110f4d4cf677b2ed57533c8967bb0fd322a56dc8: Status 404 returned error can't find the container with id 0cd198c130cb7405527db929110f4d4cf677b2ed57533c8967bb0fd322a56dc8 Jul 15 19:22:33.168747 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:33.168723 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-pn46g"] Jul 15 19:22:33.171482 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:22:33.171458 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05d64bcb_1ac1_47aa_a557_63ba6c2011ab.slice/crio-6a865a50351b4e2631bd34e287d5072a4bebb339a9ff5a7a214907aebd96b3e2 WatchSource:0}: Error finding container 6a865a50351b4e2631bd34e287d5072a4bebb339a9ff5a7a214907aebd96b3e2: Status 404 returned error can't find the container with id 6a865a50351b4e2631bd34e287d5072a4bebb339a9ff5a7a214907aebd96b3e2 Jul 15 19:22:33.333418 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:33.333387 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-pn46g" event={"ID":"05d64bcb-1ac1-47aa-a557-63ba6c2011ab","Type":"ContainerStarted","Data":"6a865a50351b4e2631bd34e287d5072a4bebb339a9ff5a7a214907aebd96b3e2"} Jul 15 19:22:33.334463 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:33.334440 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5zk6t" event={"ID":"c90bd526-03fe-4c0c-9d9d-5bd777b6bd87","Type":"ContainerStarted","Data":"0cd198c130cb7405527db929110f4d4cf677b2ed57533c8967bb0fd322a56dc8"} Jul 15 19:22:33.667026 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:33.666940 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/56c7bcac-8a82-4c32-8658-1db0bad8d228-metrics-tls\") pod \"dns-default-5zrgr\" (UID: \"56c7bcac-8a82-4c32-8658-1db0bad8d228\") " pod="openshift-dns/dns-default-5zrgr" Jul 15 19:22:33.667026 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:33.666983 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-registry-tls\") pod \"image-registry-9db745d49-xxf8k\" (UID: \"1235737d-3c8a-4c3a-af1a-dfd3351d72fb\") " pod="openshift-image-registry/image-registry-9db745d49-xxf8k" Jul 15 19:22:33.667026 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:33.667013 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/152d1d6b-5ee1-40c2-80a8-388091a57c58-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-w4b96\" (UID: \"152d1d6b-5ee1-40c2-80a8-388091a57c58\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-w4b96" Jul 15 19:22:33.667305 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:33.667137 2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 15 19:22:33.667305 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:33.667190 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 19:22:33.667305 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:33.667135 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 19:22:33.667305 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:33.667235 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9db745d49-xxf8k: secret "image-registry-tls" not found Jul 15 19:22:33.667305 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:33.667195 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/152d1d6b-5ee1-40c2-80a8-388091a57c58-networking-console-plugin-cert podName:152d1d6b-5ee1-40c2-80a8-388091a57c58 nodeName:}" failed. No retries permitted until 2026-07-15 19:23:05.667181392 +0000 UTC m=+99.244183044 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/152d1d6b-5ee1-40c2-80a8-388091a57c58-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-w4b96" (UID: "152d1d6b-5ee1-40c2-80a8-388091a57c58") : secret "networking-console-plugin-cert" not found Jul 15 19:22:33.667305 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:33.667264 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56c7bcac-8a82-4c32-8658-1db0bad8d228-metrics-tls podName:56c7bcac-8a82-4c32-8658-1db0bad8d228 nodeName:}" failed. No retries permitted until 2026-07-15 19:23:05.6672466 +0000 UTC m=+99.244248255 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/56c7bcac-8a82-4c32-8658-1db0bad8d228-metrics-tls") pod "dns-default-5zrgr" (UID: "56c7bcac-8a82-4c32-8658-1db0bad8d228") : secret "dns-default-metrics-tls" not found Jul 15 19:22:33.667305 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:22:33.667281 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-registry-tls podName:1235737d-3c8a-4c3a-af1a-dfd3351d72fb nodeName:}" failed. No retries permitted until 2026-07-15 19:23:05.667271479 +0000 UTC m=+99.244273152 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-registry-tls") pod "image-registry-9db745d49-xxf8k" (UID: "1235737d-3c8a-4c3a-af1a-dfd3351d72fb") : secret "image-registry-tls" not found Jul 15 19:22:38.347115 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:38.347087 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5zk6t" event={"ID":"c90bd526-03fe-4c0c-9d9d-5bd777b6bd87","Type":"ContainerStarted","Data":"6703aef213b11284e8615ee5985fd384a5f85009010eca3fa3b9dd6450229a7a"} Jul 15 19:22:38.347467 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:38.347227 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-5zk6t" Jul 15 19:22:38.361485 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:38.361443 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-5zk6t" podStartSLOduration=66.367914277 podStartE2EDuration="1m11.361428787s" totalImagesPullingTime="4.99351451s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:21:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:22:38.360772424 +0000 UTC m=+71.937774108" watchObservedRunningTime="2026-07-15 19:22:38.361428787 +0000 UTC m=+71.938430459" Jul 15 19:22:39.350707 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:39.350666 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-pn46g" event={"ID":"05d64bcb-1ac1-47aa-a557-63ba6c2011ab","Type":"ContainerStarted","Data":"d14106f179560eee1b9a2661eb708e7d355e7b79e37b3aa3eefcaf6cf0fe8820"} Jul 15 19:22:39.367167 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:22:39.367115 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-pn46g" podStartSLOduration=67.139728972 podStartE2EDuration="1m12.367099297s" totalImagesPullingTime="5.227370325s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:21:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:22:39.36675372 +0000 UTC m=+72.943755391" watchObservedRunningTime="2026-07-15 19:22:39.367099297 +0000 UTC m=+72.944100967" Jul 15 19:23:05.687351 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:23:05.687298 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/56c7bcac-8a82-4c32-8658-1db0bad8d228-metrics-tls\") pod \"dns-default-5zrgr\" (UID: \"56c7bcac-8a82-4c32-8658-1db0bad8d228\") " pod="openshift-dns/dns-default-5zrgr" Jul 15 19:23:05.687803 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:23:05.687359 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-registry-tls\") pod \"image-registry-9db745d49-xxf8k\" (UID: \"1235737d-3c8a-4c3a-af1a-dfd3351d72fb\") " pod="openshift-image-registry/image-registry-9db745d49-xxf8k" Jul 15 19:23:05.687803 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:23:05.687383 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/152d1d6b-5ee1-40c2-80a8-388091a57c58-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-w4b96\" (UID: \"152d1d6b-5ee1-40c2-80a8-388091a57c58\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-w4b96" Jul 15 19:23:05.687803 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:23:05.687470 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 19:23:05.687803 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:23:05.687491 2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 15 19:23:05.687803 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:23:05.687503 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 19:23:05.687803 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:23:05.687519 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9db745d49-xxf8k: secret "image-registry-tls" not found Jul 15 19:23:05.687803 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:23:05.687542 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/152d1d6b-5ee1-40c2-80a8-388091a57c58-networking-console-plugin-cert podName:152d1d6b-5ee1-40c2-80a8-388091a57c58 nodeName:}" failed. No retries permitted until 2026-07-15 19:24:09.687527362 +0000 UTC m=+163.264529011 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/152d1d6b-5ee1-40c2-80a8-388091a57c58-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-w4b96" (UID: "152d1d6b-5ee1-40c2-80a8-388091a57c58") : secret "networking-console-plugin-cert" not found Jul 15 19:23:05.687803 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:23:05.687554 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56c7bcac-8a82-4c32-8658-1db0bad8d228-metrics-tls podName:56c7bcac-8a82-4c32-8658-1db0bad8d228 nodeName:}" failed. No retries permitted until 2026-07-15 19:24:09.687548721 +0000 UTC m=+163.264550369 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/56c7bcac-8a82-4c32-8658-1db0bad8d228-metrics-tls") pod "dns-default-5zrgr" (UID: "56c7bcac-8a82-4c32-8658-1db0bad8d228") : secret "dns-default-metrics-tls" not found Jul 15 19:23:05.687803 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:23:05.687565 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-registry-tls podName:1235737d-3c8a-4c3a-af1a-dfd3351d72fb nodeName:}" failed. No retries permitted until 2026-07-15 19:24:09.687558423 +0000 UTC m=+163.264560072 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-registry-tls") pod "image-registry-9db745d49-xxf8k" (UID: "1235737d-3c8a-4c3a-af1a-dfd3351d72fb") : secret "image-registry-tls" not found Jul 15 19:23:09.353660 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:23:09.353633 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-5zk6t" Jul 15 19:23:36.791051 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:23:36.791011 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/237b9375-100a-4d62-83f0-cb271ab8e0bb-metrics-certs\") pod \"network-metrics-daemon-cmqc9\" (UID: \"237b9375-100a-4d62-83f0-cb271ab8e0bb\") " pod="openshift-multus/network-metrics-daemon-cmqc9" Jul 15 19:23:36.791546 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:23:36.791134 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 15 19:23:36.791546 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:23:36.791196 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/237b9375-100a-4d62-83f0-cb271ab8e0bb-metrics-certs podName:237b9375-100a-4d62-83f0-cb271ab8e0bb nodeName:}" failed. No retries permitted until 2026-07-15 19:25:38.791181431 +0000 UTC m=+252.368183080 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/237b9375-100a-4d62-83f0-cb271ab8e0bb-metrics-certs") pod "network-metrics-daemon-cmqc9" (UID: "237b9375-100a-4d62-83f0-cb271ab8e0bb") : secret "metrics-daemon-secret" not found Jul 15 19:24:04.812057 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:24:04.812001 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6cf856758c-w4b96" podUID="152d1d6b-5ee1-40c2-80a8-388091a57c58" Jul 15 19:24:04.831186 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:24:04.831155 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-9db745d49-xxf8k" podUID="1235737d-3c8a-4c3a-af1a-dfd3351d72fb" Jul 15 19:24:04.857441 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:24:04.857404 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-5zrgr" podUID="56c7bcac-8a82-4c32-8658-1db0bad8d228" Jul 15 19:24:05.106292 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:24:05.106205 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-cmqc9" podUID="237b9375-100a-4d62-83f0-cb271ab8e0bb" Jul 15 19:24:05.551589 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:24:05.551510 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-w4b96" Jul 15 19:24:05.551589 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:24:05.551544 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5zrgr" Jul 15 19:24:05.551756 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:24:05.551642 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9db745d49-xxf8k" Jul 15 19:24:09.716978 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:24:09.716942 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/56c7bcac-8a82-4c32-8658-1db0bad8d228-metrics-tls\") pod \"dns-default-5zrgr\" (UID: \"56c7bcac-8a82-4c32-8658-1db0bad8d228\") " pod="openshift-dns/dns-default-5zrgr" Jul 15 19:24:09.716978 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:24:09.716983 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-registry-tls\") pod \"image-registry-9db745d49-xxf8k\" (UID: \"1235737d-3c8a-4c3a-af1a-dfd3351d72fb\") " pod="openshift-image-registry/image-registry-9db745d49-xxf8k" Jul 15 19:24:09.717421 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:24:09.717008 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/152d1d6b-5ee1-40c2-80a8-388091a57c58-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-w4b96\" (UID: \"152d1d6b-5ee1-40c2-80a8-388091a57c58\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-w4b96" Jul 15 19:24:09.717421 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:24:09.717087 2577 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 15 19:24:09.717421 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:24:09.717091 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 19:24:09.717421 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:24:09.717107 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9db745d49-xxf8k: secret "image-registry-tls" not found Jul 15 19:24:09.717421 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:24:09.717133 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/152d1d6b-5ee1-40c2-80a8-388091a57c58-networking-console-plugin-cert podName:152d1d6b-5ee1-40c2-80a8-388091a57c58 nodeName:}" failed. No retries permitted until 2026-07-15 19:26:11.717119924 +0000 UTC m=+285.294121573 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/152d1d6b-5ee1-40c2-80a8-388091a57c58-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-w4b96" (UID: "152d1d6b-5ee1-40c2-80a8-388091a57c58") : secret "networking-console-plugin-cert" not found Jul 15 19:24:09.717421 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:24:09.717088 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 19:24:09.717421 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:24:09.717144 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-registry-tls podName:1235737d-3c8a-4c3a-af1a-dfd3351d72fb nodeName:}" failed. No retries permitted until 2026-07-15 19:26:11.717138751 +0000 UTC m=+285.294140400 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-registry-tls") pod "image-registry-9db745d49-xxf8k" (UID: "1235737d-3c8a-4c3a-af1a-dfd3351d72fb") : secret "image-registry-tls" not found Jul 15 19:24:09.717421 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:24:09.717163 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56c7bcac-8a82-4c32-8658-1db0bad8d228-metrics-tls podName:56c7bcac-8a82-4c32-8658-1db0bad8d228 nodeName:}" failed. No retries permitted until 2026-07-15 19:26:11.717149869 +0000 UTC m=+285.294151518 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/56c7bcac-8a82-4c32-8658-1db0bad8d228-metrics-tls") pod "dns-default-5zrgr" (UID: "56c7bcac-8a82-4c32-8658-1db0bad8d228") : secret "dns-default-metrics-tls" not found Jul 15 19:24:17.088665 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:24:17.088631 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmqc9" Jul 15 19:25:35.199342 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.199284 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-bdmq9"] Jul 15 19:25:35.201669 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.201647 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-v56sm"] Jul 15 19:25:35.201802 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.201757 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-bdmq9" Jul 15 19:25:35.203467 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.203449 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-j2rcv"] Jul 15 19:25:35.203573 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.203551 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-v56sm" Jul 15 19:25:35.204117 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.204104 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-lzkfq\"" Jul 15 19:25:35.205118 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.205098 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-j2rcv" Jul 15 19:25:35.206943 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.206924 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 15 19:25:35.207047 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.206924 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 15 19:25:35.208287 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.208271 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-m489f\"" Jul 15 19:25:35.208493 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.208477 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 15 19:25:35.208791 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.208767 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 15 19:25:35.209046 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.209032 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 15 19:25:35.209046 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.209043 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-6f28t\"" Jul 15 19:25:35.209544 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.209387 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 15 19:25:35.209728 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.209702 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 15 19:25:35.218448 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.217895 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 15 19:25:35.220442 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.220419 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-bdmq9"] Jul 15 19:25:35.235096 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.235077 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-j2rcv"] Jul 15 19:25:35.235867 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.235846 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-v56sm"] Jul 15 19:25:35.276990 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.276967 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a2ded0a5-37e2-48fd-98b6-3d64ae279896-trusted-ca\") pod \"console-operator-7866497494-j2rcv\" (UID: \"a2ded0a5-37e2-48fd-98b6-3d64ae279896\") " pod="openshift-console-operator/console-operator-7866497494-j2rcv" Jul 15 19:25:35.277080 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.276994 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a2ded0a5-37e2-48fd-98b6-3d64ae279896-serving-cert\") pod \"console-operator-7866497494-j2rcv\" (UID: \"a2ded0a5-37e2-48fd-98b6-3d64ae279896\") " pod="openshift-console-operator/console-operator-7866497494-j2rcv" Jul 15 19:25:35.277080 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.277010 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqtv9\" (UniqueName: \"kubernetes.io/projected/a2ded0a5-37e2-48fd-98b6-3d64ae279896-kube-api-access-pqtv9\") pod \"console-operator-7866497494-j2rcv\" (UID: \"a2ded0a5-37e2-48fd-98b6-3d64ae279896\") " pod="openshift-console-operator/console-operator-7866497494-j2rcv" Jul 15 19:25:35.277080 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.277051 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a2ded0a5-37e2-48fd-98b6-3d64ae279896-config\") pod \"console-operator-7866497494-j2rcv\" (UID: \"a2ded0a5-37e2-48fd-98b6-3d64ae279896\") " pod="openshift-console-operator/console-operator-7866497494-j2rcv" Jul 15 19:25:35.277080 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.277072 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfg8j\" (UniqueName: \"kubernetes.io/projected/9c943cb4-95da-40c7-9f58-fb83a855c571-kube-api-access-kfg8j\") pod \"cluster-samples-operator-65db858894-v56sm\" (UID: \"9c943cb4-95da-40c7-9f58-fb83a855c571\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-v56sm" Jul 15 19:25:35.277231 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.277108 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9c943cb4-95da-40c7-9f58-fb83a855c571-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-v56sm\" (UID: \"9c943cb4-95da-40c7-9f58-fb83a855c571\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-v56sm" Jul 15 19:25:35.277231 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.277139 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2l5xg\" (UniqueName: \"kubernetes.io/projected/76198306-69a4-410a-a5a7-2eeb75b7f075-kube-api-access-2l5xg\") pod \"network-check-source-869dcfcdcd-bdmq9\" (UID: \"76198306-69a4-410a-a5a7-2eeb75b7f075\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-bdmq9" Jul 15 19:25:35.298962 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.298943 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-wnl4r"] Jul 15 19:25:35.300774 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.300753 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-wnl4r" Jul 15 19:25:35.302978 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.302964 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-5fpbj\"" Jul 15 19:25:35.303695 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.303679 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 15 19:25:35.304056 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.304037 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 15 19:25:35.338034 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.338008 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-6c44cd97bd-5kbj2"] Jul 15 19:25:35.340014 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.339995 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-wnl4r"] Jul 15 19:25:35.340104 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.340093 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6c44cd97bd-5kbj2" Jul 15 19:25:35.342587 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.342567 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 15 19:25:35.343101 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.343085 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 15 19:25:35.343413 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.343394 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 15 19:25:35.343479 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.343401 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 15 19:25:35.343729 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.343716 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 15 19:25:35.343973 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.343958 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 15 19:25:35.344017 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.344004 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-r9t2k\"" Jul 15 19:25:35.377710 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.377690 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9c943cb4-95da-40c7-9f58-fb83a855c571-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-v56sm\" (UID: \"9c943cb4-95da-40c7-9f58-fb83a855c571\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-v56sm" Jul 15 19:25:35.377842 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.377732 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2l5xg\" (UniqueName: \"kubernetes.io/projected/76198306-69a4-410a-a5a7-2eeb75b7f075-kube-api-access-2l5xg\") pod \"network-check-source-869dcfcdcd-bdmq9\" (UID: \"76198306-69a4-410a-a5a7-2eeb75b7f075\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-bdmq9" Jul 15 19:25:35.377842 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.377758 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a2ded0a5-37e2-48fd-98b6-3d64ae279896-trusted-ca\") pod \"console-operator-7866497494-j2rcv\" (UID: \"a2ded0a5-37e2-48fd-98b6-3d64ae279896\") " pod="openshift-console-operator/console-operator-7866497494-j2rcv" Jul 15 19:25:35.377842 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:25:35.377829 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 19:25:35.378005 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:25:35.377876 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c943cb4-95da-40c7-9f58-fb83a855c571-samples-operator-tls podName:9c943cb4-95da-40c7-9f58-fb83a855c571 nodeName:}" failed. No retries permitted until 2026-07-15 19:25:35.877861746 +0000 UTC m=+249.454863396 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9c943cb4-95da-40c7-9f58-fb83a855c571-samples-operator-tls") pod "cluster-samples-operator-65db858894-v56sm" (UID: "9c943cb4-95da-40c7-9f58-fb83a855c571") : secret "samples-operator-tls" not found Jul 15 19:25:35.378005 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.377947 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a2ded0a5-37e2-48fd-98b6-3d64ae279896-serving-cert\") pod \"console-operator-7866497494-j2rcv\" (UID: \"a2ded0a5-37e2-48fd-98b6-3d64ae279896\") " pod="openshift-console-operator/console-operator-7866497494-j2rcv" Jul 15 19:25:35.378005 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.377983 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqtv9\" (UniqueName: \"kubernetes.io/projected/a2ded0a5-37e2-48fd-98b6-3d64ae279896-kube-api-access-pqtv9\") pod \"console-operator-7866497494-j2rcv\" (UID: \"a2ded0a5-37e2-48fd-98b6-3d64ae279896\") " pod="openshift-console-operator/console-operator-7866497494-j2rcv" Jul 15 19:25:35.378164 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.378037 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5ztk\" (UniqueName: \"kubernetes.io/projected/41ea5a11-d522-4aa7-a948-c9eba82c1bc6-kube-api-access-m5ztk\") pod \"volume-data-source-validator-78495597f6-wnl4r\" (UID: \"41ea5a11-d522-4aa7-a948-c9eba82c1bc6\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-wnl4r" Jul 15 19:25:35.378164 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.378096 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a2ded0a5-37e2-48fd-98b6-3d64ae279896-config\") pod \"console-operator-7866497494-j2rcv\" (UID: \"a2ded0a5-37e2-48fd-98b6-3d64ae279896\") " pod="openshift-console-operator/console-operator-7866497494-j2rcv" Jul 15 19:25:35.378164 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.378120 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfg8j\" (UniqueName: \"kubernetes.io/projected/9c943cb4-95da-40c7-9f58-fb83a855c571-kube-api-access-kfg8j\") pod \"cluster-samples-operator-65db858894-v56sm\" (UID: \"9c943cb4-95da-40c7-9f58-fb83a855c571\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-v56sm" Jul 15 19:25:35.378755 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.378734 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a2ded0a5-37e2-48fd-98b6-3d64ae279896-trusted-ca\") pod \"console-operator-7866497494-j2rcv\" (UID: \"a2ded0a5-37e2-48fd-98b6-3d64ae279896\") " pod="openshift-console-operator/console-operator-7866497494-j2rcv" Jul 15 19:25:35.378801 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.378727 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a2ded0a5-37e2-48fd-98b6-3d64ae279896-config\") pod \"console-operator-7866497494-j2rcv\" (UID: \"a2ded0a5-37e2-48fd-98b6-3d64ae279896\") " pod="openshift-console-operator/console-operator-7866497494-j2rcv" Jul 15 19:25:35.380059 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.380042 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a2ded0a5-37e2-48fd-98b6-3d64ae279896-serving-cert\") pod \"console-operator-7866497494-j2rcv\" (UID: \"a2ded0a5-37e2-48fd-98b6-3d64ae279896\") " pod="openshift-console-operator/console-operator-7866497494-j2rcv" Jul 15 19:25:35.390007 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.389983 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6c44cd97bd-5kbj2"] Jul 15 19:25:35.391224 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.391202 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2l5xg\" (UniqueName: \"kubernetes.io/projected/76198306-69a4-410a-a5a7-2eeb75b7f075-kube-api-access-2l5xg\") pod \"network-check-source-869dcfcdcd-bdmq9\" (UID: \"76198306-69a4-410a-a5a7-2eeb75b7f075\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-bdmq9" Jul 15 19:25:35.391392 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.391373 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqtv9\" (UniqueName: \"kubernetes.io/projected/a2ded0a5-37e2-48fd-98b6-3d64ae279896-kube-api-access-pqtv9\") pod \"console-operator-7866497494-j2rcv\" (UID: \"a2ded0a5-37e2-48fd-98b6-3d64ae279896\") " pod="openshift-console-operator/console-operator-7866497494-j2rcv" Jul 15 19:25:35.397354 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.397328 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfg8j\" (UniqueName: \"kubernetes.io/projected/9c943cb4-95da-40c7-9f58-fb83a855c571-kube-api-access-kfg8j\") pod \"cluster-samples-operator-65db858894-v56sm\" (UID: \"9c943cb4-95da-40c7-9f58-fb83a855c571\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-v56sm" Jul 15 19:25:35.478623 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.478565 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d1aea75b-a17d-49fe-aa61-2969e7e720d1-service-ca-bundle\") pod \"router-default-6c44cd97bd-5kbj2\" (UID: \"d1aea75b-a17d-49fe-aa61-2969e7e720d1\") " pod="openshift-ingress/router-default-6c44cd97bd-5kbj2" Jul 15 19:25:35.478723 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.478625 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcdh6\" (UniqueName: \"kubernetes.io/projected/d1aea75b-a17d-49fe-aa61-2969e7e720d1-kube-api-access-vcdh6\") pod \"router-default-6c44cd97bd-5kbj2\" (UID: \"d1aea75b-a17d-49fe-aa61-2969e7e720d1\") " pod="openshift-ingress/router-default-6c44cd97bd-5kbj2" Jul 15 19:25:35.478723 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.478679 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1aea75b-a17d-49fe-aa61-2969e7e720d1-metrics-certs\") pod \"router-default-6c44cd97bd-5kbj2\" (UID: \"d1aea75b-a17d-49fe-aa61-2969e7e720d1\") " pod="openshift-ingress/router-default-6c44cd97bd-5kbj2" Jul 15 19:25:35.478723 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.478719 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5ztk\" (UniqueName: \"kubernetes.io/projected/41ea5a11-d522-4aa7-a948-c9eba82c1bc6-kube-api-access-m5ztk\") pod \"volume-data-source-validator-78495597f6-wnl4r\" (UID: \"41ea5a11-d522-4aa7-a948-c9eba82c1bc6\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-wnl4r" Jul 15 19:25:35.478828 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.478736 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d1aea75b-a17d-49fe-aa61-2969e7e720d1-stats-auth\") pod \"router-default-6c44cd97bd-5kbj2\" (UID: \"d1aea75b-a17d-49fe-aa61-2969e7e720d1\") " pod="openshift-ingress/router-default-6c44cd97bd-5kbj2" Jul 15 19:25:35.478828 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.478761 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d1aea75b-a17d-49fe-aa61-2969e7e720d1-default-certificate\") pod \"router-default-6c44cd97bd-5kbj2\" (UID: \"d1aea75b-a17d-49fe-aa61-2969e7e720d1\") " pod="openshift-ingress/router-default-6c44cd97bd-5kbj2" Jul 15 19:25:35.491814 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.491788 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5ztk\" (UniqueName: \"kubernetes.io/projected/41ea5a11-d522-4aa7-a948-c9eba82c1bc6-kube-api-access-m5ztk\") pod \"volume-data-source-validator-78495597f6-wnl4r\" (UID: \"41ea5a11-d522-4aa7-a948-c9eba82c1bc6\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-wnl4r" Jul 15 19:25:35.512603 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.512583 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-bdmq9" Jul 15 19:25:35.526379 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.526362 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-j2rcv" Jul 15 19:25:35.579744 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.579706 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d1aea75b-a17d-49fe-aa61-2969e7e720d1-stats-auth\") pod \"router-default-6c44cd97bd-5kbj2\" (UID: \"d1aea75b-a17d-49fe-aa61-2969e7e720d1\") " pod="openshift-ingress/router-default-6c44cd97bd-5kbj2" Jul 15 19:25:35.579857 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.579780 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d1aea75b-a17d-49fe-aa61-2969e7e720d1-default-certificate\") pod \"router-default-6c44cd97bd-5kbj2\" (UID: \"d1aea75b-a17d-49fe-aa61-2969e7e720d1\") " pod="openshift-ingress/router-default-6c44cd97bd-5kbj2" Jul 15 19:25:35.579857 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.579818 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d1aea75b-a17d-49fe-aa61-2969e7e720d1-service-ca-bundle\") pod \"router-default-6c44cd97bd-5kbj2\" (UID: \"d1aea75b-a17d-49fe-aa61-2969e7e720d1\") " pod="openshift-ingress/router-default-6c44cd97bd-5kbj2" Jul 15 19:25:35.579924 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.579884 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcdh6\" (UniqueName: \"kubernetes.io/projected/d1aea75b-a17d-49fe-aa61-2969e7e720d1-kube-api-access-vcdh6\") pod \"router-default-6c44cd97bd-5kbj2\" (UID: \"d1aea75b-a17d-49fe-aa61-2969e7e720d1\") " pod="openshift-ingress/router-default-6c44cd97bd-5kbj2" Jul 15 19:25:35.579924 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.579917 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1aea75b-a17d-49fe-aa61-2969e7e720d1-metrics-certs\") pod \"router-default-6c44cd97bd-5kbj2\" (UID: \"d1aea75b-a17d-49fe-aa61-2969e7e720d1\") " pod="openshift-ingress/router-default-6c44cd97bd-5kbj2" Jul 15 19:25:35.580051 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:25:35.580032 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 15 19:25:35.580119 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:25:35.580110 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1aea75b-a17d-49fe-aa61-2969e7e720d1-metrics-certs podName:d1aea75b-a17d-49fe-aa61-2969e7e720d1 nodeName:}" failed. No retries permitted until 2026-07-15 19:25:36.080090066 +0000 UTC m=+249.657091721 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d1aea75b-a17d-49fe-aa61-2969e7e720d1-metrics-certs") pod "router-default-6c44cd97bd-5kbj2" (UID: "d1aea75b-a17d-49fe-aa61-2969e7e720d1") : secret "router-metrics-certs-default" not found Jul 15 19:25:35.580476 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:25:35.580456 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d1aea75b-a17d-49fe-aa61-2969e7e720d1-service-ca-bundle podName:d1aea75b-a17d-49fe-aa61-2969e7e720d1 nodeName:}" failed. No retries permitted until 2026-07-15 19:25:36.080440528 +0000 UTC m=+249.657442192 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d1aea75b-a17d-49fe-aa61-2969e7e720d1-service-ca-bundle") pod "router-default-6c44cd97bd-5kbj2" (UID: "d1aea75b-a17d-49fe-aa61-2969e7e720d1") : configmap references non-existent config key: service-ca.crt Jul 15 19:25:35.582830 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.582804 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d1aea75b-a17d-49fe-aa61-2969e7e720d1-stats-auth\") pod \"router-default-6c44cd97bd-5kbj2\" (UID: \"d1aea75b-a17d-49fe-aa61-2969e7e720d1\") " pod="openshift-ingress/router-default-6c44cd97bd-5kbj2" Jul 15 19:25:35.582943 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.582882 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d1aea75b-a17d-49fe-aa61-2969e7e720d1-default-certificate\") pod \"router-default-6c44cd97bd-5kbj2\" (UID: \"d1aea75b-a17d-49fe-aa61-2969e7e720d1\") " pod="openshift-ingress/router-default-6c44cd97bd-5kbj2" Jul 15 19:25:35.590714 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.590689 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcdh6\" (UniqueName: \"kubernetes.io/projected/d1aea75b-a17d-49fe-aa61-2969e7e720d1-kube-api-access-vcdh6\") pod \"router-default-6c44cd97bd-5kbj2\" (UID: \"d1aea75b-a17d-49fe-aa61-2969e7e720d1\") " pod="openshift-ingress/router-default-6c44cd97bd-5kbj2" Jul 15 19:25:35.608590 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.608378 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-wnl4r" Jul 15 19:25:35.653158 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.653134 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-bdmq9"] Jul 15 19:25:35.656808 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:25:35.656784 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76198306_69a4_410a_a5a7_2eeb75b7f075.slice/crio-384e6c9a39031817f47a87aa6d7dc4237c6ec041056b1c621518aba7578f1015 WatchSource:0}: Error finding container 384e6c9a39031817f47a87aa6d7dc4237c6ec041056b1c621518aba7578f1015: Status 404 returned error can't find the container with id 384e6c9a39031817f47a87aa6d7dc4237c6ec041056b1c621518aba7578f1015 Jul 15 19:25:35.662655 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.662584 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-j2rcv"] Jul 15 19:25:35.666002 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:25:35.665973 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2ded0a5_37e2_48fd_98b6_3d64ae279896.slice/crio-c74eae35a30b3eaba1a6eb089442aff82266b964a8d45a83d6c354b6656f84ce WatchSource:0}: Error finding container c74eae35a30b3eaba1a6eb089442aff82266b964a8d45a83d6c354b6656f84ce: Status 404 returned error can't find the container with id c74eae35a30b3eaba1a6eb089442aff82266b964a8d45a83d6c354b6656f84ce Jul 15 19:25:35.742805 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.742744 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-wnl4r"] Jul 15 19:25:35.745548 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:25:35.745522 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41ea5a11_d522_4aa7_a948_c9eba82c1bc6.slice/crio-d3b11324ec8f9e8616228add41c36bd419dca53d77c7371396ff5abc25131f32 WatchSource:0}: Error finding container d3b11324ec8f9e8616228add41c36bd419dca53d77c7371396ff5abc25131f32: Status 404 returned error can't find the container with id d3b11324ec8f9e8616228add41c36bd419dca53d77c7371396ff5abc25131f32 Jul 15 19:25:35.753890 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.753863 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-j2rcv" event={"ID":"a2ded0a5-37e2-48fd-98b6-3d64ae279896","Type":"ContainerStarted","Data":"c74eae35a30b3eaba1a6eb089442aff82266b964a8d45a83d6c354b6656f84ce"} Jul 15 19:25:35.754879 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.754859 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-wnl4r" event={"ID":"41ea5a11-d522-4aa7-a948-c9eba82c1bc6","Type":"ContainerStarted","Data":"d3b11324ec8f9e8616228add41c36bd419dca53d77c7371396ff5abc25131f32"} Jul 15 19:25:35.756112 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.756091 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-bdmq9" event={"ID":"76198306-69a4-410a-a5a7-2eeb75b7f075","Type":"ContainerStarted","Data":"d9e653d5bb8aca2d9d389af98e093b10ef1a31c0137c37518b61f8a9d464426b"} Jul 15 19:25:35.756186 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.756129 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-bdmq9" event={"ID":"76198306-69a4-410a-a5a7-2eeb75b7f075","Type":"ContainerStarted","Data":"384e6c9a39031817f47a87aa6d7dc4237c6ec041056b1c621518aba7578f1015"} Jul 15 19:25:35.774230 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.774187 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-bdmq9" podStartSLOduration=0.774174186 podStartE2EDuration="774.174186ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:25:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:25:35.773424054 +0000 UTC m=+249.350425726" watchObservedRunningTime="2026-07-15 19:25:35.774174186 +0000 UTC m=+249.351175856" Jul 15 19:25:35.882573 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:35.882546 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9c943cb4-95da-40c7-9f58-fb83a855c571-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-v56sm\" (UID: \"9c943cb4-95da-40c7-9f58-fb83a855c571\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-v56sm" Jul 15 19:25:35.882699 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:25:35.882684 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 19:25:35.882753 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:25:35.882746 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c943cb4-95da-40c7-9f58-fb83a855c571-samples-operator-tls podName:9c943cb4-95da-40c7-9f58-fb83a855c571 nodeName:}" failed. No retries permitted until 2026-07-15 19:25:36.88272868 +0000 UTC m=+250.459730332 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9c943cb4-95da-40c7-9f58-fb83a855c571-samples-operator-tls") pod "cluster-samples-operator-65db858894-v56sm" (UID: "9c943cb4-95da-40c7-9f58-fb83a855c571") : secret "samples-operator-tls" not found Jul 15 19:25:36.083989 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:36.083963 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1aea75b-a17d-49fe-aa61-2969e7e720d1-metrics-certs\") pod \"router-default-6c44cd97bd-5kbj2\" (UID: \"d1aea75b-a17d-49fe-aa61-2969e7e720d1\") " pod="openshift-ingress/router-default-6c44cd97bd-5kbj2" Jul 15 19:25:36.084134 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:25:36.084099 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 15 19:25:36.084181 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:25:36.084153 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1aea75b-a17d-49fe-aa61-2969e7e720d1-metrics-certs podName:d1aea75b-a17d-49fe-aa61-2969e7e720d1 nodeName:}" failed. No retries permitted until 2026-07-15 19:25:37.084138924 +0000 UTC m=+250.661140577 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d1aea75b-a17d-49fe-aa61-2969e7e720d1-metrics-certs") pod "router-default-6c44cd97bd-5kbj2" (UID: "d1aea75b-a17d-49fe-aa61-2969e7e720d1") : secret "router-metrics-certs-default" not found Jul 15 19:25:36.084181 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:36.084146 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d1aea75b-a17d-49fe-aa61-2969e7e720d1-service-ca-bundle\") pod \"router-default-6c44cd97bd-5kbj2\" (UID: \"d1aea75b-a17d-49fe-aa61-2969e7e720d1\") " pod="openshift-ingress/router-default-6c44cd97bd-5kbj2" Jul 15 19:25:36.084306 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:25:36.084242 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d1aea75b-a17d-49fe-aa61-2969e7e720d1-service-ca-bundle podName:d1aea75b-a17d-49fe-aa61-2969e7e720d1 nodeName:}" failed. No retries permitted until 2026-07-15 19:25:37.084227307 +0000 UTC m=+250.661228968 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d1aea75b-a17d-49fe-aa61-2969e7e720d1-service-ca-bundle") pod "router-default-6c44cd97bd-5kbj2" (UID: "d1aea75b-a17d-49fe-aa61-2969e7e720d1") : configmap references non-existent config key: service-ca.crt Jul 15 19:25:36.891403 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:36.891370 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9c943cb4-95da-40c7-9f58-fb83a855c571-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-v56sm\" (UID: \"9c943cb4-95da-40c7-9f58-fb83a855c571\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-v56sm" Jul 15 19:25:36.891809 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:25:36.891544 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 19:25:36.891809 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:25:36.891621 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c943cb4-95da-40c7-9f58-fb83a855c571-samples-operator-tls podName:9c943cb4-95da-40c7-9f58-fb83a855c571 nodeName:}" failed. No retries permitted until 2026-07-15 19:25:38.891600735 +0000 UTC m=+252.468602393 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9c943cb4-95da-40c7-9f58-fb83a855c571-samples-operator-tls") pod "cluster-samples-operator-65db858894-v56sm" (UID: "9c943cb4-95da-40c7-9f58-fb83a855c571") : secret "samples-operator-tls" not found Jul 15 19:25:37.092149 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:37.092107 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d1aea75b-a17d-49fe-aa61-2969e7e720d1-service-ca-bundle\") pod \"router-default-6c44cd97bd-5kbj2\" (UID: \"d1aea75b-a17d-49fe-aa61-2969e7e720d1\") " pod="openshift-ingress/router-default-6c44cd97bd-5kbj2" Jul 15 19:25:37.092346 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:37.092274 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1aea75b-a17d-49fe-aa61-2969e7e720d1-metrics-certs\") pod \"router-default-6c44cd97bd-5kbj2\" (UID: \"d1aea75b-a17d-49fe-aa61-2969e7e720d1\") " pod="openshift-ingress/router-default-6c44cd97bd-5kbj2" Jul 15 19:25:37.092346 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:25:37.092304 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d1aea75b-a17d-49fe-aa61-2969e7e720d1-service-ca-bundle podName:d1aea75b-a17d-49fe-aa61-2969e7e720d1 nodeName:}" failed. No retries permitted until 2026-07-15 19:25:39.092279824 +0000 UTC m=+252.669281472 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d1aea75b-a17d-49fe-aa61-2969e7e720d1-service-ca-bundle") pod "router-default-6c44cd97bd-5kbj2" (UID: "d1aea75b-a17d-49fe-aa61-2969e7e720d1") : configmap references non-existent config key: service-ca.crt Jul 15 19:25:37.092477 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:25:37.092381 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 15 19:25:37.092477 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:25:37.092434 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1aea75b-a17d-49fe-aa61-2969e7e720d1-metrics-certs podName:d1aea75b-a17d-49fe-aa61-2969e7e720d1 nodeName:}" failed. No retries permitted until 2026-07-15 19:25:39.092419482 +0000 UTC m=+252.669421132 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d1aea75b-a17d-49fe-aa61-2969e7e720d1-metrics-certs") pod "router-default-6c44cd97bd-5kbj2" (UID: "d1aea75b-a17d-49fe-aa61-2969e7e720d1") : secret "router-metrics-certs-default" not found Jul 15 19:25:37.765000 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:37.764961 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-wnl4r" event={"ID":"41ea5a11-d522-4aa7-a948-c9eba82c1bc6","Type":"ContainerStarted","Data":"c5aca90937d5a2264dc8eb70ffa1388b2cbeacf997cb09c64966f9cd3c144916"} Jul 15 19:25:37.781037 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:37.780991 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-wnl4r" podStartSLOduration=1.636579637 podStartE2EDuration="2.780978562s" totalImagesPullingTime="1.144398925s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:25:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:25:37.78020042 +0000 UTC m=+251.357202110" watchObservedRunningTime="2026-07-15 19:25:37.780978562 +0000 UTC m=+251.357980211" Jul 15 19:25:38.768733 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:38.768700 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-j2rcv_a2ded0a5-37e2-48fd-98b6-3d64ae279896/console-operator/0.log" Jul 15 19:25:38.769182 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:38.768743 2577 generic.go:356] "Generic (PLEG): container finished" podID="a2ded0a5-37e2-48fd-98b6-3d64ae279896" containerID="d373d972f753873ae49bd7835f66b16cadbb99e005d5e4c8ecc1b7a3a1811f7b" exitCode=255 Jul 15 19:25:38.769182 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:38.768838 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-j2rcv" event={"ID":"a2ded0a5-37e2-48fd-98b6-3d64ae279896","Type":"ContainerDied","Data":"d373d972f753873ae49bd7835f66b16cadbb99e005d5e4c8ecc1b7a3a1811f7b"} Jul 15 19:25:38.769182 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:38.768995 2577 scope.go:117] "RemoveContainer" containerID="d373d972f753873ae49bd7835f66b16cadbb99e005d5e4c8ecc1b7a3a1811f7b" Jul 15 19:25:38.807166 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:38.807147 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/237b9375-100a-4d62-83f0-cb271ab8e0bb-metrics-certs\") pod \"network-metrics-daemon-cmqc9\" (UID: \"237b9375-100a-4d62-83f0-cb271ab8e0bb\") " pod="openshift-multus/network-metrics-daemon-cmqc9" Jul 15 19:25:38.807306 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:25:38.807292 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 15 19:25:38.807364 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:25:38.807355 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/237b9375-100a-4d62-83f0-cb271ab8e0bb-metrics-certs podName:237b9375-100a-4d62-83f0-cb271ab8e0bb nodeName:}" failed. No retries permitted until 2026-07-15 19:27:40.807341918 +0000 UTC m=+374.384343567 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/237b9375-100a-4d62-83f0-cb271ab8e0bb-metrics-certs") pod "network-metrics-daemon-cmqc9" (UID: "237b9375-100a-4d62-83f0-cb271ab8e0bb") : secret "metrics-daemon-secret" not found Jul 15 19:25:38.908502 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:38.908472 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9c943cb4-95da-40c7-9f58-fb83a855c571-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-v56sm\" (UID: \"9c943cb4-95da-40c7-9f58-fb83a855c571\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-v56sm" Jul 15 19:25:38.908627 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:25:38.908610 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 19:25:38.908693 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:25:38.908678 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c943cb4-95da-40c7-9f58-fb83a855c571-samples-operator-tls podName:9c943cb4-95da-40c7-9f58-fb83a855c571 nodeName:}" failed. No retries permitted until 2026-07-15 19:25:42.908659734 +0000 UTC m=+256.485661398 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9c943cb4-95da-40c7-9f58-fb83a855c571-samples-operator-tls") pod "cluster-samples-operator-65db858894-v56sm" (UID: "9c943cb4-95da-40c7-9f58-fb83a855c571") : secret "samples-operator-tls" not found Jul 15 19:25:39.109803 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:39.109771 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1aea75b-a17d-49fe-aa61-2969e7e720d1-metrics-certs\") pod \"router-default-6c44cd97bd-5kbj2\" (UID: \"d1aea75b-a17d-49fe-aa61-2969e7e720d1\") " pod="openshift-ingress/router-default-6c44cd97bd-5kbj2" Jul 15 19:25:39.109951 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:39.109847 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d1aea75b-a17d-49fe-aa61-2969e7e720d1-service-ca-bundle\") pod \"router-default-6c44cd97bd-5kbj2\" (UID: \"d1aea75b-a17d-49fe-aa61-2969e7e720d1\") " pod="openshift-ingress/router-default-6c44cd97bd-5kbj2" Jul 15 19:25:39.109951 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:25:39.109896 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 15 19:25:39.109951 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:25:39.109948 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1aea75b-a17d-49fe-aa61-2969e7e720d1-metrics-certs podName:d1aea75b-a17d-49fe-aa61-2969e7e720d1 nodeName:}" failed. No retries permitted until 2026-07-15 19:25:43.109934484 +0000 UTC m=+256.686936132 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d1aea75b-a17d-49fe-aa61-2969e7e720d1-metrics-certs") pod "router-default-6c44cd97bd-5kbj2" (UID: "d1aea75b-a17d-49fe-aa61-2969e7e720d1") : secret "router-metrics-certs-default" not found Jul 15 19:25:39.110080 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:25:39.109971 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d1aea75b-a17d-49fe-aa61-2969e7e720d1-service-ca-bundle podName:d1aea75b-a17d-49fe-aa61-2969e7e720d1 nodeName:}" failed. No retries permitted until 2026-07-15 19:25:43.109955602 +0000 UTC m=+256.686957255 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d1aea75b-a17d-49fe-aa61-2969e7e720d1-service-ca-bundle") pod "router-default-6c44cd97bd-5kbj2" (UID: "d1aea75b-a17d-49fe-aa61-2969e7e720d1") : configmap references non-existent config key: service-ca.crt Jul 15 19:25:39.773262 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:39.773234 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-j2rcv_a2ded0a5-37e2-48fd-98b6-3d64ae279896/console-operator/1.log" Jul 15 19:25:39.773636 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:39.773618 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-j2rcv_a2ded0a5-37e2-48fd-98b6-3d64ae279896/console-operator/0.log" Jul 15 19:25:39.773688 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:39.773652 2577 generic.go:356] "Generic (PLEG): container finished" podID="a2ded0a5-37e2-48fd-98b6-3d64ae279896" containerID="11ee55968589acf9b788c67f8688046caed07451d225f63a2e1acc65c47dec23" exitCode=255 Jul 15 19:25:39.773720 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:39.773698 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-j2rcv" event={"ID":"a2ded0a5-37e2-48fd-98b6-3d64ae279896","Type":"ContainerDied","Data":"11ee55968589acf9b788c67f8688046caed07451d225f63a2e1acc65c47dec23"} Jul 15 19:25:39.773766 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:39.773753 2577 scope.go:117] "RemoveContainer" containerID="d373d972f753873ae49bd7835f66b16cadbb99e005d5e4c8ecc1b7a3a1811f7b" Jul 15 19:25:39.773970 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:39.773944 2577 scope.go:117] "RemoveContainer" containerID="11ee55968589acf9b788c67f8688046caed07451d225f63a2e1acc65c47dec23" Jul 15 19:25:39.774109 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:25:39.774092 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-j2rcv_openshift-console-operator(a2ded0a5-37e2-48fd-98b6-3d64ae279896)\"" pod="openshift-console-operator/console-operator-7866497494-j2rcv" podUID="a2ded0a5-37e2-48fd-98b6-3d64ae279896" Jul 15 19:25:40.530727 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:40.530693 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-fbxck"] Jul 15 19:25:40.534216 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:40.534201 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-fbxck" Jul 15 19:25:40.538792 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:40.538764 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 15 19:25:40.539329 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:40.539295 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-84s8s\"" Jul 15 19:25:40.539439 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:40.539367 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 15 19:25:40.552970 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:40.552949 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-fbxck"] Jul 15 19:25:40.622220 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:40.622194 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wzw5\" (UniqueName: \"kubernetes.io/projected/9bffa91a-d1ee-4a38-95bf-5ac9ea0d9b04-kube-api-access-2wzw5\") pod \"migrator-7f59b54f6c-fbxck\" (UID: \"9bffa91a-d1ee-4a38-95bf-5ac9ea0d9b04\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-fbxck" Jul 15 19:25:40.624375 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:40.624350 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-q6q5q"] Jul 15 19:25:40.626778 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:40.626765 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-q6q5q" Jul 15 19:25:40.629997 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:40.629977 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 15 19:25:40.630146 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:40.630130 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 15 19:25:40.630629 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:40.630615 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 15 19:25:40.630675 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:40.630615 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 15 19:25:40.632890 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:40.632871 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-cld5n\"" Jul 15 19:25:40.655326 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:40.655296 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-q6q5q"] Jul 15 19:25:40.723407 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:40.723377 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wzw5\" (UniqueName: \"kubernetes.io/projected/9bffa91a-d1ee-4a38-95bf-5ac9ea0d9b04-kube-api-access-2wzw5\") pod \"migrator-7f59b54f6c-fbxck\" (UID: \"9bffa91a-d1ee-4a38-95bf-5ac9ea0d9b04\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-fbxck" Jul 15 19:25:40.723555 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:40.723411 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3c70c39c-c761-4bf8-a95c-f260eebd731c-crio-socket\") pod \"insights-runtime-extractor-q6q5q\" (UID: \"3c70c39c-c761-4bf8-a95c-f260eebd731c\") " pod="openshift-insights/insights-runtime-extractor-q6q5q" Jul 15 19:25:40.723555 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:40.723447 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3c70c39c-c761-4bf8-a95c-f260eebd731c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q6q5q\" (UID: \"3c70c39c-c761-4bf8-a95c-f260eebd731c\") " pod="openshift-insights/insights-runtime-extractor-q6q5q" Jul 15 19:25:40.723555 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:40.723477 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3c70c39c-c761-4bf8-a95c-f260eebd731c-data-volume\") pod \"insights-runtime-extractor-q6q5q\" (UID: \"3c70c39c-c761-4bf8-a95c-f260eebd731c\") " pod="openshift-insights/insights-runtime-extractor-q6q5q" Jul 15 19:25:40.723555 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:40.723518 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hm5d9\" (UniqueName: \"kubernetes.io/projected/3c70c39c-c761-4bf8-a95c-f260eebd731c-kube-api-access-hm5d9\") pod \"insights-runtime-extractor-q6q5q\" (UID: \"3c70c39c-c761-4bf8-a95c-f260eebd731c\") " pod="openshift-insights/insights-runtime-extractor-q6q5q" Jul 15 19:25:40.723555 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:40.723537 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3c70c39c-c761-4bf8-a95c-f260eebd731c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q6q5q\" (UID: \"3c70c39c-c761-4bf8-a95c-f260eebd731c\") " pod="openshift-insights/insights-runtime-extractor-q6q5q" Jul 15 19:25:40.734683 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:40.734655 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wzw5\" (UniqueName: \"kubernetes.io/projected/9bffa91a-d1ee-4a38-95bf-5ac9ea0d9b04-kube-api-access-2wzw5\") pod \"migrator-7f59b54f6c-fbxck\" (UID: \"9bffa91a-d1ee-4a38-95bf-5ac9ea0d9b04\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-fbxck" Jul 15 19:25:40.777422 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:40.777400 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-j2rcv_a2ded0a5-37e2-48fd-98b6-3d64ae279896/console-operator/1.log" Jul 15 19:25:40.777753 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:40.777689 2577 scope.go:117] "RemoveContainer" containerID="11ee55968589acf9b788c67f8688046caed07451d225f63a2e1acc65c47dec23" Jul 15 19:25:40.777859 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:25:40.777839 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-j2rcv_openshift-console-operator(a2ded0a5-37e2-48fd-98b6-3d64ae279896)\"" pod="openshift-console-operator/console-operator-7866497494-j2rcv" podUID="a2ded0a5-37e2-48fd-98b6-3d64ae279896" Jul 15 19:25:40.824046 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:40.824023 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3c70c39c-c761-4bf8-a95c-f260eebd731c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q6q5q\" (UID: \"3c70c39c-c761-4bf8-a95c-f260eebd731c\") " pod="openshift-insights/insights-runtime-extractor-q6q5q" Jul 15 19:25:40.824146 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:40.824098 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3c70c39c-c761-4bf8-a95c-f260eebd731c-crio-socket\") pod \"insights-runtime-extractor-q6q5q\" (UID: \"3c70c39c-c761-4bf8-a95c-f260eebd731c\") " pod="openshift-insights/insights-runtime-extractor-q6q5q" Jul 15 19:25:40.824146 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:40.824129 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3c70c39c-c761-4bf8-a95c-f260eebd731c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q6q5q\" (UID: \"3c70c39c-c761-4bf8-a95c-f260eebd731c\") " pod="openshift-insights/insights-runtime-extractor-q6q5q" Jul 15 19:25:40.824224 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:40.824150 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3c70c39c-c761-4bf8-a95c-f260eebd731c-data-volume\") pod \"insights-runtime-extractor-q6q5q\" (UID: \"3c70c39c-c761-4bf8-a95c-f260eebd731c\") " pod="openshift-insights/insights-runtime-extractor-q6q5q" Jul 15 19:25:40.824224 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:25:40.824160 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 19:25:40.824224 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:25:40.824211 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c70c39c-c761-4bf8-a95c-f260eebd731c-insights-runtime-extractor-tls podName:3c70c39c-c761-4bf8-a95c-f260eebd731c nodeName:}" failed. No retries permitted until 2026-07-15 19:25:41.324195571 +0000 UTC m=+254.901197220 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3c70c39c-c761-4bf8-a95c-f260eebd731c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-q6q5q" (UID: "3c70c39c-c761-4bf8-a95c-f260eebd731c") : secret "insights-runtime-extractor-tls" not found Jul 15 19:25:40.824224 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:40.824211 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3c70c39c-c761-4bf8-a95c-f260eebd731c-crio-socket\") pod \"insights-runtime-extractor-q6q5q\" (UID: \"3c70c39c-c761-4bf8-a95c-f260eebd731c\") " pod="openshift-insights/insights-runtime-extractor-q6q5q" Jul 15 19:25:40.824389 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:25:40.824225 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 19:25:40.824389 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:40.824246 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hm5d9\" (UniqueName: \"kubernetes.io/projected/3c70c39c-c761-4bf8-a95c-f260eebd731c-kube-api-access-hm5d9\") pod \"insights-runtime-extractor-q6q5q\" (UID: \"3c70c39c-c761-4bf8-a95c-f260eebd731c\") " pod="openshift-insights/insights-runtime-extractor-q6q5q" Jul 15 19:25:40.824389 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:25:40.824273 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3c70c39c-c761-4bf8-a95c-f260eebd731c-kube-rbac-proxy-cm podName:3c70c39c-c761-4bf8-a95c-f260eebd731c nodeName:}" failed. No retries permitted until 2026-07-15 19:25:41.32425903 +0000 UTC m=+254.901260696 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3c70c39c-c761-4bf8-a95c-f260eebd731c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-q6q5q" (UID: "3c70c39c-c761-4bf8-a95c-f260eebd731c") : configmap "kube-rbac-proxy" not found Jul 15 19:25:40.824497 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:40.824431 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3c70c39c-c761-4bf8-a95c-f260eebd731c-data-volume\") pod \"insights-runtime-extractor-q6q5q\" (UID: \"3c70c39c-c761-4bf8-a95c-f260eebd731c\") " pod="openshift-insights/insights-runtime-extractor-q6q5q" Jul 15 19:25:40.842629 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:40.842611 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-fbxck" Jul 15 19:25:40.845715 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:40.845699 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hm5d9\" (UniqueName: \"kubernetes.io/projected/3c70c39c-c761-4bf8-a95c-f260eebd731c-kube-api-access-hm5d9\") pod \"insights-runtime-extractor-q6q5q\" (UID: \"3c70c39c-c761-4bf8-a95c-f260eebd731c\") " pod="openshift-insights/insights-runtime-extractor-q6q5q" Jul 15 19:25:40.973124 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:40.973097 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-fbxck"] Jul 15 19:25:40.976006 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:25:40.975979 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bffa91a_d1ee_4a38_95bf_5ac9ea0d9b04.slice/crio-b0c5886d583dea4750b790a71d8983401fe8c37f1bb4847fd36c8f841a1edcc2 WatchSource:0}: Error finding container b0c5886d583dea4750b790a71d8983401fe8c37f1bb4847fd36c8f841a1edcc2: Status 404 returned error can't find the container with id b0c5886d583dea4750b790a71d8983401fe8c37f1bb4847fd36c8f841a1edcc2 Jul 15 19:25:41.328815 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:41.328775 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3c70c39c-c761-4bf8-a95c-f260eebd731c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q6q5q\" (UID: \"3c70c39c-c761-4bf8-a95c-f260eebd731c\") " pod="openshift-insights/insights-runtime-extractor-q6q5q" Jul 15 19:25:41.328979 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:41.328834 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3c70c39c-c761-4bf8-a95c-f260eebd731c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q6q5q\" (UID: \"3c70c39c-c761-4bf8-a95c-f260eebd731c\") " pod="openshift-insights/insights-runtime-extractor-q6q5q" Jul 15 19:25:41.328979 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:25:41.328890 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 19:25:41.328979 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:25:41.328941 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 19:25:41.328979 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:25:41.328947 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3c70c39c-c761-4bf8-a95c-f260eebd731c-kube-rbac-proxy-cm podName:3c70c39c-c761-4bf8-a95c-f260eebd731c nodeName:}" failed. No retries permitted until 2026-07-15 19:25:42.328932496 +0000 UTC m=+255.905934150 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3c70c39c-c761-4bf8-a95c-f260eebd731c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-q6q5q" (UID: "3c70c39c-c761-4bf8-a95c-f260eebd731c") : configmap "kube-rbac-proxy" not found Jul 15 19:25:41.328979 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:25:41.328980 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c70c39c-c761-4bf8-a95c-f260eebd731c-insights-runtime-extractor-tls podName:3c70c39c-c761-4bf8-a95c-f260eebd731c nodeName:}" failed. No retries permitted until 2026-07-15 19:25:42.32896821 +0000 UTC m=+255.905969860 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3c70c39c-c761-4bf8-a95c-f260eebd731c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-q6q5q" (UID: "3c70c39c-c761-4bf8-a95c-f260eebd731c") : secret "insights-runtime-extractor-tls" not found Jul 15 19:25:41.781826 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:41.781751 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-fbxck" event={"ID":"9bffa91a-d1ee-4a38-95bf-5ac9ea0d9b04","Type":"ContainerStarted","Data":"b0c5886d583dea4750b790a71d8983401fe8c37f1bb4847fd36c8f841a1edcc2"} Jul 15 19:25:42.336771 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:42.336749 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3c70c39c-c761-4bf8-a95c-f260eebd731c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q6q5q\" (UID: \"3c70c39c-c761-4bf8-a95c-f260eebd731c\") " pod="openshift-insights/insights-runtime-extractor-q6q5q" Jul 15 19:25:42.336847 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:42.336795 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3c70c39c-c761-4bf8-a95c-f260eebd731c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q6q5q\" (UID: \"3c70c39c-c761-4bf8-a95c-f260eebd731c\") " pod="openshift-insights/insights-runtime-extractor-q6q5q" Jul 15 19:25:42.336891 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:25:42.336882 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 19:25:42.336962 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:25:42.336924 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c70c39c-c761-4bf8-a95c-f260eebd731c-insights-runtime-extractor-tls podName:3c70c39c-c761-4bf8-a95c-f260eebd731c nodeName:}" failed. No retries permitted until 2026-07-15 19:25:44.336910934 +0000 UTC m=+257.913912583 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3c70c39c-c761-4bf8-a95c-f260eebd731c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-q6q5q" (UID: "3c70c39c-c761-4bf8-a95c-f260eebd731c") : secret "insights-runtime-extractor-tls" not found Jul 15 19:25:42.336962 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:25:42.336926 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 19:25:42.337046 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:25:42.336977 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3c70c39c-c761-4bf8-a95c-f260eebd731c-kube-rbac-proxy-cm podName:3c70c39c-c761-4bf8-a95c-f260eebd731c nodeName:}" failed. No retries permitted until 2026-07-15 19:25:44.336962251 +0000 UTC m=+257.913963917 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3c70c39c-c761-4bf8-a95c-f260eebd731c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-q6q5q" (UID: "3c70c39c-c761-4bf8-a95c-f260eebd731c") : configmap "kube-rbac-proxy" not found Jul 15 19:25:42.786540 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:42.786505 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-fbxck" event={"ID":"9bffa91a-d1ee-4a38-95bf-5ac9ea0d9b04","Type":"ContainerStarted","Data":"324de8a1f9f210b2fa77a993563e5fd14dcfcec4cde24a55975e59fcaec75d8b"} Jul 15 19:25:42.786910 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:42.786557 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-fbxck" event={"ID":"9bffa91a-d1ee-4a38-95bf-5ac9ea0d9b04","Type":"ContainerStarted","Data":"08748a97533e283e73e68c1218fbe43fd755543139fd3128f61c4104ca63dc89"} Jul 15 19:25:42.809997 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:42.809950 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-fbxck" podStartSLOduration=1.490868981 podStartE2EDuration="2.80993355s" totalImagesPullingTime="1.319064569s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:25:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:25:42.807706961 +0000 UTC m=+256.384708631" watchObservedRunningTime="2026-07-15 19:25:42.80993355 +0000 UTC m=+256.386935246" Jul 15 19:25:42.941136 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:42.941105 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9c943cb4-95da-40c7-9f58-fb83a855c571-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-v56sm\" (UID: \"9c943cb4-95da-40c7-9f58-fb83a855c571\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-v56sm" Jul 15 19:25:42.941277 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:25:42.941206 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 19:25:42.941277 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:25:42.941254 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c943cb4-95da-40c7-9f58-fb83a855c571-samples-operator-tls podName:9c943cb4-95da-40c7-9f58-fb83a855c571 nodeName:}" failed. No retries permitted until 2026-07-15 19:25:50.941241999 +0000 UTC m=+264.518243648 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9c943cb4-95da-40c7-9f58-fb83a855c571-samples-operator-tls") pod "cluster-samples-operator-65db858894-v56sm" (UID: "9c943cb4-95da-40c7-9f58-fb83a855c571") : secret "samples-operator-tls" not found Jul 15 19:25:43.142538 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:43.142508 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1aea75b-a17d-49fe-aa61-2969e7e720d1-metrics-certs\") pod \"router-default-6c44cd97bd-5kbj2\" (UID: \"d1aea75b-a17d-49fe-aa61-2969e7e720d1\") " pod="openshift-ingress/router-default-6c44cd97bd-5kbj2" Jul 15 19:25:43.142686 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:43.142590 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d1aea75b-a17d-49fe-aa61-2969e7e720d1-service-ca-bundle\") pod \"router-default-6c44cd97bd-5kbj2\" (UID: \"d1aea75b-a17d-49fe-aa61-2969e7e720d1\") " pod="openshift-ingress/router-default-6c44cd97bd-5kbj2" Jul 15 19:25:43.142686 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:25:43.142662 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 15 19:25:43.142797 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:25:43.142711 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1aea75b-a17d-49fe-aa61-2969e7e720d1-metrics-certs podName:d1aea75b-a17d-49fe-aa61-2969e7e720d1 nodeName:}" failed. No retries permitted until 2026-07-15 19:25:51.142698194 +0000 UTC m=+264.719699846 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d1aea75b-a17d-49fe-aa61-2969e7e720d1-metrics-certs") pod "router-default-6c44cd97bd-5kbj2" (UID: "d1aea75b-a17d-49fe-aa61-2969e7e720d1") : secret "router-metrics-certs-default" not found Jul 15 19:25:43.142797 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:25:43.142762 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d1aea75b-a17d-49fe-aa61-2969e7e720d1-service-ca-bundle podName:d1aea75b-a17d-49fe-aa61-2969e7e720d1 nodeName:}" failed. No retries permitted until 2026-07-15 19:25:51.14274288 +0000 UTC m=+264.719744545 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d1aea75b-a17d-49fe-aa61-2969e7e720d1-service-ca-bundle") pod "router-default-6c44cd97bd-5kbj2" (UID: "d1aea75b-a17d-49fe-aa61-2969e7e720d1") : configmap references non-existent config key: service-ca.crt Jul 15 19:25:43.217619 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:43.217594 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9vzln_299d8c27-2943-40f5-8810-67bcfcbbb3a2/dns-node-resolver/0.log" Jul 15 19:25:44.220283 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:44.220255 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-c8jtk_b6d81326-425e-4a61-a7db-8790d035bd50/node-ca/0.log" Jul 15 19:25:44.351888 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:44.351857 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3c70c39c-c761-4bf8-a95c-f260eebd731c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q6q5q\" (UID: \"3c70c39c-c761-4bf8-a95c-f260eebd731c\") " pod="openshift-insights/insights-runtime-extractor-q6q5q" Jul 15 19:25:44.352040 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:44.351902 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3c70c39c-c761-4bf8-a95c-f260eebd731c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q6q5q\" (UID: \"3c70c39c-c761-4bf8-a95c-f260eebd731c\") " pod="openshift-insights/insights-runtime-extractor-q6q5q" Jul 15 19:25:44.352040 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:25:44.351971 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 19:25:44.352040 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:25:44.351993 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 19:25:44.352040 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:25:44.352025 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3c70c39c-c761-4bf8-a95c-f260eebd731c-kube-rbac-proxy-cm podName:3c70c39c-c761-4bf8-a95c-f260eebd731c nodeName:}" failed. No retries permitted until 2026-07-15 19:25:48.352009995 +0000 UTC m=+261.929011644 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3c70c39c-c761-4bf8-a95c-f260eebd731c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-q6q5q" (UID: "3c70c39c-c761-4bf8-a95c-f260eebd731c") : configmap "kube-rbac-proxy" not found Jul 15 19:25:44.352040 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:25:44.352038 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c70c39c-c761-4bf8-a95c-f260eebd731c-insights-runtime-extractor-tls podName:3c70c39c-c761-4bf8-a95c-f260eebd731c nodeName:}" failed. No retries permitted until 2026-07-15 19:25:48.352032179 +0000 UTC m=+261.929033828 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3c70c39c-c761-4bf8-a95c-f260eebd731c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-q6q5q" (UID: "3c70c39c-c761-4bf8-a95c-f260eebd731c") : secret "insights-runtime-extractor-tls" not found Jul 15 19:25:45.214263 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:45.214237 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rdqmw_cf8428ad-0fb7-4108-a41f-8fd01ec35c5c/serve-healthcheck-canary/0.log" Jul 15 19:25:45.527219 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:45.527147 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-j2rcv" Jul 15 19:25:45.527219 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:45.527198 2577 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-j2rcv" Jul 15 19:25:45.527586 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:45.527499 2577 scope.go:117] "RemoveContainer" containerID="11ee55968589acf9b788c67f8688046caed07451d225f63a2e1acc65c47dec23" Jul 15 19:25:45.527670 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:25:45.527624 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-j2rcv_openshift-console-operator(a2ded0a5-37e2-48fd-98b6-3d64ae279896)\"" pod="openshift-console-operator/console-operator-7866497494-j2rcv" podUID="a2ded0a5-37e2-48fd-98b6-3d64ae279896" Jul 15 19:25:45.618663 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:45.618643 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-fbxck_9bffa91a-d1ee-4a38-95bf-5ac9ea0d9b04/migrator/0.log" Jul 15 19:25:45.814805 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:45.814788 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-fbxck_9bffa91a-d1ee-4a38-95bf-5ac9ea0d9b04/graceful-termination/0.log" Jul 15 19:25:48.383289 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:48.383202 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3c70c39c-c761-4bf8-a95c-f260eebd731c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q6q5q\" (UID: \"3c70c39c-c761-4bf8-a95c-f260eebd731c\") " pod="openshift-insights/insights-runtime-extractor-q6q5q" Jul 15 19:25:48.383676 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:48.383364 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3c70c39c-c761-4bf8-a95c-f260eebd731c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q6q5q\" (UID: \"3c70c39c-c761-4bf8-a95c-f260eebd731c\") " pod="openshift-insights/insights-runtime-extractor-q6q5q" Jul 15 19:25:48.383676 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:25:48.383367 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 19:25:48.383676 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:25:48.383406 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 19:25:48.383676 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:25:48.383459 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c70c39c-c761-4bf8-a95c-f260eebd731c-insights-runtime-extractor-tls podName:3c70c39c-c761-4bf8-a95c-f260eebd731c nodeName:}" failed. No retries permitted until 2026-07-15 19:25:56.383445381 +0000 UTC m=+269.960447049 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3c70c39c-c761-4bf8-a95c-f260eebd731c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-q6q5q" (UID: "3c70c39c-c761-4bf8-a95c-f260eebd731c") : secret "insights-runtime-extractor-tls" not found Jul 15 19:25:48.383676 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:25:48.383499 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3c70c39c-c761-4bf8-a95c-f260eebd731c-kube-rbac-proxy-cm podName:3c70c39c-c761-4bf8-a95c-f260eebd731c nodeName:}" failed. No retries permitted until 2026-07-15 19:25:56.383478287 +0000 UTC m=+269.960479945 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3c70c39c-c761-4bf8-a95c-f260eebd731c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-q6q5q" (UID: "3c70c39c-c761-4bf8-a95c-f260eebd731c") : configmap "kube-rbac-proxy" not found Jul 15 19:25:51.003651 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:51.003615 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9c943cb4-95da-40c7-9f58-fb83a855c571-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-v56sm\" (UID: \"9c943cb4-95da-40c7-9f58-fb83a855c571\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-v56sm" Jul 15 19:25:51.006213 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:51.006188 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9c943cb4-95da-40c7-9f58-fb83a855c571-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-v56sm\" (UID: \"9c943cb4-95da-40c7-9f58-fb83a855c571\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-v56sm" Jul 15 19:25:51.120771 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:51.120745 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-v56sm" Jul 15 19:25:51.205814 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:51.205783 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1aea75b-a17d-49fe-aa61-2969e7e720d1-metrics-certs\") pod \"router-default-6c44cd97bd-5kbj2\" (UID: \"d1aea75b-a17d-49fe-aa61-2969e7e720d1\") " pod="openshift-ingress/router-default-6c44cd97bd-5kbj2" Jul 15 19:25:51.205957 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:51.205889 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d1aea75b-a17d-49fe-aa61-2969e7e720d1-service-ca-bundle\") pod \"router-default-6c44cd97bd-5kbj2\" (UID: \"d1aea75b-a17d-49fe-aa61-2969e7e720d1\") " pod="openshift-ingress/router-default-6c44cd97bd-5kbj2" Jul 15 19:25:51.206705 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:51.206671 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d1aea75b-a17d-49fe-aa61-2969e7e720d1-service-ca-bundle\") pod \"router-default-6c44cd97bd-5kbj2\" (UID: \"d1aea75b-a17d-49fe-aa61-2969e7e720d1\") " pod="openshift-ingress/router-default-6c44cd97bd-5kbj2" Jul 15 19:25:51.208011 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:51.207980 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d1aea75b-a17d-49fe-aa61-2969e7e720d1-metrics-certs\") pod \"router-default-6c44cd97bd-5kbj2\" (UID: \"d1aea75b-a17d-49fe-aa61-2969e7e720d1\") " pod="openshift-ingress/router-default-6c44cd97bd-5kbj2" Jul 15 19:25:51.246552 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:51.246525 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-v56sm"] Jul 15 19:25:51.248258 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:51.248237 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6c44cd97bd-5kbj2" Jul 15 19:25:51.382591 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:51.382570 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6c44cd97bd-5kbj2"] Jul 15 19:25:51.383243 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:25:51.383222 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1aea75b_a17d_49fe_aa61_2969e7e720d1.slice/crio-e221ea9be7675ddd43300d29ebee0a7d06a21e7844a776e778d17ea1ea750a5f WatchSource:0}: Error finding container e221ea9be7675ddd43300d29ebee0a7d06a21e7844a776e778d17ea1ea750a5f: Status 404 returned error can't find the container with id e221ea9be7675ddd43300d29ebee0a7d06a21e7844a776e778d17ea1ea750a5f Jul 15 19:25:51.812865 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:51.812832 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6c44cd97bd-5kbj2" event={"ID":"d1aea75b-a17d-49fe-aa61-2969e7e720d1","Type":"ContainerStarted","Data":"b6394483ed42ad4a80b92f5faa37bdb56556d83e42626e551fdded4074310d8e"} Jul 15 19:25:51.813037 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:51.812885 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6c44cd97bd-5kbj2" event={"ID":"d1aea75b-a17d-49fe-aa61-2969e7e720d1","Type":"ContainerStarted","Data":"e221ea9be7675ddd43300d29ebee0a7d06a21e7844a776e778d17ea1ea750a5f"} Jul 15 19:25:51.814043 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:51.814018 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-v56sm" event={"ID":"9c943cb4-95da-40c7-9f58-fb83a855c571","Type":"ContainerStarted","Data":"588dd477a6cd8b27c633c5827ddfdc71611fa20c2581a45f7b0bc686e4b096c7"} Jul 15 19:25:51.835746 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:51.835699 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-6c44cd97bd-5kbj2" podStartSLOduration=16.83568317 podStartE2EDuration="16.83568317s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:25:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:25:51.83505876 +0000 UTC m=+265.412060432" watchObservedRunningTime="2026-07-15 19:25:51.83568317 +0000 UTC m=+265.412684841" Jul 15 19:25:52.249442 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:52.249361 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-6c44cd97bd-5kbj2" Jul 15 19:25:52.252097 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:52.252077 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-6c44cd97bd-5kbj2" Jul 15 19:25:52.819241 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:52.819203 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-6c44cd97bd-5kbj2" Jul 15 19:25:52.820933 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:52.820895 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-6c44cd97bd-5kbj2" Jul 15 19:25:53.822880 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:53.822844 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-v56sm" event={"ID":"9c943cb4-95da-40c7-9f58-fb83a855c571","Type":"ContainerStarted","Data":"9beaa940a6d2536b218c6c55ff90eb320d917abe5cb4c8ec50f9c3de4ba22370"} Jul 15 19:25:53.823333 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:53.822924 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-v56sm" event={"ID":"9c943cb4-95da-40c7-9f58-fb83a855c571","Type":"ContainerStarted","Data":"94446f8287d071c3223bb9f1810b2cb4d6306aacb1bffa6b7b3f9baadb30b7be"} Jul 15 19:25:53.841412 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:53.841372 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-v56sm" podStartSLOduration=16.903649488 podStartE2EDuration="18.841358175s" totalImagesPullingTime="1.937708687s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:25:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:25:53.840777789 +0000 UTC m=+267.417779481" watchObservedRunningTime="2026-07-15 19:25:53.841358175 +0000 UTC m=+267.418359845" Jul 15 19:25:56.448019 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:56.447986 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3c70c39c-c761-4bf8-a95c-f260eebd731c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q6q5q\" (UID: \"3c70c39c-c761-4bf8-a95c-f260eebd731c\") " pod="openshift-insights/insights-runtime-extractor-q6q5q" Jul 15 19:25:56.448388 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:56.448082 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3c70c39c-c761-4bf8-a95c-f260eebd731c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q6q5q\" (UID: \"3c70c39c-c761-4bf8-a95c-f260eebd731c\") " pod="openshift-insights/insights-runtime-extractor-q6q5q" Jul 15 19:25:56.448580 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:56.448562 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3c70c39c-c761-4bf8-a95c-f260eebd731c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q6q5q\" (UID: \"3c70c39c-c761-4bf8-a95c-f260eebd731c\") " pod="openshift-insights/insights-runtime-extractor-q6q5q" Jul 15 19:25:56.450045 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:56.450026 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3c70c39c-c761-4bf8-a95c-f260eebd731c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q6q5q\" (UID: \"3c70c39c-c761-4bf8-a95c-f260eebd731c\") " pod="openshift-insights/insights-runtime-extractor-q6q5q" Jul 15 19:25:56.535221 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:56.535188 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-q6q5q" Jul 15 19:25:56.666767 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:56.666739 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-q6q5q"] Jul 15 19:25:56.669874 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:25:56.669852 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c70c39c_c761_4bf8_a95c_f260eebd731c.slice/crio-26a29e349a9a3a64b064fd8e516a5e1a64338d15ac455f66c6e4f8fe2fc835be WatchSource:0}: Error finding container 26a29e349a9a3a64b064fd8e516a5e1a64338d15ac455f66c6e4f8fe2fc835be: Status 404 returned error can't find the container with id 26a29e349a9a3a64b064fd8e516a5e1a64338d15ac455f66c6e4f8fe2fc835be Jul 15 19:25:56.835231 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:25:56.835198 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-q6q5q" event={"ID":"3c70c39c-c761-4bf8-a95c-f260eebd731c","Type":"ContainerStarted","Data":"26a29e349a9a3a64b064fd8e516a5e1a64338d15ac455f66c6e4f8fe2fc835be"} Jul 15 19:26:00.087106 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:00.087073 2577 scope.go:117] "RemoveContainer" containerID="11ee55968589acf9b788c67f8688046caed07451d225f63a2e1acc65c47dec23" Jul 15 19:26:01.859034 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:01.859009 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-j2rcv_a2ded0a5-37e2-48fd-98b6-3d64ae279896/console-operator/1.log" Jul 15 19:26:01.859487 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:01.859095 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-j2rcv" event={"ID":"a2ded0a5-37e2-48fd-98b6-3d64ae279896","Type":"ContainerStarted","Data":"6955ca9b0a229896efd7d9c1a2eeef6ba1e7b6565ac72b224e72da41d9895539"} Jul 15 19:26:01.859487 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:01.859392 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-j2rcv" Jul 15 19:26:01.860602 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:01.860579 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-q6q5q" event={"ID":"3c70c39c-c761-4bf8-a95c-f260eebd731c","Type":"ContainerStarted","Data":"0b912aec288a2c0d4bd443671ec37073ea87b9b2d910d82b66ee07b5b1e07cb5"} Jul 15 19:26:01.877507 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:01.877452 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-j2rcv" podStartSLOduration=24.575288066 podStartE2EDuration="26.877433317s" totalImagesPullingTime="2.302145251s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:25:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:26:01.876642936 +0000 UTC m=+275.453644806" watchObservedRunningTime="2026-07-15 19:26:01.877433317 +0000 UTC m=+275.454434989" Jul 15 19:26:02.526480 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:02.526451 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-j2rcv" Jul 15 19:26:05.876527 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:05.876488 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-q6q5q" event={"ID":"3c70c39c-c761-4bf8-a95c-f260eebd731c","Type":"ContainerStarted","Data":"146c20f2e3d8bc3ad532fe9cba3fdcb3b050ee4e3e032ec25cadd5dbdb52bff7"} Jul 15 19:26:08.552926 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:26:08.552878 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-9db745d49-xxf8k" podUID="1235737d-3c8a-4c3a-af1a-dfd3351d72fb" Jul 15 19:26:08.552926 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:26:08.552878 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6cf856758c-w4b96" podUID="152d1d6b-5ee1-40c2-80a8-388091a57c58" Jul 15 19:26:08.553439 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:26:08.552878 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-5zrgr" podUID="56c7bcac-8a82-4c32-8658-1db0bad8d228" Jul 15 19:26:08.887673 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:08.887646 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-q6q5q" event={"ID":"3c70c39c-c761-4bf8-a95c-f260eebd731c","Type":"ContainerStarted","Data":"f33dc9d5ac5bb57a8af8441a09dba4ecc014e2e129a5051b350e03dc54d8fdb7"} Jul 15 19:26:08.887781 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:08.887701 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5zrgr" Jul 15 19:26:08.887781 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:08.887750 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9db745d49-xxf8k" Jul 15 19:26:08.888023 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:08.888007 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-w4b96" Jul 15 19:26:08.910327 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:08.910283 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-q6q5q" podStartSLOduration=16.875856195 podStartE2EDuration="28.910270482s" totalImagesPullingTime="12.034414287s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:25:40 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:26:08.909669824 +0000 UTC m=+282.486671495" watchObservedRunningTime="2026-07-15 19:26:08.910270482 +0000 UTC m=+282.487272152" Jul 15 19:26:11.526026 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:11.525993 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-zqw6v"] Jul 15 19:26:11.529355 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:11.529332 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-zqw6v" Jul 15 19:26:11.531837 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:11.531817 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-fbtvn\"" Jul 15 19:26:11.531922 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:11.531817 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 15 19:26:11.541657 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:11.541634 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-zqw6v"] Jul 15 19:26:11.569422 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:11.569386 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/49f71022-3768-4912-8953-29b19518fe3a-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-zqw6v\" (UID: \"49f71022-3768-4912-8953-29b19518fe3a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-zqw6v" Jul 15 19:26:11.643098 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:11.643075 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8698c746d9-ck2m6"] Jul 15 19:26:11.646146 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:11.646129 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8698c746d9-ck2m6" Jul 15 19:26:11.648480 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:11.648457 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 15 19:26:11.648689 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:11.648678 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-9xcm9\"" Jul 15 19:26:11.648756 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:11.648694 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 15 19:26:11.648990 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:11.648959 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 15 19:26:11.648990 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:11.648968 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 15 19:26:11.649148 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:11.649023 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 15 19:26:11.649148 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:11.649040 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 15 19:26:11.649294 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:11.649280 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 15 19:26:11.657760 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:11.657742 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8698c746d9-ck2m6"] Jul 15 19:26:11.669843 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:11.669824 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cb861bd3-b9cf-4671-bc14-15e29bf00106-oauth-serving-cert\") pod \"console-8698c746d9-ck2m6\" (UID: \"cb861bd3-b9cf-4671-bc14-15e29bf00106\") " pod="openshift-console/console-8698c746d9-ck2m6" Jul 15 19:26:11.669942 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:11.669852 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cb861bd3-b9cf-4671-bc14-15e29bf00106-console-config\") pod \"console-8698c746d9-ck2m6\" (UID: \"cb861bd3-b9cf-4671-bc14-15e29bf00106\") " pod="openshift-console/console-8698c746d9-ck2m6" Jul 15 19:26:11.669942 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:11.669871 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxs5k\" (UniqueName: \"kubernetes.io/projected/cb861bd3-b9cf-4671-bc14-15e29bf00106-kube-api-access-pxs5k\") pod \"console-8698c746d9-ck2m6\" (UID: \"cb861bd3-b9cf-4671-bc14-15e29bf00106\") " pod="openshift-console/console-8698c746d9-ck2m6" Jul 15 19:26:11.670056 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:11.670017 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cb861bd3-b9cf-4671-bc14-15e29bf00106-service-ca\") pod \"console-8698c746d9-ck2m6\" (UID: \"cb861bd3-b9cf-4671-bc14-15e29bf00106\") " pod="openshift-console/console-8698c746d9-ck2m6" Jul 15 19:26:11.670137 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:11.670068 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/49f71022-3768-4912-8953-29b19518fe3a-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-zqw6v\" (UID: \"49f71022-3768-4912-8953-29b19518fe3a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-zqw6v" Jul 15 19:26:11.670137 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:11.670095 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cb861bd3-b9cf-4671-bc14-15e29bf00106-console-serving-cert\") pod \"console-8698c746d9-ck2m6\" (UID: \"cb861bd3-b9cf-4671-bc14-15e29bf00106\") " pod="openshift-console/console-8698c746d9-ck2m6" Jul 15 19:26:11.670261 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:11.670146 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cb861bd3-b9cf-4671-bc14-15e29bf00106-console-oauth-config\") pod \"console-8698c746d9-ck2m6\" (UID: \"cb861bd3-b9cf-4671-bc14-15e29bf00106\") " pod="openshift-console/console-8698c746d9-ck2m6" Jul 15 19:26:11.670261 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:26:11.670172 2577 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 15 19:26:11.670261 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:26:11.670238 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49f71022-3768-4912-8953-29b19518fe3a-tls-certificates podName:49f71022-3768-4912-8953-29b19518fe3a nodeName:}" failed. No retries permitted until 2026-07-15 19:26:12.170212962 +0000 UTC m=+285.747214616 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/49f71022-3768-4912-8953-29b19518fe3a-tls-certificates") pod "prometheus-operator-admission-webhook-77d98bc786-zqw6v" (UID: "49f71022-3768-4912-8953-29b19518fe3a") : secret "prometheus-operator-admission-webhook-tls" not found Jul 15 19:26:11.770923 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:11.770898 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cb861bd3-b9cf-4671-bc14-15e29bf00106-oauth-serving-cert\") pod \"console-8698c746d9-ck2m6\" (UID: \"cb861bd3-b9cf-4671-bc14-15e29bf00106\") " pod="openshift-console/console-8698c746d9-ck2m6" Jul 15 19:26:11.771044 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:11.770926 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cb861bd3-b9cf-4671-bc14-15e29bf00106-console-config\") pod \"console-8698c746d9-ck2m6\" (UID: \"cb861bd3-b9cf-4671-bc14-15e29bf00106\") " pod="openshift-console/console-8698c746d9-ck2m6" Jul 15 19:26:11.771044 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:11.770943 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxs5k\" (UniqueName: \"kubernetes.io/projected/cb861bd3-b9cf-4671-bc14-15e29bf00106-kube-api-access-pxs5k\") pod \"console-8698c746d9-ck2m6\" (UID: \"cb861bd3-b9cf-4671-bc14-15e29bf00106\") " pod="openshift-console/console-8698c746d9-ck2m6" Jul 15 19:26:11.771044 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:11.770963 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/56c7bcac-8a82-4c32-8658-1db0bad8d228-metrics-tls\") pod \"dns-default-5zrgr\" (UID: \"56c7bcac-8a82-4c32-8658-1db0bad8d228\") " pod="openshift-dns/dns-default-5zrgr" Jul 15 19:26:11.771206 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:11.771116 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cb861bd3-b9cf-4671-bc14-15e29bf00106-service-ca\") pod \"console-8698c746d9-ck2m6\" (UID: \"cb861bd3-b9cf-4671-bc14-15e29bf00106\") " pod="openshift-console/console-8698c746d9-ck2m6" Jul 15 19:26:11.771206 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:11.771174 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cb861bd3-b9cf-4671-bc14-15e29bf00106-console-serving-cert\") pod \"console-8698c746d9-ck2m6\" (UID: \"cb861bd3-b9cf-4671-bc14-15e29bf00106\") " pod="openshift-console/console-8698c746d9-ck2m6" Jul 15 19:26:11.771342 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:11.771227 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cb861bd3-b9cf-4671-bc14-15e29bf00106-console-oauth-config\") pod \"console-8698c746d9-ck2m6\" (UID: \"cb861bd3-b9cf-4671-bc14-15e29bf00106\") " pod="openshift-console/console-8698c746d9-ck2m6" Jul 15 19:26:11.771342 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:11.771261 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-registry-tls\") pod \"image-registry-9db745d49-xxf8k\" (UID: \"1235737d-3c8a-4c3a-af1a-dfd3351d72fb\") " pod="openshift-image-registry/image-registry-9db745d49-xxf8k" Jul 15 19:26:11.771342 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:11.771297 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/152d1d6b-5ee1-40c2-80a8-388091a57c58-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-w4b96\" (UID: \"152d1d6b-5ee1-40c2-80a8-388091a57c58\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-w4b96" Jul 15 19:26:11.771831 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:11.771809 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cb861bd3-b9cf-4671-bc14-15e29bf00106-console-config\") pod \"console-8698c746d9-ck2m6\" (UID: \"cb861bd3-b9cf-4671-bc14-15e29bf00106\") " pod="openshift-console/console-8698c746d9-ck2m6" Jul 15 19:26:11.771988 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:11.771927 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cb861bd3-b9cf-4671-bc14-15e29bf00106-service-ca\") pod \"console-8698c746d9-ck2m6\" (UID: \"cb861bd3-b9cf-4671-bc14-15e29bf00106\") " pod="openshift-console/console-8698c746d9-ck2m6" Jul 15 19:26:11.772092 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:11.771976 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cb861bd3-b9cf-4671-bc14-15e29bf00106-oauth-serving-cert\") pod \"console-8698c746d9-ck2m6\" (UID: \"cb861bd3-b9cf-4671-bc14-15e29bf00106\") " pod="openshift-console/console-8698c746d9-ck2m6" Jul 15 19:26:11.773603 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:11.773581 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/56c7bcac-8a82-4c32-8658-1db0bad8d228-metrics-tls\") pod \"dns-default-5zrgr\" (UID: \"56c7bcac-8a82-4c32-8658-1db0bad8d228\") " pod="openshift-dns/dns-default-5zrgr" Jul 15 19:26:11.773882 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:11.773851 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cb861bd3-b9cf-4671-bc14-15e29bf00106-console-serving-cert\") pod \"console-8698c746d9-ck2m6\" (UID: \"cb861bd3-b9cf-4671-bc14-15e29bf00106\") " pod="openshift-console/console-8698c746d9-ck2m6" Jul 15 19:26:11.774073 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:11.774047 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cb861bd3-b9cf-4671-bc14-15e29bf00106-console-oauth-config\") pod \"console-8698c746d9-ck2m6\" (UID: \"cb861bd3-b9cf-4671-bc14-15e29bf00106\") " pod="openshift-console/console-8698c746d9-ck2m6" Jul 15 19:26:11.774137 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:11.774062 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-registry-tls\") pod \"image-registry-9db745d49-xxf8k\" (UID: \"1235737d-3c8a-4c3a-af1a-dfd3351d72fb\") " pod="openshift-image-registry/image-registry-9db745d49-xxf8k" Jul 15 19:26:11.774193 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:11.774146 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/152d1d6b-5ee1-40c2-80a8-388091a57c58-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-w4b96\" (UID: \"152d1d6b-5ee1-40c2-80a8-388091a57c58\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-w4b96" Jul 15 19:26:11.778755 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:11.778706 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxs5k\" (UniqueName: \"kubernetes.io/projected/cb861bd3-b9cf-4671-bc14-15e29bf00106-kube-api-access-pxs5k\") pod \"console-8698c746d9-ck2m6\" (UID: \"cb861bd3-b9cf-4671-bc14-15e29bf00106\") " pod="openshift-console/console-8698c746d9-ck2m6" Jul 15 19:26:11.891623 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:11.891587 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-j42p2\"" Jul 15 19:26:11.891623 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:11.891599 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bw2gg\"" Jul 15 19:26:11.891866 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:11.891704 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-r4g6n\"" Jul 15 19:26:11.899536 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:11.899511 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5zrgr" Jul 15 19:26:11.899536 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:11.899528 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9db745d49-xxf8k" Jul 15 19:26:11.899690 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:11.899590 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-w4b96" Jul 15 19:26:11.955652 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:11.955611 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8698c746d9-ck2m6" Jul 15 19:26:12.087707 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:12.087645 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5zrgr"] Jul 15 19:26:12.091863 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:26:12.091828 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56c7bcac_8a82_4c32_8658_1db0bad8d228.slice/crio-6581a17241380235063509c46a5374444f21d6df316a330e1da3e4cb175c08fd WatchSource:0}: Error finding container 6581a17241380235063509c46a5374444f21d6df316a330e1da3e4cb175c08fd: Status 404 returned error can't find the container with id 6581a17241380235063509c46a5374444f21d6df316a330e1da3e4cb175c08fd Jul 15 19:26:12.098793 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:12.098772 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-w4b96"] Jul 15 19:26:12.102883 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:26:12.102861 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod152d1d6b_5ee1_40c2_80a8_388091a57c58.slice/crio-34125730e6f2bee13456985b8bbda9f337bc47bdc613137b96ed2be8181814db WatchSource:0}: Error finding container 34125730e6f2bee13456985b8bbda9f337bc47bdc613137b96ed2be8181814db: Status 404 returned error can't find the container with id 34125730e6f2bee13456985b8bbda9f337bc47bdc613137b96ed2be8181814db Jul 15 19:26:12.120757 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:12.120734 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-9db745d49-xxf8k"] Jul 15 19:26:12.124169 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:26:12.124145 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1235737d_3c8a_4c3a_af1a_dfd3351d72fb.slice/crio-41d2602bf9f526f3a000234b405e0512141d67dfd71e557645ebd075039fdd7c WatchSource:0}: Error finding container 41d2602bf9f526f3a000234b405e0512141d67dfd71e557645ebd075039fdd7c: Status 404 returned error can't find the container with id 41d2602bf9f526f3a000234b405e0512141d67dfd71e557645ebd075039fdd7c Jul 15 19:26:12.148060 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:12.148039 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8698c746d9-ck2m6"] Jul 15 19:26:12.154260 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:26:12.154234 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb861bd3_b9cf_4671_bc14_15e29bf00106.slice/crio-96e71dff144109669a4e5a133f2dbe977ea9791bbf8e0bbd8ba41e2e3fbb08d6 WatchSource:0}: Error finding container 96e71dff144109669a4e5a133f2dbe977ea9791bbf8e0bbd8ba41e2e3fbb08d6: Status 404 returned error can't find the container with id 96e71dff144109669a4e5a133f2dbe977ea9791bbf8e0bbd8ba41e2e3fbb08d6 Jul 15 19:26:12.175089 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:12.175072 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/49f71022-3768-4912-8953-29b19518fe3a-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-zqw6v\" (UID: \"49f71022-3768-4912-8953-29b19518fe3a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-zqw6v" Jul 15 19:26:12.177268 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:12.177247 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/49f71022-3768-4912-8953-29b19518fe3a-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-zqw6v\" (UID: \"49f71022-3768-4912-8953-29b19518fe3a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-zqw6v" Jul 15 19:26:12.437764 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:12.437731 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-zqw6v" Jul 15 19:26:12.567918 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:12.567788 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-zqw6v"] Jul 15 19:26:12.570478 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:26:12.570454 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49f71022_3768_4912_8953_29b19518fe3a.slice/crio-53c23c05f8f55d268b690e34ad19bf11324ac2871cbb8ca860bd6663f164a44c WatchSource:0}: Error finding container 53c23c05f8f55d268b690e34ad19bf11324ac2871cbb8ca860bd6663f164a44c: Status 404 returned error can't find the container with id 53c23c05f8f55d268b690e34ad19bf11324ac2871cbb8ca860bd6663f164a44c Jul 15 19:26:12.901998 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:12.901960 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-zqw6v" event={"ID":"49f71022-3768-4912-8953-29b19518fe3a","Type":"ContainerStarted","Data":"53c23c05f8f55d268b690e34ad19bf11324ac2871cbb8ca860bd6663f164a44c"} Jul 15 19:26:12.903224 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:12.903186 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5zrgr" event={"ID":"56c7bcac-8a82-4c32-8658-1db0bad8d228","Type":"ContainerStarted","Data":"6581a17241380235063509c46a5374444f21d6df316a330e1da3e4cb175c08fd"} Jul 15 19:26:12.904529 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:12.904499 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8698c746d9-ck2m6" event={"ID":"cb861bd3-b9cf-4671-bc14-15e29bf00106","Type":"ContainerStarted","Data":"96e71dff144109669a4e5a133f2dbe977ea9791bbf8e0bbd8ba41e2e3fbb08d6"} Jul 15 19:26:12.905737 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:12.905713 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-w4b96" event={"ID":"152d1d6b-5ee1-40c2-80a8-388091a57c58","Type":"ContainerStarted","Data":"34125730e6f2bee13456985b8bbda9f337bc47bdc613137b96ed2be8181814db"} Jul 15 19:26:12.907264 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:12.907230 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9db745d49-xxf8k" event={"ID":"1235737d-3c8a-4c3a-af1a-dfd3351d72fb","Type":"ContainerStarted","Data":"a22cd7e977e85aeed975b76c75be8053de3517f8ed9b62018b4e97820d745d3a"} Jul 15 19:26:12.907359 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:12.907270 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9db745d49-xxf8k" event={"ID":"1235737d-3c8a-4c3a-af1a-dfd3351d72fb","Type":"ContainerStarted","Data":"41d2602bf9f526f3a000234b405e0512141d67dfd71e557645ebd075039fdd7c"} Jul 15 19:26:12.907419 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:12.907377 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-9db745d49-xxf8k" Jul 15 19:26:15.921368 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:15.921329 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-zqw6v" event={"ID":"49f71022-3768-4912-8953-29b19518fe3a","Type":"ContainerStarted","Data":"7aee666917476109f3c26dfceb23665a931b7bd65a17fd5f8d1e79f6d858b0fb"} Jul 15 19:26:15.921823 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:15.921508 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-zqw6v" Jul 15 19:26:15.923299 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:15.923266 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5zrgr" event={"ID":"56c7bcac-8a82-4c32-8658-1db0bad8d228","Type":"ContainerStarted","Data":"1ce57456b3bee99670735369f6e84cb588289b763259532f12c14ba935cc6b8b"} Jul 15 19:26:15.923460 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:15.923345 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-5zrgr" Jul 15 19:26:15.923460 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:15.923372 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5zrgr" event={"ID":"56c7bcac-8a82-4c32-8658-1db0bad8d228","Type":"ContainerStarted","Data":"2fcbadd9ea80d06846d1dea939fc9f4146812eb71d6e6e86665f0b677dca66b7"} Jul 15 19:26:15.924923 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:15.924898 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8698c746d9-ck2m6" event={"ID":"cb861bd3-b9cf-4671-bc14-15e29bf00106","Type":"ContainerStarted","Data":"95ede16f9e0522ace911d0385c12e9a7234a70f3a7b2e27facb178c370a55896"} Jul 15 19:26:15.926779 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:15.926761 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-zqw6v" Jul 15 19:26:15.926902 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:15.926794 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-w4b96" event={"ID":"152d1d6b-5ee1-40c2-80a8-388091a57c58","Type":"ContainerStarted","Data":"4032c2c802ca2b6c77d7941044061d8c92cedfd8e3f2859ae6cb6193672a05c4"} Jul 15 19:26:15.946889 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:15.946848 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-zqw6v" podStartSLOduration=2.081448877 podStartE2EDuration="4.946835504s" totalImagesPullingTime="2.865386627s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:26:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:26:15.945227761 +0000 UTC m=+289.522229424" watchObservedRunningTime="2026-07-15 19:26:15.946835504 +0000 UTC m=+289.523837185" Jul 15 19:26:15.947016 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:15.946999 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-9db745d49-xxf8k" podStartSLOduration=288.946995338 podStartE2EDuration="4m48.946995338s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:21:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:26:12.953682739 +0000 UTC m=+286.530684420" watchObservedRunningTime="2026-07-15 19:26:15.946995338 +0000 UTC m=+289.523996987" Jul 15 19:26:15.966331 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:15.966280 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-5zrgr" podStartSLOduration=251.627217859 podStartE2EDuration="4m14.966269017s" totalImagesPullingTime="3.339051158s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:22:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:26:15.965191019 +0000 UTC m=+289.542192690" watchObservedRunningTime="2026-07-15 19:26:15.966269017 +0000 UTC m=+289.543270679" Jul 15 19:26:16.003964 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:16.003925 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8698c746d9-ck2m6" podStartSLOduration=1.712842633 podStartE2EDuration="5.003913444s" totalImagesPullingTime="3.291070811s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:26:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:26:16.003347034 +0000 UTC m=+289.580348729" watchObservedRunningTime="2026-07-15 19:26:16.003913444 +0000 UTC m=+289.580915151" Jul 15 19:26:16.021596 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:16.021561 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-w4b96" podStartSLOduration=280.693387446 podStartE2EDuration="4m44.021545098s" totalImagesPullingTime="3.328157652s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:21:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:26:16.020371296 +0000 UTC m=+289.597372986" watchObservedRunningTime="2026-07-15 19:26:16.021545098 +0000 UTC m=+289.598546827" Jul 15 19:26:20.089236 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:26:20.089197 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-cmqc9" podUID="237b9375-100a-4d62-83f0-cb271ab8e0bb" Jul 15 19:26:21.956162 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:21.956131 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8698c746d9-ck2m6" Jul 15 19:26:21.956539 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:21.956188 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8698c746d9-ck2m6" Jul 15 19:26:21.957125 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:21.957103 2577 patch_prober.go:28] interesting pod/console-8698c746d9-ck2m6 container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.132.0.18:8443/health\": dial tcp 10.132.0.18:8443: connect: connection refused" start-of-body= Jul 15 19:26:21.957171 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:21.957146 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-8698c746d9-ck2m6" podUID="cb861bd3-b9cf-4671-bc14-15e29bf00106" containerName="console" probeResult="failure" output="Get \"https://10.132.0.18:8443/health\": dial tcp 10.132.0.18:8443: connect: connection refused" Jul 15 19:26:25.932414 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:25.932385 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-5zrgr" Jul 15 19:26:26.865217 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:26.865188 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-j2rcv_a2ded0a5-37e2-48fd-98b6-3d64ae279896/console-operator/1.log" Jul 15 19:26:26.866705 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:26.866675 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-j2rcv_a2ded0a5-37e2-48fd-98b6-3d64ae279896/console-operator/1.log" Jul 15 19:26:26.872633 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:26.872613 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fw6vl_89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21/ovn-acl-logging/0.log" Jul 15 19:26:26.874351 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:26.874332 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fw6vl_89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21/ovn-acl-logging/0.log" Jul 15 19:26:26.875800 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:26.875780 2577 kubelet.go:1648] "Image garbage collection succeeded" Jul 15 19:26:29.058193 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:29.058162 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-9db745d49-xxf8k"] Jul 15 19:26:29.063659 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:29.062497 2577 patch_prober.go:28] interesting pod/image-registry-9db745d49-xxf8k container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 15 19:26:29.063659 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:29.062533 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-9db745d49-xxf8k" podUID="1235737d-3c8a-4c3a-af1a-dfd3351d72fb" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 19:26:30.088981 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.088950 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-4strs"] Jul 15 19:26:30.092656 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.092636 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4strs" Jul 15 19:26:30.095472 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.095445 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 15 19:26:30.095765 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.095715 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-fk99v"] Jul 15 19:26:30.095992 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.095971 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 15 19:26:30.096109 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.096088 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 15 19:26:30.096172 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.096120 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 15 19:26:30.096977 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.096958 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 15 19:26:30.100640 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.100618 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-r29fs\"" Jul 15 19:26:30.101126 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.101109 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-fk99v" Jul 15 19:26:30.103660 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.103638 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-ljl2t\"" Jul 15 19:26:30.103761 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.103692 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 15 19:26:30.103986 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.103968 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-4strs"] Jul 15 19:26:30.104046 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.103995 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 15 19:26:30.104046 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.104008 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 15 19:26:30.109768 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.109747 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-ptl2w"] Jul 15 19:26:30.113410 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.113393 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ptl2w" Jul 15 19:26:30.115788 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.115769 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 15 19:26:30.116015 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.115995 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 15 19:26:30.116105 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.116036 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-qvh6l\"" Jul 15 19:26:30.116275 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.116261 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 15 19:26:30.123043 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.123024 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-fk99v"] Jul 15 19:26:30.204265 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.204238 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/44d02d02-4986-4a68-8398-894bdd356164-root\") pod \"node-exporter-ptl2w\" (UID: \"44d02d02-4986-4a68-8398-894bdd356164\") " pod="openshift-monitoring/node-exporter-ptl2w" Jul 15 19:26:30.204265 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.204266 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/68f7eee7-04d4-49ff-89d7-f10b984a5f55-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-fk99v\" (UID: \"68f7eee7-04d4-49ff-89d7-f10b984a5f55\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-fk99v" Jul 15 19:26:30.204452 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.204286 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/44d02d02-4986-4a68-8398-894bdd356164-sys\") pod \"node-exporter-ptl2w\" (UID: \"44d02d02-4986-4a68-8398-894bdd356164\") " pod="openshift-monitoring/node-exporter-ptl2w" Jul 15 19:26:30.204452 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.204303 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/44d02d02-4986-4a68-8398-894bdd356164-node-exporter-tls\") pod \"node-exporter-ptl2w\" (UID: \"44d02d02-4986-4a68-8398-894bdd356164\") " pod="openshift-monitoring/node-exporter-ptl2w" Jul 15 19:26:30.204452 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.204352 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wnpg\" (UniqueName: \"kubernetes.io/projected/44d02d02-4986-4a68-8398-894bdd356164-kube-api-access-8wnpg\") pod \"node-exporter-ptl2w\" (UID: \"44d02d02-4986-4a68-8398-894bdd356164\") " pod="openshift-monitoring/node-exporter-ptl2w" Jul 15 19:26:30.204452 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.204407 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jm9v\" (UniqueName: \"kubernetes.io/projected/68f7eee7-04d4-49ff-89d7-f10b984a5f55-kube-api-access-8jm9v\") pod \"kube-state-metrics-785778ff74-fk99v\" (UID: \"68f7eee7-04d4-49ff-89d7-f10b984a5f55\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-fk99v" Jul 15 19:26:30.204624 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.204455 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpc2d\" (UniqueName: \"kubernetes.io/projected/7ffe651c-2927-43a4-96f1-f7bf31b7f598-kube-api-access-kpc2d\") pod \"openshift-state-metrics-84bd68b896-4strs\" (UID: \"7ffe651c-2927-43a4-96f1-f7bf31b7f598\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4strs" Jul 15 19:26:30.204624 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.204496 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/68f7eee7-04d4-49ff-89d7-f10b984a5f55-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-fk99v\" (UID: \"68f7eee7-04d4-49ff-89d7-f10b984a5f55\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-fk99v" Jul 15 19:26:30.204624 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.204523 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/44d02d02-4986-4a68-8398-894bdd356164-node-exporter-wtmp\") pod \"node-exporter-ptl2w\" (UID: \"44d02d02-4986-4a68-8398-894bdd356164\") " pod="openshift-monitoring/node-exporter-ptl2w" Jul 15 19:26:30.204624 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.204543 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7ffe651c-2927-43a4-96f1-f7bf31b7f598-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-4strs\" (UID: \"7ffe651c-2927-43a4-96f1-f7bf31b7f598\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4strs" Jul 15 19:26:30.204624 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.204560 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/44d02d02-4986-4a68-8398-894bdd356164-metrics-client-ca\") pod \"node-exporter-ptl2w\" (UID: \"44d02d02-4986-4a68-8398-894bdd356164\") " pod="openshift-monitoring/node-exporter-ptl2w" Jul 15 19:26:30.204624 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.204595 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ffe651c-2927-43a4-96f1-f7bf31b7f598-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-4strs\" (UID: \"7ffe651c-2927-43a4-96f1-f7bf31b7f598\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4strs" Jul 15 19:26:30.204624 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.204618 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/44d02d02-4986-4a68-8398-894bdd356164-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ptl2w\" (UID: \"44d02d02-4986-4a68-8398-894bdd356164\") " pod="openshift-monitoring/node-exporter-ptl2w" Jul 15 19:26:30.204948 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.204634 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68f7eee7-04d4-49ff-89d7-f10b984a5f55-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-fk99v\" (UID: \"68f7eee7-04d4-49ff-89d7-f10b984a5f55\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-fk99v" Jul 15 19:26:30.204948 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.204651 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7ffe651c-2927-43a4-96f1-f7bf31b7f598-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-4strs\" (UID: \"7ffe651c-2927-43a4-96f1-f7bf31b7f598\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4strs" Jul 15 19:26:30.204948 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.204672 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/44d02d02-4986-4a68-8398-894bdd356164-node-exporter-accelerators-collector-config\") pod \"node-exporter-ptl2w\" (UID: \"44d02d02-4986-4a68-8398-894bdd356164\") " pod="openshift-monitoring/node-exporter-ptl2w" Jul 15 19:26:30.204948 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.204716 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/44d02d02-4986-4a68-8398-894bdd356164-node-exporter-textfile\") pod \"node-exporter-ptl2w\" (UID: \"44d02d02-4986-4a68-8398-894bdd356164\") " pod="openshift-monitoring/node-exporter-ptl2w" Jul 15 19:26:30.204948 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.204749 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/68f7eee7-04d4-49ff-89d7-f10b984a5f55-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-fk99v\" (UID: \"68f7eee7-04d4-49ff-89d7-f10b984a5f55\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-fk99v" Jul 15 19:26:30.204948 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.204775 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/68f7eee7-04d4-49ff-89d7-f10b984a5f55-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-fk99v\" (UID: \"68f7eee7-04d4-49ff-89d7-f10b984a5f55\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-fk99v" Jul 15 19:26:30.305816 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.305791 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/68f7eee7-04d4-49ff-89d7-f10b984a5f55-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-fk99v\" (UID: \"68f7eee7-04d4-49ff-89d7-f10b984a5f55\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-fk99v" Jul 15 19:26:30.305967 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.305821 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/44d02d02-4986-4a68-8398-894bdd356164-node-exporter-wtmp\") pod \"node-exporter-ptl2w\" (UID: \"44d02d02-4986-4a68-8398-894bdd356164\") " pod="openshift-monitoring/node-exporter-ptl2w" Jul 15 19:26:30.305967 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.305842 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7ffe651c-2927-43a4-96f1-f7bf31b7f598-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-4strs\" (UID: \"7ffe651c-2927-43a4-96f1-f7bf31b7f598\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4strs" Jul 15 19:26:30.305967 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.305857 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/44d02d02-4986-4a68-8398-894bdd356164-metrics-client-ca\") pod \"node-exporter-ptl2w\" (UID: \"44d02d02-4986-4a68-8398-894bdd356164\") " pod="openshift-monitoring/node-exporter-ptl2w" Jul 15 19:26:30.305967 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.305887 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ffe651c-2927-43a4-96f1-f7bf31b7f598-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-4strs\" (UID: \"7ffe651c-2927-43a4-96f1-f7bf31b7f598\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4strs" Jul 15 19:26:30.305967 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.305908 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/44d02d02-4986-4a68-8398-894bdd356164-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ptl2w\" (UID: \"44d02d02-4986-4a68-8398-894bdd356164\") " pod="openshift-monitoring/node-exporter-ptl2w" Jul 15 19:26:30.305967 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.305931 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68f7eee7-04d4-49ff-89d7-f10b984a5f55-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-fk99v\" (UID: \"68f7eee7-04d4-49ff-89d7-f10b984a5f55\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-fk99v" Jul 15 19:26:30.305967 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.305953 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7ffe651c-2927-43a4-96f1-f7bf31b7f598-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-4strs\" (UID: \"7ffe651c-2927-43a4-96f1-f7bf31b7f598\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4strs" Jul 15 19:26:30.306334 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.305976 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/44d02d02-4986-4a68-8398-894bdd356164-node-exporter-accelerators-collector-config\") pod \"node-exporter-ptl2w\" (UID: \"44d02d02-4986-4a68-8398-894bdd356164\") " pod="openshift-monitoring/node-exporter-ptl2w" Jul 15 19:26:30.306334 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.305997 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/44d02d02-4986-4a68-8398-894bdd356164-node-exporter-wtmp\") pod \"node-exporter-ptl2w\" (UID: \"44d02d02-4986-4a68-8398-894bdd356164\") " pod="openshift-monitoring/node-exporter-ptl2w" Jul 15 19:26:30.306334 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.306002 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/44d02d02-4986-4a68-8398-894bdd356164-node-exporter-textfile\") pod \"node-exporter-ptl2w\" (UID: \"44d02d02-4986-4a68-8398-894bdd356164\") " pod="openshift-monitoring/node-exporter-ptl2w" Jul 15 19:26:30.306334 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.306030 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/68f7eee7-04d4-49ff-89d7-f10b984a5f55-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-fk99v\" (UID: \"68f7eee7-04d4-49ff-89d7-f10b984a5f55\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-fk99v" Jul 15 19:26:30.306334 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.306054 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/68f7eee7-04d4-49ff-89d7-f10b984a5f55-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-fk99v\" (UID: \"68f7eee7-04d4-49ff-89d7-f10b984a5f55\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-fk99v" Jul 15 19:26:30.306334 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:26:30.306089 2577 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 15 19:26:30.306334 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.306126 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/44d02d02-4986-4a68-8398-894bdd356164-root\") pod \"node-exporter-ptl2w\" (UID: \"44d02d02-4986-4a68-8398-894bdd356164\") " pod="openshift-monitoring/node-exporter-ptl2w" Jul 15 19:26:30.306334 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:26:30.306155 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ffe651c-2927-43a4-96f1-f7bf31b7f598-openshift-state-metrics-tls podName:7ffe651c-2927-43a4-96f1-f7bf31b7f598 nodeName:}" failed. No retries permitted until 2026-07-15 19:26:30.806135444 +0000 UTC m=+304.383137093 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/7ffe651c-2927-43a4-96f1-f7bf31b7f598-openshift-state-metrics-tls") pod "openshift-state-metrics-84bd68b896-4strs" (UID: "7ffe651c-2927-43a4-96f1-f7bf31b7f598") : secret "openshift-state-metrics-tls" not found Jul 15 19:26:30.306334 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.306256 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/68f7eee7-04d4-49ff-89d7-f10b984a5f55-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-fk99v\" (UID: \"68f7eee7-04d4-49ff-89d7-f10b984a5f55\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-fk99v" Jul 15 19:26:30.306334 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.306089 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/44d02d02-4986-4a68-8398-894bdd356164-root\") pod \"node-exporter-ptl2w\" (UID: \"44d02d02-4986-4a68-8398-894bdd356164\") " pod="openshift-monitoring/node-exporter-ptl2w" Jul 15 19:26:30.306845 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.306352 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/68f7eee7-04d4-49ff-89d7-f10b984a5f55-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-fk99v\" (UID: \"68f7eee7-04d4-49ff-89d7-f10b984a5f55\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-fk99v" Jul 15 19:26:30.306845 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.306394 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/44d02d02-4986-4a68-8398-894bdd356164-sys\") pod \"node-exporter-ptl2w\" (UID: \"44d02d02-4986-4a68-8398-894bdd356164\") " pod="openshift-monitoring/node-exporter-ptl2w" Jul 15 19:26:30.306845 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.306419 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/44d02d02-4986-4a68-8398-894bdd356164-node-exporter-tls\") pod \"node-exporter-ptl2w\" (UID: \"44d02d02-4986-4a68-8398-894bdd356164\") " pod="openshift-monitoring/node-exporter-ptl2w" Jul 15 19:26:30.306845 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.306446 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wnpg\" (UniqueName: \"kubernetes.io/projected/44d02d02-4986-4a68-8398-894bdd356164-kube-api-access-8wnpg\") pod \"node-exporter-ptl2w\" (UID: \"44d02d02-4986-4a68-8398-894bdd356164\") " pod="openshift-monitoring/node-exporter-ptl2w" Jul 15 19:26:30.306845 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.306484 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jm9v\" (UniqueName: \"kubernetes.io/projected/68f7eee7-04d4-49ff-89d7-f10b984a5f55-kube-api-access-8jm9v\") pod \"kube-state-metrics-785778ff74-fk99v\" (UID: \"68f7eee7-04d4-49ff-89d7-f10b984a5f55\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-fk99v" Jul 15 19:26:30.306845 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.306536 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpc2d\" (UniqueName: \"kubernetes.io/projected/7ffe651c-2927-43a4-96f1-f7bf31b7f598-kube-api-access-kpc2d\") pod \"openshift-state-metrics-84bd68b896-4strs\" (UID: \"7ffe651c-2927-43a4-96f1-f7bf31b7f598\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4strs" Jul 15 19:26:30.306845 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.306565 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/44d02d02-4986-4a68-8398-894bdd356164-metrics-client-ca\") pod \"node-exporter-ptl2w\" (UID: \"44d02d02-4986-4a68-8398-894bdd356164\") " pod="openshift-monitoring/node-exporter-ptl2w" Jul 15 19:26:30.306845 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.306649 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/44d02d02-4986-4a68-8398-894bdd356164-node-exporter-accelerators-collector-config\") pod \"node-exporter-ptl2w\" (UID: \"44d02d02-4986-4a68-8398-894bdd356164\") " pod="openshift-monitoring/node-exporter-ptl2w" Jul 15 19:26:30.306845 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.306665 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ffe651c-2927-43a4-96f1-f7bf31b7f598-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-4strs\" (UID: \"7ffe651c-2927-43a4-96f1-f7bf31b7f598\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4strs" Jul 15 19:26:30.306845 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.306823 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68f7eee7-04d4-49ff-89d7-f10b984a5f55-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-fk99v\" (UID: \"68f7eee7-04d4-49ff-89d7-f10b984a5f55\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-fk99v" Jul 15 19:26:30.307375 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.306910 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/44d02d02-4986-4a68-8398-894bdd356164-node-exporter-textfile\") pod \"node-exporter-ptl2w\" (UID: \"44d02d02-4986-4a68-8398-894bdd356164\") " pod="openshift-monitoring/node-exporter-ptl2w" Jul 15 19:26:30.307375 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:26:30.306940 2577 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 15 19:26:30.307375 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:26:30.306987 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68f7eee7-04d4-49ff-89d7-f10b984a5f55-kube-state-metrics-tls podName:68f7eee7-04d4-49ff-89d7-f10b984a5f55 nodeName:}" failed. No retries permitted until 2026-07-15 19:26:30.80697209 +0000 UTC m=+304.383973740 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/68f7eee7-04d4-49ff-89d7-f10b984a5f55-kube-state-metrics-tls") pod "kube-state-metrics-785778ff74-fk99v" (UID: "68f7eee7-04d4-49ff-89d7-f10b984a5f55") : secret "kube-state-metrics-tls" not found Jul 15 19:26:30.307375 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.307194 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/44d02d02-4986-4a68-8398-894bdd356164-sys\") pod \"node-exporter-ptl2w\" (UID: \"44d02d02-4986-4a68-8398-894bdd356164\") " pod="openshift-monitoring/node-exporter-ptl2w" Jul 15 19:26:30.307375 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.307250 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/68f7eee7-04d4-49ff-89d7-f10b984a5f55-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-fk99v\" (UID: \"68f7eee7-04d4-49ff-89d7-f10b984a5f55\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-fk99v" Jul 15 19:26:30.308579 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.308560 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/44d02d02-4986-4a68-8398-894bdd356164-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ptl2w\" (UID: \"44d02d02-4986-4a68-8398-894bdd356164\") " pod="openshift-monitoring/node-exporter-ptl2w" Jul 15 19:26:30.308917 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.308894 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7ffe651c-2927-43a4-96f1-f7bf31b7f598-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-4strs\" (UID: \"7ffe651c-2927-43a4-96f1-f7bf31b7f598\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4strs" Jul 15 19:26:30.309003 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.308987 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/44d02d02-4986-4a68-8398-894bdd356164-node-exporter-tls\") pod \"node-exporter-ptl2w\" (UID: \"44d02d02-4986-4a68-8398-894bdd356164\") " pod="openshift-monitoring/node-exporter-ptl2w" Jul 15 19:26:30.309232 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.309211 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/68f7eee7-04d4-49ff-89d7-f10b984a5f55-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-fk99v\" (UID: \"68f7eee7-04d4-49ff-89d7-f10b984a5f55\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-fk99v" Jul 15 19:26:30.315588 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.315559 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpc2d\" (UniqueName: \"kubernetes.io/projected/7ffe651c-2927-43a4-96f1-f7bf31b7f598-kube-api-access-kpc2d\") pod \"openshift-state-metrics-84bd68b896-4strs\" (UID: \"7ffe651c-2927-43a4-96f1-f7bf31b7f598\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4strs" Jul 15 19:26:30.316065 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.316044 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wnpg\" (UniqueName: \"kubernetes.io/projected/44d02d02-4986-4a68-8398-894bdd356164-kube-api-access-8wnpg\") pod \"node-exporter-ptl2w\" (UID: \"44d02d02-4986-4a68-8398-894bdd356164\") " pod="openshift-monitoring/node-exporter-ptl2w" Jul 15 19:26:30.316151 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.316099 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jm9v\" (UniqueName: \"kubernetes.io/projected/68f7eee7-04d4-49ff-89d7-f10b984a5f55-kube-api-access-8jm9v\") pod \"kube-state-metrics-785778ff74-fk99v\" (UID: \"68f7eee7-04d4-49ff-89d7-f10b984a5f55\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-fk99v" Jul 15 19:26:30.423678 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.423622 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ptl2w" Jul 15 19:26:30.436278 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:26:30.436251 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44d02d02_4986_4a68_8398_894bdd356164.slice/crio-7b86a94bf3f6df74fdc5a6629a859e3248622c752fdc4b4e0b748348b698139d WatchSource:0}: Error finding container 7b86a94bf3f6df74fdc5a6629a859e3248622c752fdc4b4e0b748348b698139d: Status 404 returned error can't find the container with id 7b86a94bf3f6df74fdc5a6629a859e3248622c752fdc4b4e0b748348b698139d Jul 15 19:26:30.438322 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.438293 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 19:26:30.810036 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.810003 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7ffe651c-2927-43a4-96f1-f7bf31b7f598-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-4strs\" (UID: \"7ffe651c-2927-43a4-96f1-f7bf31b7f598\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4strs" Jul 15 19:26:30.810189 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.810053 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/68f7eee7-04d4-49ff-89d7-f10b984a5f55-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-fk99v\" (UID: \"68f7eee7-04d4-49ff-89d7-f10b984a5f55\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-fk99v" Jul 15 19:26:30.810189 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:26:30.810158 2577 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 15 19:26:30.810189 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:26:30.810172 2577 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 15 19:26:30.810290 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:26:30.810221 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ffe651c-2927-43a4-96f1-f7bf31b7f598-openshift-state-metrics-tls podName:7ffe651c-2927-43a4-96f1-f7bf31b7f598 nodeName:}" failed. No retries permitted until 2026-07-15 19:26:31.810206375 +0000 UTC m=+305.387208024 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/7ffe651c-2927-43a4-96f1-f7bf31b7f598-openshift-state-metrics-tls") pod "openshift-state-metrics-84bd68b896-4strs" (UID: "7ffe651c-2927-43a4-96f1-f7bf31b7f598") : secret "openshift-state-metrics-tls" not found Jul 15 19:26:30.810290 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:26:30.810235 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68f7eee7-04d4-49ff-89d7-f10b984a5f55-kube-state-metrics-tls podName:68f7eee7-04d4-49ff-89d7-f10b984a5f55 nodeName:}" failed. No retries permitted until 2026-07-15 19:26:31.810229182 +0000 UTC m=+305.387230830 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/68f7eee7-04d4-49ff-89d7-f10b984a5f55-kube-state-metrics-tls") pod "kube-state-metrics-785778ff74-fk99v" (UID: "68f7eee7-04d4-49ff-89d7-f10b984a5f55") : secret "kube-state-metrics-tls" not found Jul 15 19:26:30.977025 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:30.976984 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ptl2w" event={"ID":"44d02d02-4986-4a68-8398-894bdd356164","Type":"ContainerStarted","Data":"7b86a94bf3f6df74fdc5a6629a859e3248622c752fdc4b4e0b748348b698139d"} Jul 15 19:26:31.817756 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:31.817734 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7ffe651c-2927-43a4-96f1-f7bf31b7f598-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-4strs\" (UID: \"7ffe651c-2927-43a4-96f1-f7bf31b7f598\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4strs" Jul 15 19:26:31.818096 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:31.817771 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/68f7eee7-04d4-49ff-89d7-f10b984a5f55-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-fk99v\" (UID: \"68f7eee7-04d4-49ff-89d7-f10b984a5f55\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-fk99v" Jul 15 19:26:31.819923 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:31.819902 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7ffe651c-2927-43a4-96f1-f7bf31b7f598-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-4strs\" (UID: \"7ffe651c-2927-43a4-96f1-f7bf31b7f598\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4strs" Jul 15 19:26:31.820090 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:31.820067 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/68f7eee7-04d4-49ff-89d7-f10b984a5f55-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-fk99v\" (UID: \"68f7eee7-04d4-49ff-89d7-f10b984a5f55\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-fk99v" Jul 15 19:26:31.902646 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:31.902613 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4strs" Jul 15 19:26:31.910418 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:31.910398 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-fk99v" Jul 15 19:26:31.962627 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:31.962444 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8698c746d9-ck2m6" Jul 15 19:26:31.969537 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:31.969492 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8698c746d9-ck2m6" Jul 15 19:26:31.982294 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:31.982261 2577 generic.go:356] "Generic (PLEG): container finished" podID="44d02d02-4986-4a68-8398-894bdd356164" containerID="e414f2cfc6472e21aa0404b34cfcb333b27de598ce619fb936b9e416c9a1c152" exitCode=0 Jul 15 19:26:31.982450 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:31.982345 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ptl2w" event={"ID":"44d02d02-4986-4a68-8398-894bdd356164","Type":"ContainerDied","Data":"e414f2cfc6472e21aa0404b34cfcb333b27de598ce619fb936b9e416c9a1c152"} Jul 15 19:26:32.056285 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:26:32.054485 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ffe651c_2927_43a4_96f1_f7bf31b7f598.slice/crio-d89878e9ff50183f42b0e4e73df1975fdb0aba61990f3b2523ee9877760f29e3 WatchSource:0}: Error finding container d89878e9ff50183f42b0e4e73df1975fdb0aba61990f3b2523ee9877760f29e3: Status 404 returned error can't find the container with id d89878e9ff50183f42b0e4e73df1975fdb0aba61990f3b2523ee9877760f29e3 Jul 15 19:26:32.056285 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:32.056212 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-4strs"] Jul 15 19:26:32.070279 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:32.070118 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-fk99v"] Jul 15 19:26:32.072337 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:26:32.072295 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68f7eee7_04d4_49ff_89d7_f10b984a5f55.slice/crio-02ff71db53ab0bf5cb6d87b5919bf01bf2d4721d7ac6e627ab037accf663f097 WatchSource:0}: Error finding container 02ff71db53ab0bf5cb6d87b5919bf01bf2d4721d7ac6e627ab037accf663f097: Status 404 returned error can't find the container with id 02ff71db53ab0bf5cb6d87b5919bf01bf2d4721d7ac6e627ab037accf663f097 Jul 15 19:26:32.087947 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:32.087922 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmqc9" Jul 15 19:26:32.988702 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:32.988664 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-fk99v" event={"ID":"68f7eee7-04d4-49ff-89d7-f10b984a5f55","Type":"ContainerStarted","Data":"02ff71db53ab0bf5cb6d87b5919bf01bf2d4721d7ac6e627ab037accf663f097"} Jul 15 19:26:32.991261 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:32.991117 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4strs" event={"ID":"7ffe651c-2927-43a4-96f1-f7bf31b7f598","Type":"ContainerStarted","Data":"6daeeaccf0041c8e4a9973da55f5e6450a0fa98f48ba33982dac4435e6a38d9d"} Jul 15 19:26:32.991261 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:32.991185 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4strs" event={"ID":"7ffe651c-2927-43a4-96f1-f7bf31b7f598","Type":"ContainerStarted","Data":"99607004c37c304447e6cb5082a1c5a859ca12f8b32e6ccb3a9886fe2ad405d5"} Jul 15 19:26:32.991261 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:32.991223 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4strs" event={"ID":"7ffe651c-2927-43a4-96f1-f7bf31b7f598","Type":"ContainerStarted","Data":"d89878e9ff50183f42b0e4e73df1975fdb0aba61990f3b2523ee9877760f29e3"} Jul 15 19:26:32.994933 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:32.994104 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ptl2w" event={"ID":"44d02d02-4986-4a68-8398-894bdd356164","Type":"ContainerStarted","Data":"13e5fb1baaf3940f97969f19da24c4d9945d75e2fa95c24af03dce76590311d0"} Jul 15 19:26:32.994933 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:32.994160 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ptl2w" event={"ID":"44d02d02-4986-4a68-8398-894bdd356164","Type":"ContainerStarted","Data":"f8cad02061574a35dae415a6cf1853df07b93a08f31067fc06622102cab993c0"} Jul 15 19:26:33.028072 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:33.028002 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-ptl2w" podStartSLOduration=1.900677121 podStartE2EDuration="3.027984513s" totalImagesPullingTime="1.064088678s" totalInitContainerRuntime="63.218714ms" isStatefulPod=true podCreationTimestamp="2026-07-15 19:26:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:26:33.026908065 +0000 UTC m=+306.603909764" watchObservedRunningTime="2026-07-15 19:26:33.027984513 +0000 UTC m=+306.604986183" Jul 15 19:26:34.000084 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:34.000045 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-fk99v" event={"ID":"68f7eee7-04d4-49ff-89d7-f10b984a5f55","Type":"ContainerStarted","Data":"a7beae4eb243f4499b548cb656d2c31309918be295700cbc35bad6b8fb8d882b"} Jul 15 19:26:34.000431 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:34.000115 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-fk99v" event={"ID":"68f7eee7-04d4-49ff-89d7-f10b984a5f55","Type":"ContainerStarted","Data":"151164d14cfcde245048ae22b52b491aae3cb126fee5a5ec00d86fa09f45e87c"} Jul 15 19:26:34.001893 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:34.001868 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4strs" event={"ID":"7ffe651c-2927-43a4-96f1-f7bf31b7f598","Type":"ContainerStarted","Data":"c0904521a3c491c2346cad06a4ab5644fcc8020a616dd098962fdd464cf03e9e"} Jul 15 19:26:34.073566 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:34.073520 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4strs" podStartSLOduration=2.5526409230000002 podStartE2EDuration="4.073504171s" totalImagesPullingTime="1.520863248s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:26:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:26:34.070856468 +0000 UTC m=+307.647858139" watchObservedRunningTime="2026-07-15 19:26:34.073504171 +0000 UTC m=+307.650505842" Jul 15 19:26:34.158875 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:34.158842 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8698c746d9-ck2m6"] Jul 15 19:26:34.819177 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:34.819144 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-sx7ln"] Jul 15 19:26:34.893455 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:34.893424 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-sx7ln"] Jul 15 19:26:34.893591 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:34.893514 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-sx7ln" Jul 15 19:26:34.901227 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:34.901208 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 15 19:26:34.901922 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:34.901907 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-62vg7\"" Jul 15 19:26:35.007262 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:35.007228 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-fk99v" event={"ID":"68f7eee7-04d4-49ff-89d7-f10b984a5f55","Type":"ContainerStarted","Data":"760563ecad2c38955560b85870a726aa98c2f23319a97a694d03f024f1d69e72"} Jul 15 19:26:35.030364 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:35.030324 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-fk99v" podStartSLOduration=3.403338771 podStartE2EDuration="5.030295447s" totalImagesPullingTime="1.626956676s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:26:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:26:35.029476836 +0000 UTC m=+308.606478507" watchObservedRunningTime="2026-07-15 19:26:35.030295447 +0000 UTC m=+308.607297117" Jul 15 19:26:35.047722 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:35.047696 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7g4vf\" (UniqueName: \"kubernetes.io/projected/83282529-556c-418b-98bb-874278f184b7-kube-api-access-7g4vf\") pod \"monitoring-plugin-5f68df7f58-sx7ln\" (UID: \"83282529-556c-418b-98bb-874278f184b7\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-sx7ln" Jul 15 19:26:35.047817 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:35.047739 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/83282529-556c-418b-98bb-874278f184b7-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-sx7ln\" (UID: \"83282529-556c-418b-98bb-874278f184b7\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-sx7ln" Jul 15 19:26:35.148558 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:35.148497 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7g4vf\" (UniqueName: \"kubernetes.io/projected/83282529-556c-418b-98bb-874278f184b7-kube-api-access-7g4vf\") pod \"monitoring-plugin-5f68df7f58-sx7ln\" (UID: \"83282529-556c-418b-98bb-874278f184b7\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-sx7ln" Jul 15 19:26:35.148558 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:35.148552 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/83282529-556c-418b-98bb-874278f184b7-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-sx7ln\" (UID: \"83282529-556c-418b-98bb-874278f184b7\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-sx7ln" Jul 15 19:26:35.150850 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:35.150821 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/83282529-556c-418b-98bb-874278f184b7-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-sx7ln\" (UID: \"83282529-556c-418b-98bb-874278f184b7\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-sx7ln" Jul 15 19:26:35.157718 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:35.157698 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7g4vf\" (UniqueName: \"kubernetes.io/projected/83282529-556c-418b-98bb-874278f184b7-kube-api-access-7g4vf\") pod \"monitoring-plugin-5f68df7f58-sx7ln\" (UID: \"83282529-556c-418b-98bb-874278f184b7\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-sx7ln" Jul 15 19:26:35.202027 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:35.202005 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-sx7ln" Jul 15 19:26:35.358748 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:35.358716 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-sx7ln"] Jul 15 19:26:35.363115 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:26:35.363083 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83282529_556c_418b_98bb_874278f184b7.slice/crio-6800dd132ceec3f1f480540845c758ff11f5e089b734088b2d580c18e2a9b5a4 WatchSource:0}: Error finding container 6800dd132ceec3f1f480540845c758ff11f5e089b734088b2d580c18e2a9b5a4: Status 404 returned error can't find the container with id 6800dd132ceec3f1f480540845c758ff11f5e089b734088b2d580c18e2a9b5a4 Jul 15 19:26:36.011639 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:36.011605 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-sx7ln" event={"ID":"83282529-556c-418b-98bb-874278f184b7","Type":"ContainerStarted","Data":"6800dd132ceec3f1f480540845c758ff11f5e089b734088b2d580c18e2a9b5a4"} Jul 15 19:26:36.447426 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:36.447381 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6fff6645b4-vc7cd"] Jul 15 19:26:36.467411 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:36.467352 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6fff6645b4-vc7cd"] Jul 15 19:26:36.467579 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:36.467431 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6fff6645b4-vc7cd" Jul 15 19:26:36.475285 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:36.475257 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 15 19:26:36.561054 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:36.560967 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c10de9f-5f4e-425d-b08d-3a33a0f21721-service-ca\") pod \"console-6fff6645b4-vc7cd\" (UID: \"1c10de9f-5f4e-425d-b08d-3a33a0f21721\") " pod="openshift-console/console-6fff6645b4-vc7cd" Jul 15 19:26:36.561054 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:36.561008 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c10de9f-5f4e-425d-b08d-3a33a0f21721-trusted-ca-bundle\") pod \"console-6fff6645b4-vc7cd\" (UID: \"1c10de9f-5f4e-425d-b08d-3a33a0f21721\") " pod="openshift-console/console-6fff6645b4-vc7cd" Jul 15 19:26:36.561054 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:36.561039 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c10de9f-5f4e-425d-b08d-3a33a0f21721-console-serving-cert\") pod \"console-6fff6645b4-vc7cd\" (UID: \"1c10de9f-5f4e-425d-b08d-3a33a0f21721\") " pod="openshift-console/console-6fff6645b4-vc7cd" Jul 15 19:26:36.561481 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:36.561067 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-985x9\" (UniqueName: \"kubernetes.io/projected/1c10de9f-5f4e-425d-b08d-3a33a0f21721-kube-api-access-985x9\") pod \"console-6fff6645b4-vc7cd\" (UID: \"1c10de9f-5f4e-425d-b08d-3a33a0f21721\") " pod="openshift-console/console-6fff6645b4-vc7cd" Jul 15 19:26:36.561481 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:36.561097 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c10de9f-5f4e-425d-b08d-3a33a0f21721-oauth-serving-cert\") pod \"console-6fff6645b4-vc7cd\" (UID: \"1c10de9f-5f4e-425d-b08d-3a33a0f21721\") " pod="openshift-console/console-6fff6645b4-vc7cd" Jul 15 19:26:36.561481 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:36.561120 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c10de9f-5f4e-425d-b08d-3a33a0f21721-console-oauth-config\") pod \"console-6fff6645b4-vc7cd\" (UID: \"1c10de9f-5f4e-425d-b08d-3a33a0f21721\") " pod="openshift-console/console-6fff6645b4-vc7cd" Jul 15 19:26:36.561481 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:36.561212 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c10de9f-5f4e-425d-b08d-3a33a0f21721-console-config\") pod \"console-6fff6645b4-vc7cd\" (UID: \"1c10de9f-5f4e-425d-b08d-3a33a0f21721\") " pod="openshift-console/console-6fff6645b4-vc7cd" Jul 15 19:26:36.662347 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:36.662297 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c10de9f-5f4e-425d-b08d-3a33a0f21721-trusted-ca-bundle\") pod \"console-6fff6645b4-vc7cd\" (UID: \"1c10de9f-5f4e-425d-b08d-3a33a0f21721\") " pod="openshift-console/console-6fff6645b4-vc7cd" Jul 15 19:26:36.662519 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:36.662360 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c10de9f-5f4e-425d-b08d-3a33a0f21721-console-serving-cert\") pod \"console-6fff6645b4-vc7cd\" (UID: \"1c10de9f-5f4e-425d-b08d-3a33a0f21721\") " pod="openshift-console/console-6fff6645b4-vc7cd" Jul 15 19:26:36.662519 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:36.662382 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-985x9\" (UniqueName: \"kubernetes.io/projected/1c10de9f-5f4e-425d-b08d-3a33a0f21721-kube-api-access-985x9\") pod \"console-6fff6645b4-vc7cd\" (UID: \"1c10de9f-5f4e-425d-b08d-3a33a0f21721\") " pod="openshift-console/console-6fff6645b4-vc7cd" Jul 15 19:26:36.662519 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:36.662402 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c10de9f-5f4e-425d-b08d-3a33a0f21721-oauth-serving-cert\") pod \"console-6fff6645b4-vc7cd\" (UID: \"1c10de9f-5f4e-425d-b08d-3a33a0f21721\") " pod="openshift-console/console-6fff6645b4-vc7cd" Jul 15 19:26:36.662519 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:36.662419 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c10de9f-5f4e-425d-b08d-3a33a0f21721-console-oauth-config\") pod \"console-6fff6645b4-vc7cd\" (UID: \"1c10de9f-5f4e-425d-b08d-3a33a0f21721\") " pod="openshift-console/console-6fff6645b4-vc7cd" Jul 15 19:26:36.662519 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:36.662448 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c10de9f-5f4e-425d-b08d-3a33a0f21721-console-config\") pod \"console-6fff6645b4-vc7cd\" (UID: \"1c10de9f-5f4e-425d-b08d-3a33a0f21721\") " pod="openshift-console/console-6fff6645b4-vc7cd" Jul 15 19:26:36.662790 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:36.662528 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c10de9f-5f4e-425d-b08d-3a33a0f21721-service-ca\") pod \"console-6fff6645b4-vc7cd\" (UID: \"1c10de9f-5f4e-425d-b08d-3a33a0f21721\") " pod="openshift-console/console-6fff6645b4-vc7cd" Jul 15 19:26:36.663199 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:36.663152 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c10de9f-5f4e-425d-b08d-3a33a0f21721-service-ca\") pod \"console-6fff6645b4-vc7cd\" (UID: \"1c10de9f-5f4e-425d-b08d-3a33a0f21721\") " pod="openshift-console/console-6fff6645b4-vc7cd" Jul 15 19:26:36.663199 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:36.663165 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c10de9f-5f4e-425d-b08d-3a33a0f21721-oauth-serving-cert\") pod \"console-6fff6645b4-vc7cd\" (UID: \"1c10de9f-5f4e-425d-b08d-3a33a0f21721\") " pod="openshift-console/console-6fff6645b4-vc7cd" Jul 15 19:26:36.663452 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:36.663431 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c10de9f-5f4e-425d-b08d-3a33a0f21721-trusted-ca-bundle\") pod \"console-6fff6645b4-vc7cd\" (UID: \"1c10de9f-5f4e-425d-b08d-3a33a0f21721\") " pod="openshift-console/console-6fff6645b4-vc7cd" Jul 15 19:26:36.663522 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:36.663507 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c10de9f-5f4e-425d-b08d-3a33a0f21721-console-config\") pod \"console-6fff6645b4-vc7cd\" (UID: \"1c10de9f-5f4e-425d-b08d-3a33a0f21721\") " pod="openshift-console/console-6fff6645b4-vc7cd" Jul 15 19:26:36.664815 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:36.664795 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c10de9f-5f4e-425d-b08d-3a33a0f21721-console-oauth-config\") pod \"console-6fff6645b4-vc7cd\" (UID: \"1c10de9f-5f4e-425d-b08d-3a33a0f21721\") " pod="openshift-console/console-6fff6645b4-vc7cd" Jul 15 19:26:36.664903 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:36.664881 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c10de9f-5f4e-425d-b08d-3a33a0f21721-console-serving-cert\") pod \"console-6fff6645b4-vc7cd\" (UID: \"1c10de9f-5f4e-425d-b08d-3a33a0f21721\") " pod="openshift-console/console-6fff6645b4-vc7cd" Jul 15 19:26:36.670967 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:36.670939 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-985x9\" (UniqueName: \"kubernetes.io/projected/1c10de9f-5f4e-425d-b08d-3a33a0f21721-kube-api-access-985x9\") pod \"console-6fff6645b4-vc7cd\" (UID: \"1c10de9f-5f4e-425d-b08d-3a33a0f21721\") " pod="openshift-console/console-6fff6645b4-vc7cd" Jul 15 19:26:36.780391 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:36.780279 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6fff6645b4-vc7cd" Jul 15 19:26:37.295561 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:37.295536 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6fff6645b4-vc7cd"] Jul 15 19:26:37.325617 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:26:37.325586 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c10de9f_5f4e_425d_b08d_3a33a0f21721.slice/crio-164793059ff13e9b83b6dd12416bc07186afc2e6c3d82acf62a80fd78c7db2ff WatchSource:0}: Error finding container 164793059ff13e9b83b6dd12416bc07186afc2e6c3d82acf62a80fd78c7db2ff: Status 404 returned error can't find the container with id 164793059ff13e9b83b6dd12416bc07186afc2e6c3d82acf62a80fd78c7db2ff Jul 15 19:26:38.021497 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:38.021459 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-sx7ln" event={"ID":"83282529-556c-418b-98bb-874278f184b7","Type":"ContainerStarted","Data":"75b6c59be5c5b5060cc555391da61ac599f7499300f522af2d66f7a158ee19c2"} Jul 15 19:26:38.021754 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:38.021659 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-sx7ln" Jul 15 19:26:38.028082 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:38.028032 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6fff6645b4-vc7cd" event={"ID":"1c10de9f-5f4e-425d-b08d-3a33a0f21721","Type":"ContainerStarted","Data":"142acd72c9d9ce70b4e4784eaff2122390cbafe82c8a6c1df264bd6601448d5d"} Jul 15 19:26:38.028601 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:38.028567 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6fff6645b4-vc7cd" event={"ID":"1c10de9f-5f4e-425d-b08d-3a33a0f21721","Type":"ContainerStarted","Data":"164793059ff13e9b83b6dd12416bc07186afc2e6c3d82acf62a80fd78c7db2ff"} Jul 15 19:26:38.028727 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:38.028692 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-sx7ln" Jul 15 19:26:38.041237 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:38.041199 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-sx7ln" podStartSLOduration=2.064149719 podStartE2EDuration="4.041186584s" totalImagesPullingTime="1.977036865s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:26:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:26:38.040192197 +0000 UTC m=+311.617193871" watchObservedRunningTime="2026-07-15 19:26:38.041186584 +0000 UTC m=+311.618188295" Jul 15 19:26:38.080771 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:38.080725 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6fff6645b4-vc7cd" podStartSLOduration=2.080711381 podStartE2EDuration="2.080711381s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:26:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:26:38.079429208 +0000 UTC m=+311.656430889" watchObservedRunningTime="2026-07-15 19:26:38.080711381 +0000 UTC m=+311.657713052" Jul 15 19:26:39.063105 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:39.063077 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-9db745d49-xxf8k" Jul 15 19:26:42.084363 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:42.084332 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6fff6645b4-vc7cd"] Jul 15 19:26:42.143297 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:42.143265 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7bfbfd54-h6t58"] Jul 15 19:26:42.157718 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:42.157686 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bfbfd54-h6t58" Jul 15 19:26:42.179765 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:42.179742 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7bfbfd54-h6t58"] Jul 15 19:26:42.309988 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:42.309955 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6d9b88c-1112-4f0e-878a-afdc7cd36495-trusted-ca-bundle\") pod \"console-7bfbfd54-h6t58\" (UID: \"d6d9b88c-1112-4f0e-878a-afdc7cd36495\") " pod="openshift-console/console-7bfbfd54-h6t58" Jul 15 19:26:42.309988 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:42.309989 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d6d9b88c-1112-4f0e-878a-afdc7cd36495-oauth-serving-cert\") pod \"console-7bfbfd54-h6t58\" (UID: \"d6d9b88c-1112-4f0e-878a-afdc7cd36495\") " pod="openshift-console/console-7bfbfd54-h6t58" Jul 15 19:26:42.310187 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:42.310041 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d6d9b88c-1112-4f0e-878a-afdc7cd36495-service-ca\") pod \"console-7bfbfd54-h6t58\" (UID: \"d6d9b88c-1112-4f0e-878a-afdc7cd36495\") " pod="openshift-console/console-7bfbfd54-h6t58" Jul 15 19:26:42.310187 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:42.310065 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d6d9b88c-1112-4f0e-878a-afdc7cd36495-console-serving-cert\") pod \"console-7bfbfd54-h6t58\" (UID: \"d6d9b88c-1112-4f0e-878a-afdc7cd36495\") " pod="openshift-console/console-7bfbfd54-h6t58" Jul 15 19:26:42.310187 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:42.310080 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d6d9b88c-1112-4f0e-878a-afdc7cd36495-console-config\") pod \"console-7bfbfd54-h6t58\" (UID: \"d6d9b88c-1112-4f0e-878a-afdc7cd36495\") " pod="openshift-console/console-7bfbfd54-h6t58" Jul 15 19:26:42.310187 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:42.310097 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sp77m\" (UniqueName: \"kubernetes.io/projected/d6d9b88c-1112-4f0e-878a-afdc7cd36495-kube-api-access-sp77m\") pod \"console-7bfbfd54-h6t58\" (UID: \"d6d9b88c-1112-4f0e-878a-afdc7cd36495\") " pod="openshift-console/console-7bfbfd54-h6t58" Jul 15 19:26:42.310187 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:42.310171 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d6d9b88c-1112-4f0e-878a-afdc7cd36495-console-oauth-config\") pod \"console-7bfbfd54-h6t58\" (UID: \"d6d9b88c-1112-4f0e-878a-afdc7cd36495\") " pod="openshift-console/console-7bfbfd54-h6t58" Jul 15 19:26:42.410860 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:42.410795 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d6d9b88c-1112-4f0e-878a-afdc7cd36495-console-serving-cert\") pod \"console-7bfbfd54-h6t58\" (UID: \"d6d9b88c-1112-4f0e-878a-afdc7cd36495\") " pod="openshift-console/console-7bfbfd54-h6t58" Jul 15 19:26:42.410860 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:42.410828 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d6d9b88c-1112-4f0e-878a-afdc7cd36495-console-config\") pod \"console-7bfbfd54-h6t58\" (UID: \"d6d9b88c-1112-4f0e-878a-afdc7cd36495\") " pod="openshift-console/console-7bfbfd54-h6t58" Jul 15 19:26:42.410860 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:42.410849 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sp77m\" (UniqueName: \"kubernetes.io/projected/d6d9b88c-1112-4f0e-878a-afdc7cd36495-kube-api-access-sp77m\") pod \"console-7bfbfd54-h6t58\" (UID: \"d6d9b88c-1112-4f0e-878a-afdc7cd36495\") " pod="openshift-console/console-7bfbfd54-h6t58" Jul 15 19:26:42.411112 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:42.410867 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d6d9b88c-1112-4f0e-878a-afdc7cd36495-console-oauth-config\") pod \"console-7bfbfd54-h6t58\" (UID: \"d6d9b88c-1112-4f0e-878a-afdc7cd36495\") " pod="openshift-console/console-7bfbfd54-h6t58" Jul 15 19:26:42.411112 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:42.411000 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6d9b88c-1112-4f0e-878a-afdc7cd36495-trusted-ca-bundle\") pod \"console-7bfbfd54-h6t58\" (UID: \"d6d9b88c-1112-4f0e-878a-afdc7cd36495\") " pod="openshift-console/console-7bfbfd54-h6t58" Jul 15 19:26:42.411112 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:42.411040 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d6d9b88c-1112-4f0e-878a-afdc7cd36495-oauth-serving-cert\") pod \"console-7bfbfd54-h6t58\" (UID: \"d6d9b88c-1112-4f0e-878a-afdc7cd36495\") " pod="openshift-console/console-7bfbfd54-h6t58" Jul 15 19:26:42.411272 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:42.411144 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d6d9b88c-1112-4f0e-878a-afdc7cd36495-service-ca\") pod \"console-7bfbfd54-h6t58\" (UID: \"d6d9b88c-1112-4f0e-878a-afdc7cd36495\") " pod="openshift-console/console-7bfbfd54-h6t58" Jul 15 19:26:42.411650 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:42.411622 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d6d9b88c-1112-4f0e-878a-afdc7cd36495-console-config\") pod \"console-7bfbfd54-h6t58\" (UID: \"d6d9b88c-1112-4f0e-878a-afdc7cd36495\") " pod="openshift-console/console-7bfbfd54-h6t58" Jul 15 19:26:42.411766 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:42.411726 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d6d9b88c-1112-4f0e-878a-afdc7cd36495-oauth-serving-cert\") pod \"console-7bfbfd54-h6t58\" (UID: \"d6d9b88c-1112-4f0e-878a-afdc7cd36495\") " pod="openshift-console/console-7bfbfd54-h6t58" Jul 15 19:26:42.411827 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:42.411806 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6d9b88c-1112-4f0e-878a-afdc7cd36495-trusted-ca-bundle\") pod \"console-7bfbfd54-h6t58\" (UID: \"d6d9b88c-1112-4f0e-878a-afdc7cd36495\") " pod="openshift-console/console-7bfbfd54-h6t58" Jul 15 19:26:42.411883 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:42.411870 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d6d9b88c-1112-4f0e-878a-afdc7cd36495-service-ca\") pod \"console-7bfbfd54-h6t58\" (UID: \"d6d9b88c-1112-4f0e-878a-afdc7cd36495\") " pod="openshift-console/console-7bfbfd54-h6t58" Jul 15 19:26:42.413281 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:42.413255 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d6d9b88c-1112-4f0e-878a-afdc7cd36495-console-serving-cert\") pod \"console-7bfbfd54-h6t58\" (UID: \"d6d9b88c-1112-4f0e-878a-afdc7cd36495\") " pod="openshift-console/console-7bfbfd54-h6t58" Jul 15 19:26:42.413438 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:42.413421 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d6d9b88c-1112-4f0e-878a-afdc7cd36495-console-oauth-config\") pod \"console-7bfbfd54-h6t58\" (UID: \"d6d9b88c-1112-4f0e-878a-afdc7cd36495\") " pod="openshift-console/console-7bfbfd54-h6t58" Jul 15 19:26:42.423548 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:42.423528 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sp77m\" (UniqueName: \"kubernetes.io/projected/d6d9b88c-1112-4f0e-878a-afdc7cd36495-kube-api-access-sp77m\") pod \"console-7bfbfd54-h6t58\" (UID: \"d6d9b88c-1112-4f0e-878a-afdc7cd36495\") " pod="openshift-console/console-7bfbfd54-h6t58" Jul 15 19:26:42.467658 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:42.467629 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bfbfd54-h6t58" Jul 15 19:26:42.598202 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:42.598085 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7bfbfd54-h6t58"] Jul 15 19:26:42.600837 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:26:42.600806 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6d9b88c_1112_4f0e_878a_afdc7cd36495.slice/crio-5e2b88fc8849ae510fe10a2462a4ff8b237a4e3b7ff4e8ae092ca3621403e03e WatchSource:0}: Error finding container 5e2b88fc8849ae510fe10a2462a4ff8b237a4e3b7ff4e8ae092ca3621403e03e: Status 404 returned error can't find the container with id 5e2b88fc8849ae510fe10a2462a4ff8b237a4e3b7ff4e8ae092ca3621403e03e Jul 15 19:26:43.048758 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:43.048727 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bfbfd54-h6t58" event={"ID":"d6d9b88c-1112-4f0e-878a-afdc7cd36495","Type":"ContainerStarted","Data":"367745c73862c9c1b6419fa29bd2eedf9c5546786f5d64fb8e7854133b79c9d6"} Jul 15 19:26:43.049044 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:43.048775 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bfbfd54-h6t58" event={"ID":"d6d9b88c-1112-4f0e-878a-afdc7cd36495","Type":"ContainerStarted","Data":"5e2b88fc8849ae510fe10a2462a4ff8b237a4e3b7ff4e8ae092ca3621403e03e"} Jul 15 19:26:43.075698 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:43.075648 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7bfbfd54-h6t58" podStartSLOduration=1.075633588 podStartE2EDuration="1.075633588s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:26:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:26:43.075210221 +0000 UTC m=+316.652211893" watchObservedRunningTime="2026-07-15 19:26:43.075633588 +0000 UTC m=+316.652635259" Jul 15 19:26:46.780573 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:46.780538 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6fff6645b4-vc7cd" Jul 15 19:26:52.468255 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:52.468225 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7bfbfd54-h6t58" Jul 15 19:26:52.468619 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:52.468274 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7bfbfd54-h6t58" Jul 15 19:26:52.472979 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:52.472951 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7bfbfd54-h6t58" Jul 15 19:26:53.091240 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:53.091204 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7bfbfd54-h6t58" Jul 15 19:26:54.077887 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:54.077851 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-9db745d49-xxf8k" podUID="1235737d-3c8a-4c3a-af1a-dfd3351d72fb" containerName="registry" containerID="cri-o://a22cd7e977e85aeed975b76c75be8053de3517f8ed9b62018b4e97820d745d3a" gracePeriod=30 Jul 15 19:26:54.336674 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:54.336620 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9db745d49-xxf8k" Jul 15 19:26:54.508853 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:54.508825 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-installation-pull-secrets\") pod \"1235737d-3c8a-4c3a-af1a-dfd3351d72fb\" (UID: \"1235737d-3c8a-4c3a-af1a-dfd3351d72fb\") " Jul 15 19:26:54.509010 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:54.508909 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-registry-certificates\") pod \"1235737d-3c8a-4c3a-af1a-dfd3351d72fb\" (UID: \"1235737d-3c8a-4c3a-af1a-dfd3351d72fb\") " Jul 15 19:26:54.509010 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:54.508942 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-ca-trust-extracted\") pod \"1235737d-3c8a-4c3a-af1a-dfd3351d72fb\" (UID: \"1235737d-3c8a-4c3a-af1a-dfd3351d72fb\") " Jul 15 19:26:54.509010 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:54.508967 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-registry-tls\") pod \"1235737d-3c8a-4c3a-af1a-dfd3351d72fb\" (UID: \"1235737d-3c8a-4c3a-af1a-dfd3351d72fb\") " Jul 15 19:26:54.509010 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:54.508993 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-image-registry-private-configuration\") pod \"1235737d-3c8a-4c3a-af1a-dfd3351d72fb\" (UID: \"1235737d-3c8a-4c3a-af1a-dfd3351d72fb\") " Jul 15 19:26:54.509215 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:54.509043 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-trusted-ca\") pod \"1235737d-3c8a-4c3a-af1a-dfd3351d72fb\" (UID: \"1235737d-3c8a-4c3a-af1a-dfd3351d72fb\") " Jul 15 19:26:54.509215 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:54.509066 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-bound-sa-token\") pod \"1235737d-3c8a-4c3a-af1a-dfd3351d72fb\" (UID: \"1235737d-3c8a-4c3a-af1a-dfd3351d72fb\") " Jul 15 19:26:54.509215 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:54.509093 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kb7v2\" (UniqueName: \"kubernetes.io/projected/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-kube-api-access-kb7v2\") pod \"1235737d-3c8a-4c3a-af1a-dfd3351d72fb\" (UID: \"1235737d-3c8a-4c3a-af1a-dfd3351d72fb\") " Jul 15 19:26:54.509386 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:54.509348 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "1235737d-3c8a-4c3a-af1a-dfd3351d72fb" (UID: "1235737d-3c8a-4c3a-af1a-dfd3351d72fb"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:26:54.509446 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:54.509418 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "1235737d-3c8a-4c3a-af1a-dfd3351d72fb" (UID: "1235737d-3c8a-4c3a-af1a-dfd3351d72fb"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:26:54.511657 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:54.511603 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "1235737d-3c8a-4c3a-af1a-dfd3351d72fb" (UID: "1235737d-3c8a-4c3a-af1a-dfd3351d72fb"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:26:54.511657 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:54.511620 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "1235737d-3c8a-4c3a-af1a-dfd3351d72fb" (UID: "1235737d-3c8a-4c3a-af1a-dfd3351d72fb"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:26:54.511838 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:54.511770 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "1235737d-3c8a-4c3a-af1a-dfd3351d72fb" (UID: "1235737d-3c8a-4c3a-af1a-dfd3351d72fb"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:26:54.511908 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:54.511886 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "1235737d-3c8a-4c3a-af1a-dfd3351d72fb" (UID: "1235737d-3c8a-4c3a-af1a-dfd3351d72fb"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:26:54.511965 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:54.511931 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-kube-api-access-kb7v2" (OuterVolumeSpecName: "kube-api-access-kb7v2") pod "1235737d-3c8a-4c3a-af1a-dfd3351d72fb" (UID: "1235737d-3c8a-4c3a-af1a-dfd3351d72fb"). InnerVolumeSpecName "kube-api-access-kb7v2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:26:54.518296 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:54.518274 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "1235737d-3c8a-4c3a-af1a-dfd3351d72fb" (UID: "1235737d-3c8a-4c3a-af1a-dfd3351d72fb"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:26:54.610070 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:54.610013 2577 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-bound-sa-token\") on node \"ip-10-0-141-62.ec2.internal\" DevicePath \"\"" Jul 15 19:26:54.610070 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:54.610034 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kb7v2\" (UniqueName: \"kubernetes.io/projected/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-kube-api-access-kb7v2\") on node \"ip-10-0-141-62.ec2.internal\" DevicePath \"\"" Jul 15 19:26:54.610070 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:54.610043 2577 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-installation-pull-secrets\") on node \"ip-10-0-141-62.ec2.internal\" DevicePath \"\"" Jul 15 19:26:54.610070 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:54.610052 2577 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-registry-certificates\") on node \"ip-10-0-141-62.ec2.internal\" DevicePath \"\"" Jul 15 19:26:54.610070 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:54.610063 2577 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-ca-trust-extracted\") on node \"ip-10-0-141-62.ec2.internal\" DevicePath \"\"" Jul 15 19:26:54.610070 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:54.610071 2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-registry-tls\") on node \"ip-10-0-141-62.ec2.internal\" DevicePath \"\"" Jul 15 19:26:54.610344 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:54.610079 2577 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-image-registry-private-configuration\") on node \"ip-10-0-141-62.ec2.internal\" DevicePath \"\"" Jul 15 19:26:54.610344 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:54.610088 2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1235737d-3c8a-4c3a-af1a-dfd3351d72fb-trusted-ca\") on node \"ip-10-0-141-62.ec2.internal\" DevicePath \"\"" Jul 15 19:26:55.092194 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:55.092161 2577 generic.go:356] "Generic (PLEG): container finished" podID="1235737d-3c8a-4c3a-af1a-dfd3351d72fb" containerID="a22cd7e977e85aeed975b76c75be8053de3517f8ed9b62018b4e97820d745d3a" exitCode=0 Jul 15 19:26:55.092687 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:55.092200 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9db745d49-xxf8k" Jul 15 19:26:55.092687 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:55.092241 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9db745d49-xxf8k" event={"ID":"1235737d-3c8a-4c3a-af1a-dfd3351d72fb","Type":"ContainerDied","Data":"a22cd7e977e85aeed975b76c75be8053de3517f8ed9b62018b4e97820d745d3a"} Jul 15 19:26:55.092687 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:55.092282 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9db745d49-xxf8k" event={"ID":"1235737d-3c8a-4c3a-af1a-dfd3351d72fb","Type":"ContainerDied","Data":"41d2602bf9f526f3a000234b405e0512141d67dfd71e557645ebd075039fdd7c"} Jul 15 19:26:55.092687 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:55.092302 2577 scope.go:117] "RemoveContainer" containerID="a22cd7e977e85aeed975b76c75be8053de3517f8ed9b62018b4e97820d745d3a" Jul 15 19:26:55.101057 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:55.101040 2577 scope.go:117] "RemoveContainer" containerID="a22cd7e977e85aeed975b76c75be8053de3517f8ed9b62018b4e97820d745d3a" Jul 15 19:26:55.101328 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:26:55.101292 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a22cd7e977e85aeed975b76c75be8053de3517f8ed9b62018b4e97820d745d3a\": container with ID starting with a22cd7e977e85aeed975b76c75be8053de3517f8ed9b62018b4e97820d745d3a not found: ID does not exist" containerID="a22cd7e977e85aeed975b76c75be8053de3517f8ed9b62018b4e97820d745d3a" Jul 15 19:26:55.101401 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:55.101336 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a22cd7e977e85aeed975b76c75be8053de3517f8ed9b62018b4e97820d745d3a"} err="failed to get container status \"a22cd7e977e85aeed975b76c75be8053de3517f8ed9b62018b4e97820d745d3a\": rpc error: code = NotFound desc = could not find container \"a22cd7e977e85aeed975b76c75be8053de3517f8ed9b62018b4e97820d745d3a\": container with ID starting with a22cd7e977e85aeed975b76c75be8053de3517f8ed9b62018b4e97820d745d3a not found: ID does not exist" Jul 15 19:26:55.115731 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:55.115710 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-9db745d49-xxf8k"] Jul 15 19:26:55.120551 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:55.120532 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-9db745d49-xxf8k"] Jul 15 19:26:57.091940 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:57.091911 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1235737d-3c8a-4c3a-af1a-dfd3351d72fb" path="/var/lib/kubelet/pods/1235737d-3c8a-4c3a-af1a-dfd3351d72fb/volumes" Jul 15 19:26:59.181039 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:59.181005 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-8698c746d9-ck2m6" podUID="cb861bd3-b9cf-4671-bc14-15e29bf00106" containerName="console" containerID="cri-o://95ede16f9e0522ace911d0385c12e9a7234a70f3a7b2e27facb178c370a55896" gracePeriod=15 Jul 15 19:26:59.419584 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:59.419564 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8698c746d9-ck2m6" Jul 15 19:26:59.545080 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:59.545020 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cb861bd3-b9cf-4671-bc14-15e29bf00106-console-serving-cert\") pod \"cb861bd3-b9cf-4671-bc14-15e29bf00106\" (UID: \"cb861bd3-b9cf-4671-bc14-15e29bf00106\") " Jul 15 19:26:59.545080 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:59.545064 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cb861bd3-b9cf-4671-bc14-15e29bf00106-service-ca\") pod \"cb861bd3-b9cf-4671-bc14-15e29bf00106\" (UID: \"cb861bd3-b9cf-4671-bc14-15e29bf00106\") " Jul 15 19:26:59.545244 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:59.545096 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pxs5k\" (UniqueName: \"kubernetes.io/projected/cb861bd3-b9cf-4671-bc14-15e29bf00106-kube-api-access-pxs5k\") pod \"cb861bd3-b9cf-4671-bc14-15e29bf00106\" (UID: \"cb861bd3-b9cf-4671-bc14-15e29bf00106\") " Jul 15 19:26:59.545244 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:59.545142 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cb861bd3-b9cf-4671-bc14-15e29bf00106-console-oauth-config\") pod \"cb861bd3-b9cf-4671-bc14-15e29bf00106\" (UID: \"cb861bd3-b9cf-4671-bc14-15e29bf00106\") " Jul 15 19:26:59.545244 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:59.545167 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cb861bd3-b9cf-4671-bc14-15e29bf00106-console-config\") pod \"cb861bd3-b9cf-4671-bc14-15e29bf00106\" (UID: \"cb861bd3-b9cf-4671-bc14-15e29bf00106\") " Jul 15 19:26:59.545244 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:59.545242 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cb861bd3-b9cf-4671-bc14-15e29bf00106-oauth-serving-cert\") pod \"cb861bd3-b9cf-4671-bc14-15e29bf00106\" (UID: \"cb861bd3-b9cf-4671-bc14-15e29bf00106\") " Jul 15 19:26:59.545489 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:59.545460 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb861bd3-b9cf-4671-bc14-15e29bf00106-service-ca" (OuterVolumeSpecName: "service-ca") pod "cb861bd3-b9cf-4671-bc14-15e29bf00106" (UID: "cb861bd3-b9cf-4671-bc14-15e29bf00106"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:26:59.545622 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:59.545591 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb861bd3-b9cf-4671-bc14-15e29bf00106-console-config" (OuterVolumeSpecName: "console-config") pod "cb861bd3-b9cf-4671-bc14-15e29bf00106" (UID: "cb861bd3-b9cf-4671-bc14-15e29bf00106"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:26:59.545720 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:59.545640 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb861bd3-b9cf-4671-bc14-15e29bf00106-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "cb861bd3-b9cf-4671-bc14-15e29bf00106" (UID: "cb861bd3-b9cf-4671-bc14-15e29bf00106"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:26:59.547296 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:59.547268 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb861bd3-b9cf-4671-bc14-15e29bf00106-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "cb861bd3-b9cf-4671-bc14-15e29bf00106" (UID: "cb861bd3-b9cf-4671-bc14-15e29bf00106"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:26:59.547517 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:59.547501 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb861bd3-b9cf-4671-bc14-15e29bf00106-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "cb861bd3-b9cf-4671-bc14-15e29bf00106" (UID: "cb861bd3-b9cf-4671-bc14-15e29bf00106"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:26:59.547636 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:59.547617 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb861bd3-b9cf-4671-bc14-15e29bf00106-kube-api-access-pxs5k" (OuterVolumeSpecName: "kube-api-access-pxs5k") pod "cb861bd3-b9cf-4671-bc14-15e29bf00106" (UID: "cb861bd3-b9cf-4671-bc14-15e29bf00106"). InnerVolumeSpecName "kube-api-access-pxs5k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:26:59.646495 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:59.646471 2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cb861bd3-b9cf-4671-bc14-15e29bf00106-service-ca\") on node \"ip-10-0-141-62.ec2.internal\" DevicePath \"\"" Jul 15 19:26:59.646495 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:59.646490 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pxs5k\" (UniqueName: \"kubernetes.io/projected/cb861bd3-b9cf-4671-bc14-15e29bf00106-kube-api-access-pxs5k\") on node \"ip-10-0-141-62.ec2.internal\" DevicePath \"\"" Jul 15 19:26:59.646608 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:59.646499 2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cb861bd3-b9cf-4671-bc14-15e29bf00106-console-oauth-config\") on node \"ip-10-0-141-62.ec2.internal\" DevicePath \"\"" Jul 15 19:26:59.646608 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:59.646509 2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cb861bd3-b9cf-4671-bc14-15e29bf00106-console-config\") on node \"ip-10-0-141-62.ec2.internal\" DevicePath \"\"" Jul 15 19:26:59.646608 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:59.646517 2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cb861bd3-b9cf-4671-bc14-15e29bf00106-oauth-serving-cert\") on node \"ip-10-0-141-62.ec2.internal\" DevicePath \"\"" Jul 15 19:26:59.646608 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:26:59.646525 2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cb861bd3-b9cf-4671-bc14-15e29bf00106-console-serving-cert\") on node \"ip-10-0-141-62.ec2.internal\" DevicePath \"\"" Jul 15 19:27:00.116448 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:00.116416 2577 generic.go:356] "Generic (PLEG): container finished" podID="cb861bd3-b9cf-4671-bc14-15e29bf00106" containerID="95ede16f9e0522ace911d0385c12e9a7234a70f3a7b2e27facb178c370a55896" exitCode=0 Jul 15 19:27:00.116621 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:00.116451 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8698c746d9-ck2m6" event={"ID":"cb861bd3-b9cf-4671-bc14-15e29bf00106","Type":"ContainerDied","Data":"95ede16f9e0522ace911d0385c12e9a7234a70f3a7b2e27facb178c370a55896"} Jul 15 19:27:00.116621 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:00.116482 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8698c746d9-ck2m6" event={"ID":"cb861bd3-b9cf-4671-bc14-15e29bf00106","Type":"ContainerDied","Data":"96e71dff144109669a4e5a133f2dbe977ea9791bbf8e0bbd8ba41e2e3fbb08d6"} Jul 15 19:27:00.116621 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:00.116454 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8698c746d9-ck2m6" Jul 15 19:27:00.116621 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:00.116500 2577 scope.go:117] "RemoveContainer" containerID="95ede16f9e0522ace911d0385c12e9a7234a70f3a7b2e27facb178c370a55896" Jul 15 19:27:00.125142 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:00.125123 2577 scope.go:117] "RemoveContainer" containerID="95ede16f9e0522ace911d0385c12e9a7234a70f3a7b2e27facb178c370a55896" Jul 15 19:27:00.125400 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:27:00.125382 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95ede16f9e0522ace911d0385c12e9a7234a70f3a7b2e27facb178c370a55896\": container with ID starting with 95ede16f9e0522ace911d0385c12e9a7234a70f3a7b2e27facb178c370a55896 not found: ID does not exist" containerID="95ede16f9e0522ace911d0385c12e9a7234a70f3a7b2e27facb178c370a55896" Jul 15 19:27:00.125445 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:00.125408 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95ede16f9e0522ace911d0385c12e9a7234a70f3a7b2e27facb178c370a55896"} err="failed to get container status \"95ede16f9e0522ace911d0385c12e9a7234a70f3a7b2e27facb178c370a55896\": rpc error: code = NotFound desc = could not find container \"95ede16f9e0522ace911d0385c12e9a7234a70f3a7b2e27facb178c370a55896\": container with ID starting with 95ede16f9e0522ace911d0385c12e9a7234a70f3a7b2e27facb178c370a55896 not found: ID does not exist" Jul 15 19:27:00.150477 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:00.150455 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8698c746d9-ck2m6"] Jul 15 19:27:00.160000 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:00.159980 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8698c746d9-ck2m6"] Jul 15 19:27:01.091042 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:01.091012 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb861bd3-b9cf-4671-bc14-15e29bf00106" path="/var/lib/kubelet/pods/cb861bd3-b9cf-4671-bc14-15e29bf00106/volumes" Jul 15 19:27:07.109834 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:07.109798 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6fff6645b4-vc7cd" podUID="1c10de9f-5f4e-425d-b08d-3a33a0f21721" containerName="console" containerID="cri-o://142acd72c9d9ce70b4e4784eaff2122390cbafe82c8a6c1df264bd6601448d5d" gracePeriod=15 Jul 15 19:27:07.350087 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:07.350065 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6fff6645b4-vc7cd" Jul 15 19:27:07.404035 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:07.403972 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c10de9f-5f4e-425d-b08d-3a33a0f21721-console-config\") pod \"1c10de9f-5f4e-425d-b08d-3a33a0f21721\" (UID: \"1c10de9f-5f4e-425d-b08d-3a33a0f21721\") " Jul 15 19:27:07.404035 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:07.404015 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c10de9f-5f4e-425d-b08d-3a33a0f21721-service-ca\") pod \"1c10de9f-5f4e-425d-b08d-3a33a0f21721\" (UID: \"1c10de9f-5f4e-425d-b08d-3a33a0f21721\") " Jul 15 19:27:07.404188 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:07.404044 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c10de9f-5f4e-425d-b08d-3a33a0f21721-trusted-ca-bundle\") pod \"1c10de9f-5f4e-425d-b08d-3a33a0f21721\" (UID: \"1c10de9f-5f4e-425d-b08d-3a33a0f21721\") " Jul 15 19:27:07.404188 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:07.404068 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c10de9f-5f4e-425d-b08d-3a33a0f21721-console-serving-cert\") pod \"1c10de9f-5f4e-425d-b08d-3a33a0f21721\" (UID: \"1c10de9f-5f4e-425d-b08d-3a33a0f21721\") " Jul 15 19:27:07.404188 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:07.404104 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-985x9\" (UniqueName: \"kubernetes.io/projected/1c10de9f-5f4e-425d-b08d-3a33a0f21721-kube-api-access-985x9\") pod \"1c10de9f-5f4e-425d-b08d-3a33a0f21721\" (UID: \"1c10de9f-5f4e-425d-b08d-3a33a0f21721\") " Jul 15 19:27:07.404188 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:07.404130 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c10de9f-5f4e-425d-b08d-3a33a0f21721-oauth-serving-cert\") pod \"1c10de9f-5f4e-425d-b08d-3a33a0f21721\" (UID: \"1c10de9f-5f4e-425d-b08d-3a33a0f21721\") " Jul 15 19:27:07.404188 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:07.404159 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c10de9f-5f4e-425d-b08d-3a33a0f21721-console-oauth-config\") pod \"1c10de9f-5f4e-425d-b08d-3a33a0f21721\" (UID: \"1c10de9f-5f4e-425d-b08d-3a33a0f21721\") " Jul 15 19:27:07.404440 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:07.404419 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c10de9f-5f4e-425d-b08d-3a33a0f21721-console-config" (OuterVolumeSpecName: "console-config") pod "1c10de9f-5f4e-425d-b08d-3a33a0f21721" (UID: "1c10de9f-5f4e-425d-b08d-3a33a0f21721"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:27:07.404508 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:07.404483 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c10de9f-5f4e-425d-b08d-3a33a0f21721-service-ca" (OuterVolumeSpecName: "service-ca") pod "1c10de9f-5f4e-425d-b08d-3a33a0f21721" (UID: "1c10de9f-5f4e-425d-b08d-3a33a0f21721"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:27:07.404654 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:07.404629 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c10de9f-5f4e-425d-b08d-3a33a0f21721-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1c10de9f-5f4e-425d-b08d-3a33a0f21721" (UID: "1c10de9f-5f4e-425d-b08d-3a33a0f21721"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:27:07.404762 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:07.404724 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c10de9f-5f4e-425d-b08d-3a33a0f21721-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1c10de9f-5f4e-425d-b08d-3a33a0f21721" (UID: "1c10de9f-5f4e-425d-b08d-3a33a0f21721"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:27:07.406281 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:07.406258 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c10de9f-5f4e-425d-b08d-3a33a0f21721-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1c10de9f-5f4e-425d-b08d-3a33a0f21721" (UID: "1c10de9f-5f4e-425d-b08d-3a33a0f21721"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:27:07.406378 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:07.406286 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c10de9f-5f4e-425d-b08d-3a33a0f21721-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1c10de9f-5f4e-425d-b08d-3a33a0f21721" (UID: "1c10de9f-5f4e-425d-b08d-3a33a0f21721"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:27:07.406378 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:07.406349 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c10de9f-5f4e-425d-b08d-3a33a0f21721-kube-api-access-985x9" (OuterVolumeSpecName: "kube-api-access-985x9") pod "1c10de9f-5f4e-425d-b08d-3a33a0f21721" (UID: "1c10de9f-5f4e-425d-b08d-3a33a0f21721"). InnerVolumeSpecName "kube-api-access-985x9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:27:07.504859 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:07.504834 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-985x9\" (UniqueName: \"kubernetes.io/projected/1c10de9f-5f4e-425d-b08d-3a33a0f21721-kube-api-access-985x9\") on node \"ip-10-0-141-62.ec2.internal\" DevicePath \"\"" Jul 15 19:27:07.504859 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:07.504857 2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c10de9f-5f4e-425d-b08d-3a33a0f21721-oauth-serving-cert\") on node \"ip-10-0-141-62.ec2.internal\" DevicePath \"\"" Jul 15 19:27:07.504982 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:07.504866 2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c10de9f-5f4e-425d-b08d-3a33a0f21721-console-oauth-config\") on node \"ip-10-0-141-62.ec2.internal\" DevicePath \"\"" Jul 15 19:27:07.504982 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:07.504876 2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c10de9f-5f4e-425d-b08d-3a33a0f21721-console-config\") on node \"ip-10-0-141-62.ec2.internal\" DevicePath \"\"" Jul 15 19:27:07.504982 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:07.504884 2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c10de9f-5f4e-425d-b08d-3a33a0f21721-service-ca\") on node \"ip-10-0-141-62.ec2.internal\" DevicePath \"\"" Jul 15 19:27:07.504982 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:07.504891 2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c10de9f-5f4e-425d-b08d-3a33a0f21721-trusted-ca-bundle\") on node \"ip-10-0-141-62.ec2.internal\" DevicePath \"\"" Jul 15 19:27:07.504982 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:07.504900 2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c10de9f-5f4e-425d-b08d-3a33a0f21721-console-serving-cert\") on node \"ip-10-0-141-62.ec2.internal\" DevicePath \"\"" Jul 15 19:27:08.152737 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:08.152701 2577 generic.go:356] "Generic (PLEG): container finished" podID="1c10de9f-5f4e-425d-b08d-3a33a0f21721" containerID="142acd72c9d9ce70b4e4784eaff2122390cbafe82c8a6c1df264bd6601448d5d" exitCode=0 Jul 15 19:27:08.153171 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:08.152758 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6fff6645b4-vc7cd" Jul 15 19:27:08.153171 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:08.152789 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6fff6645b4-vc7cd" event={"ID":"1c10de9f-5f4e-425d-b08d-3a33a0f21721","Type":"ContainerDied","Data":"142acd72c9d9ce70b4e4784eaff2122390cbafe82c8a6c1df264bd6601448d5d"} Jul 15 19:27:08.153171 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:08.152846 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6fff6645b4-vc7cd" event={"ID":"1c10de9f-5f4e-425d-b08d-3a33a0f21721","Type":"ContainerDied","Data":"164793059ff13e9b83b6dd12416bc07186afc2e6c3d82acf62a80fd78c7db2ff"} Jul 15 19:27:08.153171 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:08.152876 2577 scope.go:117] "RemoveContainer" containerID="142acd72c9d9ce70b4e4784eaff2122390cbafe82c8a6c1df264bd6601448d5d" Jul 15 19:27:08.161968 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:08.161955 2577 scope.go:117] "RemoveContainer" containerID="142acd72c9d9ce70b4e4784eaff2122390cbafe82c8a6c1df264bd6601448d5d" Jul 15 19:27:08.162194 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:27:08.162177 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"142acd72c9d9ce70b4e4784eaff2122390cbafe82c8a6c1df264bd6601448d5d\": container with ID starting with 142acd72c9d9ce70b4e4784eaff2122390cbafe82c8a6c1df264bd6601448d5d not found: ID does not exist" containerID="142acd72c9d9ce70b4e4784eaff2122390cbafe82c8a6c1df264bd6601448d5d" Jul 15 19:27:08.162245 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:08.162203 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"142acd72c9d9ce70b4e4784eaff2122390cbafe82c8a6c1df264bd6601448d5d"} err="failed to get container status \"142acd72c9d9ce70b4e4784eaff2122390cbafe82c8a6c1df264bd6601448d5d\": rpc error: code = NotFound desc = could not find container \"142acd72c9d9ce70b4e4784eaff2122390cbafe82c8a6c1df264bd6601448d5d\": container with ID starting with 142acd72c9d9ce70b4e4784eaff2122390cbafe82c8a6c1df264bd6601448d5d not found: ID does not exist" Jul 15 19:27:08.174677 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:08.174656 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6fff6645b4-vc7cd"] Jul 15 19:27:08.180110 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:08.180090 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6fff6645b4-vc7cd"] Jul 15 19:27:09.091873 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:09.091843 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c10de9f-5f4e-425d-b08d-3a33a0f21721" path="/var/lib/kubelet/pods/1c10de9f-5f4e-425d-b08d-3a33a0f21721/volumes" Jul 15 19:27:16.570680 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:16.570651 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6c44cd97bd-5kbj2_d1aea75b-a17d-49fe-aa61-2969e7e720d1/router/0.log" Jul 15 19:27:16.585409 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:16.585370 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rdqmw_cf8428ad-0fb7-4108-a41f-8fd01ec35c5c/serve-healthcheck-canary/0.log" Jul 15 19:27:40.850185 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:40.850151 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/237b9375-100a-4d62-83f0-cb271ab8e0bb-metrics-certs\") pod \"network-metrics-daemon-cmqc9\" (UID: \"237b9375-100a-4d62-83f0-cb271ab8e0bb\") " pod="openshift-multus/network-metrics-daemon-cmqc9" Jul 15 19:27:40.852483 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:40.852460 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/237b9375-100a-4d62-83f0-cb271ab8e0bb-metrics-certs\") pod \"network-metrics-daemon-cmqc9\" (UID: \"237b9375-100a-4d62-83f0-cb271ab8e0bb\") " pod="openshift-multus/network-metrics-daemon-cmqc9" Jul 15 19:27:41.093616 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:41.093586 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-dmtm7\"" Jul 15 19:27:41.099285 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:41.099269 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cmqc9" Jul 15 19:27:41.226578 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:41.226548 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cmqc9"] Jul 15 19:27:41.229925 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:27:41.229894 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod237b9375_100a_4d62_83f0_cb271ab8e0bb.slice/crio-64929a70eda144a473f0b149e9b2f3fc45b47f4bd6bbac7c500103ddcd61a464 WatchSource:0}: Error finding container 64929a70eda144a473f0b149e9b2f3fc45b47f4bd6bbac7c500103ddcd61a464: Status 404 returned error can't find the container with id 64929a70eda144a473f0b149e9b2f3fc45b47f4bd6bbac7c500103ddcd61a464 Jul 15 19:27:41.274140 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:41.274104 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cmqc9" event={"ID":"237b9375-100a-4d62-83f0-cb271ab8e0bb","Type":"ContainerStarted","Data":"64929a70eda144a473f0b149e9b2f3fc45b47f4bd6bbac7c500103ddcd61a464"} Jul 15 19:27:42.280827 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:42.280544 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cmqc9" event={"ID":"237b9375-100a-4d62-83f0-cb271ab8e0bb","Type":"ContainerStarted","Data":"4c79d5e87eac4f656a03a638f6a6758fb9c9d0a42b8c7eec2c6b028c93484be2"} Jul 15 19:27:42.280827 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:42.280606 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cmqc9" event={"ID":"237b9375-100a-4d62-83f0-cb271ab8e0bb","Type":"ContainerStarted","Data":"0750e856ce4c2d2ad20cfb12a9eada4de92dd93a1086da4ee430ecbc11b25767"} Jul 15 19:27:42.299450 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:42.299407 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-cmqc9" podStartSLOduration=374.431457992 podStartE2EDuration="6m15.299390025s" totalImagesPullingTime="867.932033ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:21:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:27:42.298038965 +0000 UTC m=+375.875040646" watchObservedRunningTime="2026-07-15 19:27:42.299390025 +0000 UTC m=+375.876391695" Jul 15 19:27:59.587191 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:59.587150 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7c65576f67-dwfqz"] Jul 15 19:27:59.587667 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:59.587553 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1235737d-3c8a-4c3a-af1a-dfd3351d72fb" containerName="registry" Jul 15 19:27:59.587667 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:59.587572 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1235737d-3c8a-4c3a-af1a-dfd3351d72fb" containerName="registry" Jul 15 19:27:59.587667 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:59.587589 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb861bd3-b9cf-4671-bc14-15e29bf00106" containerName="console" Jul 15 19:27:59.587667 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:59.587595 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb861bd3-b9cf-4671-bc14-15e29bf00106" containerName="console" Jul 15 19:27:59.587667 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:59.587603 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c10de9f-5f4e-425d-b08d-3a33a0f21721" containerName="console" Jul 15 19:27:59.587667 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:59.587611 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c10de9f-5f4e-425d-b08d-3a33a0f21721" containerName="console" Jul 15 19:27:59.587667 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:59.587663 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1235737d-3c8a-4c3a-af1a-dfd3351d72fb" containerName="registry" Jul 15 19:27:59.587667 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:59.587671 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb861bd3-b9cf-4671-bc14-15e29bf00106" containerName="console" Jul 15 19:27:59.587905 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:59.587680 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c10de9f-5f4e-425d-b08d-3a33a0f21721" containerName="console" Jul 15 19:27:59.590738 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:59.590638 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c65576f67-dwfqz" Jul 15 19:27:59.601880 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:59.601858 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c65576f67-dwfqz"] Jul 15 19:27:59.680523 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:59.680492 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxmrq\" (UniqueName: \"kubernetes.io/projected/9ec7c062-7d57-45f4-92fa-248c7f9f96c2-kube-api-access-cxmrq\") pod \"console-7c65576f67-dwfqz\" (UID: \"9ec7c062-7d57-45f4-92fa-248c7f9f96c2\") " pod="openshift-console/console-7c65576f67-dwfqz" Jul 15 19:27:59.680688 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:59.680529 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9ec7c062-7d57-45f4-92fa-248c7f9f96c2-console-serving-cert\") pod \"console-7c65576f67-dwfqz\" (UID: \"9ec7c062-7d57-45f4-92fa-248c7f9f96c2\") " pod="openshift-console/console-7c65576f67-dwfqz" Jul 15 19:27:59.680688 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:59.680550 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9ec7c062-7d57-45f4-92fa-248c7f9f96c2-service-ca\") pod \"console-7c65576f67-dwfqz\" (UID: \"9ec7c062-7d57-45f4-92fa-248c7f9f96c2\") " pod="openshift-console/console-7c65576f67-dwfqz" Jul 15 19:27:59.680688 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:59.680629 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9ec7c062-7d57-45f4-92fa-248c7f9f96c2-console-oauth-config\") pod \"console-7c65576f67-dwfqz\" (UID: \"9ec7c062-7d57-45f4-92fa-248c7f9f96c2\") " pod="openshift-console/console-7c65576f67-dwfqz" Jul 15 19:27:59.680688 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:59.680656 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9ec7c062-7d57-45f4-92fa-248c7f9f96c2-console-config\") pod \"console-7c65576f67-dwfqz\" (UID: \"9ec7c062-7d57-45f4-92fa-248c7f9f96c2\") " pod="openshift-console/console-7c65576f67-dwfqz" Jul 15 19:27:59.680688 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:59.680673 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9ec7c062-7d57-45f4-92fa-248c7f9f96c2-oauth-serving-cert\") pod \"console-7c65576f67-dwfqz\" (UID: \"9ec7c062-7d57-45f4-92fa-248c7f9f96c2\") " pod="openshift-console/console-7c65576f67-dwfqz" Jul 15 19:27:59.680847 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:59.680702 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ec7c062-7d57-45f4-92fa-248c7f9f96c2-trusted-ca-bundle\") pod \"console-7c65576f67-dwfqz\" (UID: \"9ec7c062-7d57-45f4-92fa-248c7f9f96c2\") " pod="openshift-console/console-7c65576f67-dwfqz" Jul 15 19:27:59.781206 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:59.781180 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9ec7c062-7d57-45f4-92fa-248c7f9f96c2-console-config\") pod \"console-7c65576f67-dwfqz\" (UID: \"9ec7c062-7d57-45f4-92fa-248c7f9f96c2\") " pod="openshift-console/console-7c65576f67-dwfqz" Jul 15 19:27:59.781400 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:59.781214 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9ec7c062-7d57-45f4-92fa-248c7f9f96c2-oauth-serving-cert\") pod \"console-7c65576f67-dwfqz\" (UID: \"9ec7c062-7d57-45f4-92fa-248c7f9f96c2\") " pod="openshift-console/console-7c65576f67-dwfqz" Jul 15 19:27:59.781400 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:59.781244 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ec7c062-7d57-45f4-92fa-248c7f9f96c2-trusted-ca-bundle\") pod \"console-7c65576f67-dwfqz\" (UID: \"9ec7c062-7d57-45f4-92fa-248c7f9f96c2\") " pod="openshift-console/console-7c65576f67-dwfqz" Jul 15 19:27:59.781400 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:59.781295 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxmrq\" (UniqueName: \"kubernetes.io/projected/9ec7c062-7d57-45f4-92fa-248c7f9f96c2-kube-api-access-cxmrq\") pod \"console-7c65576f67-dwfqz\" (UID: \"9ec7c062-7d57-45f4-92fa-248c7f9f96c2\") " pod="openshift-console/console-7c65576f67-dwfqz" Jul 15 19:27:59.782707 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:59.781839 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9ec7c062-7d57-45f4-92fa-248c7f9f96c2-console-serving-cert\") pod \"console-7c65576f67-dwfqz\" (UID: \"9ec7c062-7d57-45f4-92fa-248c7f9f96c2\") " pod="openshift-console/console-7c65576f67-dwfqz" Jul 15 19:27:59.782707 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:59.781910 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9ec7c062-7d57-45f4-92fa-248c7f9f96c2-service-ca\") pod \"console-7c65576f67-dwfqz\" (UID: \"9ec7c062-7d57-45f4-92fa-248c7f9f96c2\") " pod="openshift-console/console-7c65576f67-dwfqz" Jul 15 19:27:59.782707 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:59.781948 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9ec7c062-7d57-45f4-92fa-248c7f9f96c2-console-oauth-config\") pod \"console-7c65576f67-dwfqz\" (UID: \"9ec7c062-7d57-45f4-92fa-248c7f9f96c2\") " pod="openshift-console/console-7c65576f67-dwfqz" Jul 15 19:27:59.782707 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:59.782507 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9ec7c062-7d57-45f4-92fa-248c7f9f96c2-console-config\") pod \"console-7c65576f67-dwfqz\" (UID: \"9ec7c062-7d57-45f4-92fa-248c7f9f96c2\") " pod="openshift-console/console-7c65576f67-dwfqz" Jul 15 19:27:59.782707 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:59.782659 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ec7c062-7d57-45f4-92fa-248c7f9f96c2-trusted-ca-bundle\") pod \"console-7c65576f67-dwfqz\" (UID: \"9ec7c062-7d57-45f4-92fa-248c7f9f96c2\") " pod="openshift-console/console-7c65576f67-dwfqz" Jul 15 19:27:59.782707 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:59.782671 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9ec7c062-7d57-45f4-92fa-248c7f9f96c2-oauth-serving-cert\") pod \"console-7c65576f67-dwfqz\" (UID: \"9ec7c062-7d57-45f4-92fa-248c7f9f96c2\") " pod="openshift-console/console-7c65576f67-dwfqz" Jul 15 19:27:59.784079 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:59.784057 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9ec7c062-7d57-45f4-92fa-248c7f9f96c2-service-ca\") pod \"console-7c65576f67-dwfqz\" (UID: \"9ec7c062-7d57-45f4-92fa-248c7f9f96c2\") " pod="openshift-console/console-7c65576f67-dwfqz" Jul 15 19:27:59.785350 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:59.785331 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9ec7c062-7d57-45f4-92fa-248c7f9f96c2-console-oauth-config\") pod \"console-7c65576f67-dwfqz\" (UID: \"9ec7c062-7d57-45f4-92fa-248c7f9f96c2\") " pod="openshift-console/console-7c65576f67-dwfqz" Jul 15 19:27:59.785496 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:59.785457 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9ec7c062-7d57-45f4-92fa-248c7f9f96c2-console-serving-cert\") pod \"console-7c65576f67-dwfqz\" (UID: \"9ec7c062-7d57-45f4-92fa-248c7f9f96c2\") " pod="openshift-console/console-7c65576f67-dwfqz" Jul 15 19:27:59.790326 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:59.790291 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxmrq\" (UniqueName: \"kubernetes.io/projected/9ec7c062-7d57-45f4-92fa-248c7f9f96c2-kube-api-access-cxmrq\") pod \"console-7c65576f67-dwfqz\" (UID: \"9ec7c062-7d57-45f4-92fa-248c7f9f96c2\") " pod="openshift-console/console-7c65576f67-dwfqz" Jul 15 19:27:59.902342 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:27:59.902266 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c65576f67-dwfqz" Jul 15 19:28:00.027861 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:00.027831 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c65576f67-dwfqz"] Jul 15 19:28:00.031784 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:28:00.031753 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ec7c062_7d57_45f4_92fa_248c7f9f96c2.slice/crio-b317d3432e4ec9f4ee7365cf2dd301895b1b82bd39dd3f9deefead09657d6721 WatchSource:0}: Error finding container b317d3432e4ec9f4ee7365cf2dd301895b1b82bd39dd3f9deefead09657d6721: Status 404 returned error can't find the container with id b317d3432e4ec9f4ee7365cf2dd301895b1b82bd39dd3f9deefead09657d6721 Jul 15 19:28:00.346793 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:00.346757 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c65576f67-dwfqz" event={"ID":"9ec7c062-7d57-45f4-92fa-248c7f9f96c2","Type":"ContainerStarted","Data":"613a7e0716c35388c204b92820cc40bd97865a070f6048f7f8d9e4a5cb05d5fa"} Jul 15 19:28:00.346964 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:00.346814 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c65576f67-dwfqz" event={"ID":"9ec7c062-7d57-45f4-92fa-248c7f9f96c2","Type":"ContainerStarted","Data":"b317d3432e4ec9f4ee7365cf2dd301895b1b82bd39dd3f9deefead09657d6721"} Jul 15 19:28:00.365954 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:00.365908 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7c65576f67-dwfqz" podStartSLOduration=1.365893077 podStartE2EDuration="1.365893077s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:27:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:28:00.364297609 +0000 UTC m=+393.941299280" watchObservedRunningTime="2026-07-15 19:28:00.365893077 +0000 UTC m=+393.942894752" Jul 15 19:28:09.903328 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:09.903290 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7c65576f67-dwfqz" Jul 15 19:28:09.903851 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:09.903371 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7c65576f67-dwfqz" Jul 15 19:28:09.908467 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:09.908441 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7c65576f67-dwfqz" Jul 15 19:28:10.385900 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:10.385875 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7c65576f67-dwfqz" Jul 15 19:28:10.435825 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:10.435785 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7bfbfd54-h6t58"] Jul 15 19:28:35.462392 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:35.462331 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7bfbfd54-h6t58" podUID="d6d9b88c-1112-4f0e-878a-afdc7cd36495" containerName="console" containerID="cri-o://367745c73862c9c1b6419fa29bd2eedf9c5546786f5d64fb8e7854133b79c9d6" gracePeriod=15 Jul 15 19:28:35.703760 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:35.703738 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bfbfd54-h6t58" Jul 15 19:28:35.749847 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:35.749781 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d6d9b88c-1112-4f0e-878a-afdc7cd36495-oauth-serving-cert\") pod \"d6d9b88c-1112-4f0e-878a-afdc7cd36495\" (UID: \"d6d9b88c-1112-4f0e-878a-afdc7cd36495\") " Jul 15 19:28:35.749847 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:35.749811 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d6d9b88c-1112-4f0e-878a-afdc7cd36495-service-ca\") pod \"d6d9b88c-1112-4f0e-878a-afdc7cd36495\" (UID: \"d6d9b88c-1112-4f0e-878a-afdc7cd36495\") " Jul 15 19:28:35.749847 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:35.749839 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d6d9b88c-1112-4f0e-878a-afdc7cd36495-console-oauth-config\") pod \"d6d9b88c-1112-4f0e-878a-afdc7cd36495\" (UID: \"d6d9b88c-1112-4f0e-878a-afdc7cd36495\") " Jul 15 19:28:35.750063 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:35.749865 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d6d9b88c-1112-4f0e-878a-afdc7cd36495-console-config\") pod \"d6d9b88c-1112-4f0e-878a-afdc7cd36495\" (UID: \"d6d9b88c-1112-4f0e-878a-afdc7cd36495\") " Jul 15 19:28:35.750063 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:35.749892 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6d9b88c-1112-4f0e-878a-afdc7cd36495-trusted-ca-bundle\") pod \"d6d9b88c-1112-4f0e-878a-afdc7cd36495\" (UID: \"d6d9b88c-1112-4f0e-878a-afdc7cd36495\") " Jul 15 19:28:35.750063 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:35.749947 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sp77m\" (UniqueName: \"kubernetes.io/projected/d6d9b88c-1112-4f0e-878a-afdc7cd36495-kube-api-access-sp77m\") pod \"d6d9b88c-1112-4f0e-878a-afdc7cd36495\" (UID: \"d6d9b88c-1112-4f0e-878a-afdc7cd36495\") " Jul 15 19:28:35.750063 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:35.749998 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d6d9b88c-1112-4f0e-878a-afdc7cd36495-console-serving-cert\") pod \"d6d9b88c-1112-4f0e-878a-afdc7cd36495\" (UID: \"d6d9b88c-1112-4f0e-878a-afdc7cd36495\") " Jul 15 19:28:35.750252 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:35.750182 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6d9b88c-1112-4f0e-878a-afdc7cd36495-service-ca" (OuterVolumeSpecName: "service-ca") pod "d6d9b88c-1112-4f0e-878a-afdc7cd36495" (UID: "d6d9b88c-1112-4f0e-878a-afdc7cd36495"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:28:35.750364 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:35.750210 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6d9b88c-1112-4f0e-878a-afdc7cd36495-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d6d9b88c-1112-4f0e-878a-afdc7cd36495" (UID: "d6d9b88c-1112-4f0e-878a-afdc7cd36495"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:28:35.750464 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:35.750428 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6d9b88c-1112-4f0e-878a-afdc7cd36495-console-config" (OuterVolumeSpecName: "console-config") pod "d6d9b88c-1112-4f0e-878a-afdc7cd36495" (UID: "d6d9b88c-1112-4f0e-878a-afdc7cd36495"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:28:35.750595 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:35.750463 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6d9b88c-1112-4f0e-878a-afdc7cd36495-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d6d9b88c-1112-4f0e-878a-afdc7cd36495" (UID: "d6d9b88c-1112-4f0e-878a-afdc7cd36495"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:28:35.750595 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:35.750442 2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d6d9b88c-1112-4f0e-878a-afdc7cd36495-oauth-serving-cert\") on node \"ip-10-0-141-62.ec2.internal\" DevicePath \"\"" Jul 15 19:28:35.750595 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:35.750502 2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d6d9b88c-1112-4f0e-878a-afdc7cd36495-service-ca\") on node \"ip-10-0-141-62.ec2.internal\" DevicePath \"\"" Jul 15 19:28:35.752022 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:35.752003 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6d9b88c-1112-4f0e-878a-afdc7cd36495-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d6d9b88c-1112-4f0e-878a-afdc7cd36495" (UID: "d6d9b88c-1112-4f0e-878a-afdc7cd36495"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:28:35.752098 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:35.752025 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6d9b88c-1112-4f0e-878a-afdc7cd36495-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d6d9b88c-1112-4f0e-878a-afdc7cd36495" (UID: "d6d9b88c-1112-4f0e-878a-afdc7cd36495"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:28:35.752137 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:35.752109 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6d9b88c-1112-4f0e-878a-afdc7cd36495-kube-api-access-sp77m" (OuterVolumeSpecName: "kube-api-access-sp77m") pod "d6d9b88c-1112-4f0e-878a-afdc7cd36495" (UID: "d6d9b88c-1112-4f0e-878a-afdc7cd36495"). InnerVolumeSpecName "kube-api-access-sp77m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:28:35.851667 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:35.851643 2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d6d9b88c-1112-4f0e-878a-afdc7cd36495-console-oauth-config\") on node \"ip-10-0-141-62.ec2.internal\" DevicePath \"\"" Jul 15 19:28:35.851667 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:35.851664 2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d6d9b88c-1112-4f0e-878a-afdc7cd36495-console-config\") on node \"ip-10-0-141-62.ec2.internal\" DevicePath \"\"" Jul 15 19:28:35.851790 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:35.851673 2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6d9b88c-1112-4f0e-878a-afdc7cd36495-trusted-ca-bundle\") on node \"ip-10-0-141-62.ec2.internal\" DevicePath \"\"" Jul 15 19:28:35.851790 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:35.851682 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sp77m\" (UniqueName: \"kubernetes.io/projected/d6d9b88c-1112-4f0e-878a-afdc7cd36495-kube-api-access-sp77m\") on node \"ip-10-0-141-62.ec2.internal\" DevicePath \"\"" Jul 15 19:28:35.851790 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:35.851690 2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d6d9b88c-1112-4f0e-878a-afdc7cd36495-console-serving-cert\") on node \"ip-10-0-141-62.ec2.internal\" DevicePath \"\"" Jul 15 19:28:36.479247 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:36.479211 2577 generic.go:356] "Generic (PLEG): container finished" podID="d6d9b88c-1112-4f0e-878a-afdc7cd36495" containerID="367745c73862c9c1b6419fa29bd2eedf9c5546786f5d64fb8e7854133b79c9d6" exitCode=0 Jul 15 19:28:36.479654 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:36.479276 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bfbfd54-h6t58" Jul 15 19:28:36.479654 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:36.479295 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bfbfd54-h6t58" event={"ID":"d6d9b88c-1112-4f0e-878a-afdc7cd36495","Type":"ContainerDied","Data":"367745c73862c9c1b6419fa29bd2eedf9c5546786f5d64fb8e7854133b79c9d6"} Jul 15 19:28:36.479654 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:36.479366 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bfbfd54-h6t58" event={"ID":"d6d9b88c-1112-4f0e-878a-afdc7cd36495","Type":"ContainerDied","Data":"5e2b88fc8849ae510fe10a2462a4ff8b237a4e3b7ff4e8ae092ca3621403e03e"} Jul 15 19:28:36.479654 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:36.479394 2577 scope.go:117] "RemoveContainer" containerID="367745c73862c9c1b6419fa29bd2eedf9c5546786f5d64fb8e7854133b79c9d6" Jul 15 19:28:36.487672 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:36.487655 2577 scope.go:117] "RemoveContainer" containerID="367745c73862c9c1b6419fa29bd2eedf9c5546786f5d64fb8e7854133b79c9d6" Jul 15 19:28:36.487920 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:28:36.487904 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"367745c73862c9c1b6419fa29bd2eedf9c5546786f5d64fb8e7854133b79c9d6\": container with ID starting with 367745c73862c9c1b6419fa29bd2eedf9c5546786f5d64fb8e7854133b79c9d6 not found: ID does not exist" containerID="367745c73862c9c1b6419fa29bd2eedf9c5546786f5d64fb8e7854133b79c9d6" Jul 15 19:28:36.487961 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:36.487927 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"367745c73862c9c1b6419fa29bd2eedf9c5546786f5d64fb8e7854133b79c9d6"} err="failed to get container status \"367745c73862c9c1b6419fa29bd2eedf9c5546786f5d64fb8e7854133b79c9d6\": rpc error: code = NotFound desc = could not find container \"367745c73862c9c1b6419fa29bd2eedf9c5546786f5d64fb8e7854133b79c9d6\": container with ID starting with 367745c73862c9c1b6419fa29bd2eedf9c5546786f5d64fb8e7854133b79c9d6 not found: ID does not exist" Jul 15 19:28:36.499573 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:36.499551 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7bfbfd54-h6t58"] Jul 15 19:28:36.501439 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:36.501421 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7bfbfd54-h6t58"] Jul 15 19:28:37.090250 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:37.090220 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6d9b88c-1112-4f0e-878a-afdc7cd36495" path="/var/lib/kubelet/pods/d6d9b88c-1112-4f0e-878a-afdc7cd36495/volumes" Jul 15 19:28:57.194287 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:57.194217 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407dxjh"] Jul 15 19:28:57.194718 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:57.194511 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6d9b88c-1112-4f0e-878a-afdc7cd36495" containerName="console" Jul 15 19:28:57.194718 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:57.194522 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6d9b88c-1112-4f0e-878a-afdc7cd36495" containerName="console" Jul 15 19:28:57.194718 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:57.194591 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6d9b88c-1112-4f0e-878a-afdc7cd36495" containerName="console" Jul 15 19:28:57.197734 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:57.197718 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407dxjh" Jul 15 19:28:57.199779 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:57.199758 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 15 19:28:57.200101 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:57.200082 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 15 19:28:57.200165 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:57.200085 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-zzfn2\"" Jul 15 19:28:57.204612 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:57.204590 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407dxjh"] Jul 15 19:28:57.313267 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:57.313239 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txzjv\" (UniqueName: \"kubernetes.io/projected/ce2a0310-fcae-4761-9ae3-cd553d08a75f-kube-api-access-txzjv\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407dxjh\" (UID: \"ce2a0310-fcae-4761-9ae3-cd553d08a75f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407dxjh" Jul 15 19:28:57.313435 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:57.313277 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ce2a0310-fcae-4761-9ae3-cd553d08a75f-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407dxjh\" (UID: \"ce2a0310-fcae-4761-9ae3-cd553d08a75f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407dxjh" Jul 15 19:28:57.313435 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:57.313376 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ce2a0310-fcae-4761-9ae3-cd553d08a75f-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407dxjh\" (UID: \"ce2a0310-fcae-4761-9ae3-cd553d08a75f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407dxjh" Jul 15 19:28:57.414487 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:57.414454 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txzjv\" (UniqueName: \"kubernetes.io/projected/ce2a0310-fcae-4761-9ae3-cd553d08a75f-kube-api-access-txzjv\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407dxjh\" (UID: \"ce2a0310-fcae-4761-9ae3-cd553d08a75f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407dxjh" Jul 15 19:28:57.414635 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:57.414495 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ce2a0310-fcae-4761-9ae3-cd553d08a75f-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407dxjh\" (UID: \"ce2a0310-fcae-4761-9ae3-cd553d08a75f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407dxjh" Jul 15 19:28:57.414635 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:57.414618 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ce2a0310-fcae-4761-9ae3-cd553d08a75f-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407dxjh\" (UID: \"ce2a0310-fcae-4761-9ae3-cd553d08a75f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407dxjh" Jul 15 19:28:57.414872 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:57.414854 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ce2a0310-fcae-4761-9ae3-cd553d08a75f-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407dxjh\" (UID: \"ce2a0310-fcae-4761-9ae3-cd553d08a75f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407dxjh" Jul 15 19:28:57.414921 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:57.414906 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ce2a0310-fcae-4761-9ae3-cd553d08a75f-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407dxjh\" (UID: \"ce2a0310-fcae-4761-9ae3-cd553d08a75f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407dxjh" Jul 15 19:28:57.422569 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:57.422548 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-txzjv\" (UniqueName: \"kubernetes.io/projected/ce2a0310-fcae-4761-9ae3-cd553d08a75f-kube-api-access-txzjv\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407dxjh\" (UID: \"ce2a0310-fcae-4761-9ae3-cd553d08a75f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407dxjh" Jul 15 19:28:57.507102 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:57.507049 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407dxjh" Jul 15 19:28:57.633236 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:57.633206 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407dxjh"] Jul 15 19:28:57.635659 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:28:57.635629 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce2a0310_fcae_4761_9ae3_cd553d08a75f.slice/crio-0dcaff2fef17605a98205d0737b4641d5505d4c021bf30a7daa1a0d2c5c85fc4 WatchSource:0}: Error finding container 0dcaff2fef17605a98205d0737b4641d5505d4c021bf30a7daa1a0d2c5c85fc4: Status 404 returned error can't find the container with id 0dcaff2fef17605a98205d0737b4641d5505d4c021bf30a7daa1a0d2c5c85fc4 Jul 15 19:28:58.561739 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:28:58.561705 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407dxjh" event={"ID":"ce2a0310-fcae-4761-9ae3-cd553d08a75f","Type":"ContainerStarted","Data":"0dcaff2fef17605a98205d0737b4641d5505d4c021bf30a7daa1a0d2c5c85fc4"} Jul 15 19:29:05.591880 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:05.591844 2577 generic.go:356] "Generic (PLEG): container finished" podID="ce2a0310-fcae-4761-9ae3-cd553d08a75f" containerID="a0bb81d3c95688b4b609ef084e84e5c5c3d9c67cc509c1260829b5b96783e676" exitCode=0 Jul 15 19:29:05.592278 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:05.591933 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407dxjh" event={"ID":"ce2a0310-fcae-4761-9ae3-cd553d08a75f","Type":"ContainerDied","Data":"a0bb81d3c95688b4b609ef084e84e5c5c3d9c67cc509c1260829b5b96783e676"} Jul 15 19:29:08.605608 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:08.605575 2577 generic.go:356] "Generic (PLEG): container finished" podID="ce2a0310-fcae-4761-9ae3-cd553d08a75f" containerID="51f9e0f3c8ed649d10e748d896611519b7b042bfdfdac1f3d6829371e1e11083" exitCode=0 Jul 15 19:29:08.605970 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:08.605619 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407dxjh" event={"ID":"ce2a0310-fcae-4761-9ae3-cd553d08a75f","Type":"ContainerDied","Data":"51f9e0f3c8ed649d10e748d896611519b7b042bfdfdac1f3d6829371e1e11083"} Jul 15 19:29:18.650579 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:18.650543 2577 generic.go:356] "Generic (PLEG): container finished" podID="ce2a0310-fcae-4761-9ae3-cd553d08a75f" containerID="dd1739f60d54bdf75a703ab83f8da86b44caee12b162b9d44ed53bf13e98da57" exitCode=0 Jul 15 19:29:18.650948 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:18.650622 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407dxjh" event={"ID":"ce2a0310-fcae-4761-9ae3-cd553d08a75f","Type":"ContainerDied","Data":"dd1739f60d54bdf75a703ab83f8da86b44caee12b162b9d44ed53bf13e98da57"} Jul 15 19:29:19.782690 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:19.782664 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407dxjh" Jul 15 19:29:19.898169 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:19.898137 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-txzjv\" (UniqueName: \"kubernetes.io/projected/ce2a0310-fcae-4761-9ae3-cd553d08a75f-kube-api-access-txzjv\") pod \"ce2a0310-fcae-4761-9ae3-cd553d08a75f\" (UID: \"ce2a0310-fcae-4761-9ae3-cd553d08a75f\") " Jul 15 19:29:19.898335 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:19.898180 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ce2a0310-fcae-4761-9ae3-cd553d08a75f-util\") pod \"ce2a0310-fcae-4761-9ae3-cd553d08a75f\" (UID: \"ce2a0310-fcae-4761-9ae3-cd553d08a75f\") " Jul 15 19:29:19.898335 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:19.898226 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ce2a0310-fcae-4761-9ae3-cd553d08a75f-bundle\") pod \"ce2a0310-fcae-4761-9ae3-cd553d08a75f\" (UID: \"ce2a0310-fcae-4761-9ae3-cd553d08a75f\") " Jul 15 19:29:19.898861 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:19.898838 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce2a0310-fcae-4761-9ae3-cd553d08a75f-bundle" (OuterVolumeSpecName: "bundle") pod "ce2a0310-fcae-4761-9ae3-cd553d08a75f" (UID: "ce2a0310-fcae-4761-9ae3-cd553d08a75f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:29:19.900243 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:19.900218 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce2a0310-fcae-4761-9ae3-cd553d08a75f-kube-api-access-txzjv" (OuterVolumeSpecName: "kube-api-access-txzjv") pod "ce2a0310-fcae-4761-9ae3-cd553d08a75f" (UID: "ce2a0310-fcae-4761-9ae3-cd553d08a75f"). InnerVolumeSpecName "kube-api-access-txzjv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:29:19.902692 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:19.902669 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce2a0310-fcae-4761-9ae3-cd553d08a75f-util" (OuterVolumeSpecName: "util") pod "ce2a0310-fcae-4761-9ae3-cd553d08a75f" (UID: "ce2a0310-fcae-4761-9ae3-cd553d08a75f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:29:19.998834 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:19.998784 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ce2a0310-fcae-4761-9ae3-cd553d08a75f-bundle\") on node \"ip-10-0-141-62.ec2.internal\" DevicePath \"\"" Jul 15 19:29:19.998834 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:19.998805 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-txzjv\" (UniqueName: \"kubernetes.io/projected/ce2a0310-fcae-4761-9ae3-cd553d08a75f-kube-api-access-txzjv\") on node \"ip-10-0-141-62.ec2.internal\" DevicePath \"\"" Jul 15 19:29:19.998834 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:19.998817 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ce2a0310-fcae-4761-9ae3-cd553d08a75f-util\") on node \"ip-10-0-141-62.ec2.internal\" DevicePath \"\"" Jul 15 19:29:20.660858 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:20.660827 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407dxjh" event={"ID":"ce2a0310-fcae-4761-9ae3-cd553d08a75f","Type":"ContainerDied","Data":"0dcaff2fef17605a98205d0737b4641d5505d4c021bf30a7daa1a0d2c5c85fc4"} Jul 15 19:29:20.660858 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:20.660860 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407dxjh" Jul 15 19:29:20.661067 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:20.660869 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0dcaff2fef17605a98205d0737b4641d5505d4c021bf30a7daa1a0d2c5c85fc4" Jul 15 19:29:31.692934 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:31.692902 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7k798"] Jul 15 19:29:31.693307 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:31.693176 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce2a0310-fcae-4761-9ae3-cd553d08a75f" containerName="util" Jul 15 19:29:31.693307 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:31.693187 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce2a0310-fcae-4761-9ae3-cd553d08a75f" containerName="util" Jul 15 19:29:31.693307 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:31.693199 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce2a0310-fcae-4761-9ae3-cd553d08a75f" containerName="extract" Jul 15 19:29:31.693307 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:31.693205 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce2a0310-fcae-4761-9ae3-cd553d08a75f" containerName="extract" Jul 15 19:29:31.693307 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:31.693221 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce2a0310-fcae-4761-9ae3-cd553d08a75f" containerName="pull" Jul 15 19:29:31.693307 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:31.693227 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce2a0310-fcae-4761-9ae3-cd553d08a75f" containerName="pull" Jul 15 19:29:31.693307 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:31.693275 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce2a0310-fcae-4761-9ae3-cd553d08a75f" containerName="extract" Jul 15 19:29:31.695735 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:31.695718 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7k798" Jul 15 19:29:31.697861 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:31.697833 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-pp75c\"" Jul 15 19:29:31.697861 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:31.697842 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 15 19:29:31.698054 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:31.697873 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 15 19:29:31.708268 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:31.708248 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7k798"] Jul 15 19:29:31.887363 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:31.887299 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ed298985-e74d-41be-b13e-0a2931161c7f-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-7k798\" (UID: \"ed298985-e74d-41be-b13e-0a2931161c7f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7k798" Jul 15 19:29:31.887516 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:31.887391 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6c2gm\" (UniqueName: \"kubernetes.io/projected/ed298985-e74d-41be-b13e-0a2931161c7f-kube-api-access-6c2gm\") pod \"cert-manager-operator-controller-manager-55c444bf86-7k798\" (UID: \"ed298985-e74d-41be-b13e-0a2931161c7f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7k798" Jul 15 19:29:31.988286 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:31.988215 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ed298985-e74d-41be-b13e-0a2931161c7f-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-7k798\" (UID: \"ed298985-e74d-41be-b13e-0a2931161c7f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7k798" Jul 15 19:29:31.988286 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:31.988265 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6c2gm\" (UniqueName: \"kubernetes.io/projected/ed298985-e74d-41be-b13e-0a2931161c7f-kube-api-access-6c2gm\") pod \"cert-manager-operator-controller-manager-55c444bf86-7k798\" (UID: \"ed298985-e74d-41be-b13e-0a2931161c7f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7k798" Jul 15 19:29:31.988585 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:31.988561 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ed298985-e74d-41be-b13e-0a2931161c7f-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-7k798\" (UID: \"ed298985-e74d-41be-b13e-0a2931161c7f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7k798" Jul 15 19:29:32.002495 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:32.002464 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6c2gm\" (UniqueName: \"kubernetes.io/projected/ed298985-e74d-41be-b13e-0a2931161c7f-kube-api-access-6c2gm\") pod \"cert-manager-operator-controller-manager-55c444bf86-7k798\" (UID: \"ed298985-e74d-41be-b13e-0a2931161c7f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7k798" Jul 15 19:29:32.005095 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:32.005079 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7k798" Jul 15 19:29:32.134586 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:32.134515 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7k798"] Jul 15 19:29:32.137103 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:29:32.137080 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded298985_e74d_41be_b13e_0a2931161c7f.slice/crio-900bf25508e21776c6cbfc7826cf49b10a4bf8c2ce1ca21715a02811257a4e25 WatchSource:0}: Error finding container 900bf25508e21776c6cbfc7826cf49b10a4bf8c2ce1ca21715a02811257a4e25: Status 404 returned error can't find the container with id 900bf25508e21776c6cbfc7826cf49b10a4bf8c2ce1ca21715a02811257a4e25 Jul 15 19:29:32.706919 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:32.706889 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7k798" event={"ID":"ed298985-e74d-41be-b13e-0a2931161c7f","Type":"ContainerStarted","Data":"900bf25508e21776c6cbfc7826cf49b10a4bf8c2ce1ca21715a02811257a4e25"} Jul 15 19:29:34.717287 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:34.717250 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7k798" event={"ID":"ed298985-e74d-41be-b13e-0a2931161c7f","Type":"ContainerStarted","Data":"382a3dabfe166e5c563773024dbef66c12dffe5b9dd7a5f3166579b3a8c545ee"} Jul 15 19:29:34.737710 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:34.737637 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7k798" podStartSLOduration=2.1625194580000002 podStartE2EDuration="3.737620771s" totalImagesPullingTime="1.575101313s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:29:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:29:34.736228127 +0000 UTC m=+488.313229797" watchObservedRunningTime="2026-07-15 19:29:34.737620771 +0000 UTC m=+488.314622442" Jul 15 19:29:36.573337 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:36.573283 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-9vh8b"] Jul 15 19:29:36.575658 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:36.575642 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-9vh8b" Jul 15 19:29:36.577672 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:36.577650 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 15 19:29:36.577823 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:36.577799 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 15 19:29:36.578421 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:36.578405 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-87cdq\"" Jul 15 19:29:36.584255 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:36.584234 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-9vh8b"] Jul 15 19:29:36.624976 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:36.624956 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5582772f-8b84-4fac-a53e-cf632d3695d6-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-9vh8b\" (UID: \"5582772f-8b84-4fac-a53e-cf632d3695d6\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9vh8b" Jul 15 19:29:36.625074 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:36.624999 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fg69z\" (UniqueName: \"kubernetes.io/projected/5582772f-8b84-4fac-a53e-cf632d3695d6-kube-api-access-fg69z\") pod \"cert-manager-webhook-6486ffd997-9vh8b\" (UID: \"5582772f-8b84-4fac-a53e-cf632d3695d6\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9vh8b" Jul 15 19:29:36.725608 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:36.725582 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5582772f-8b84-4fac-a53e-cf632d3695d6-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-9vh8b\" (UID: \"5582772f-8b84-4fac-a53e-cf632d3695d6\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9vh8b" Jul 15 19:29:36.725716 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:36.725630 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fg69z\" (UniqueName: \"kubernetes.io/projected/5582772f-8b84-4fac-a53e-cf632d3695d6-kube-api-access-fg69z\") pod \"cert-manager-webhook-6486ffd997-9vh8b\" (UID: \"5582772f-8b84-4fac-a53e-cf632d3695d6\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9vh8b" Jul 15 19:29:36.733369 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:36.733341 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5582772f-8b84-4fac-a53e-cf632d3695d6-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-9vh8b\" (UID: \"5582772f-8b84-4fac-a53e-cf632d3695d6\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9vh8b" Jul 15 19:29:36.733524 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:36.733507 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fg69z\" (UniqueName: \"kubernetes.io/projected/5582772f-8b84-4fac-a53e-cf632d3695d6-kube-api-access-fg69z\") pod \"cert-manager-webhook-6486ffd997-9vh8b\" (UID: \"5582772f-8b84-4fac-a53e-cf632d3695d6\") " pod="cert-manager/cert-manager-webhook-6486ffd997-9vh8b" Jul 15 19:29:36.898239 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:36.898171 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-9vh8b" Jul 15 19:29:37.025074 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:37.025044 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-9vh8b"] Jul 15 19:29:37.028213 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:29:37.028187 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5582772f_8b84_4fac_a53e_cf632d3695d6.slice/crio-4cd5d8e558c3e39c60d4ce1f61f6ed8dd6b31b9934e805a53630f5bd77591b2e WatchSource:0}: Error finding container 4cd5d8e558c3e39c60d4ce1f61f6ed8dd6b31b9934e805a53630f5bd77591b2e: Status 404 returned error can't find the container with id 4cd5d8e558c3e39c60d4ce1f61f6ed8dd6b31b9934e805a53630f5bd77591b2e Jul 15 19:29:37.730453 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:37.730421 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-9vh8b" event={"ID":"5582772f-8b84-4fac-a53e-cf632d3695d6","Type":"ContainerStarted","Data":"4cd5d8e558c3e39c60d4ce1f61f6ed8dd6b31b9934e805a53630f5bd77591b2e"} Jul 15 19:29:37.786819 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:37.786790 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwmjkj"] Jul 15 19:29:37.789930 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:37.789913 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwmjkj" Jul 15 19:29:37.792102 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:37.792081 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 15 19:29:37.792226 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:37.792145 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-zzfn2\"" Jul 15 19:29:37.792720 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:37.792702 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 15 19:29:37.801987 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:37.801964 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwmjkj"] Jul 15 19:29:37.833743 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:37.833719 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aaf90a54-99a1-43fb-8d89-34028366e276-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwmjkj\" (UID: \"aaf90a54-99a1-43fb-8d89-34028366e276\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwmjkj" Jul 15 19:29:37.833878 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:37.833758 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aaf90a54-99a1-43fb-8d89-34028366e276-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwmjkj\" (UID: \"aaf90a54-99a1-43fb-8d89-34028366e276\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwmjkj" Jul 15 19:29:37.833878 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:37.833777 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7g589\" (UniqueName: \"kubernetes.io/projected/aaf90a54-99a1-43fb-8d89-34028366e276-kube-api-access-7g589\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwmjkj\" (UID: \"aaf90a54-99a1-43fb-8d89-34028366e276\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwmjkj" Jul 15 19:29:37.934801 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:37.934659 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aaf90a54-99a1-43fb-8d89-34028366e276-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwmjkj\" (UID: \"aaf90a54-99a1-43fb-8d89-34028366e276\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwmjkj" Jul 15 19:29:37.934954 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:37.934864 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aaf90a54-99a1-43fb-8d89-34028366e276-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwmjkj\" (UID: \"aaf90a54-99a1-43fb-8d89-34028366e276\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwmjkj" Jul 15 19:29:37.934954 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:37.934901 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7g589\" (UniqueName: \"kubernetes.io/projected/aaf90a54-99a1-43fb-8d89-34028366e276-kube-api-access-7g589\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwmjkj\" (UID: \"aaf90a54-99a1-43fb-8d89-34028366e276\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwmjkj" Jul 15 19:29:37.935049 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:37.934947 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aaf90a54-99a1-43fb-8d89-34028366e276-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwmjkj\" (UID: \"aaf90a54-99a1-43fb-8d89-34028366e276\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwmjkj" Jul 15 19:29:37.935274 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:37.935252 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aaf90a54-99a1-43fb-8d89-34028366e276-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwmjkj\" (UID: \"aaf90a54-99a1-43fb-8d89-34028366e276\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwmjkj" Jul 15 19:29:37.944071 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:37.944046 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7g589\" (UniqueName: \"kubernetes.io/projected/aaf90a54-99a1-43fb-8d89-34028366e276-kube-api-access-7g589\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwmjkj\" (UID: \"aaf90a54-99a1-43fb-8d89-34028366e276\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwmjkj" Jul 15 19:29:38.099204 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:38.099167 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwmjkj" Jul 15 19:29:38.165427 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:38.165398 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-4j2wf"] Jul 15 19:29:38.168793 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:38.168772 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-4j2wf" Jul 15 19:29:38.170815 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:38.170754 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-7jt2t\"" Jul 15 19:29:38.178663 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:38.178094 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-4j2wf"] Jul 15 19:29:38.237144 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:38.237107 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rn64\" (UniqueName: \"kubernetes.io/projected/983ac4fd-6751-4204-823e-79d126f0cf11-kube-api-access-7rn64\") pod \"cert-manager-cainjector-559cdd8c87-4j2wf\" (UID: \"983ac4fd-6751-4204-823e-79d126f0cf11\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-4j2wf" Jul 15 19:29:38.237322 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:38.237226 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/983ac4fd-6751-4204-823e-79d126f0cf11-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-4j2wf\" (UID: \"983ac4fd-6751-4204-823e-79d126f0cf11\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-4j2wf" Jul 15 19:29:38.249734 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:38.249706 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwmjkj"] Jul 15 19:29:38.252857 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:29:38.252819 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaaf90a54_99a1_43fb_8d89_34028366e276.slice/crio-339db059be2150e4c137b99760a501b14c0b2f97fd370807ed27fa2907b92854 WatchSource:0}: Error finding container 339db059be2150e4c137b99760a501b14c0b2f97fd370807ed27fa2907b92854: Status 404 returned error can't find the container with id 339db059be2150e4c137b99760a501b14c0b2f97fd370807ed27fa2907b92854 Jul 15 19:29:38.338668 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:38.338642 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rn64\" (UniqueName: \"kubernetes.io/projected/983ac4fd-6751-4204-823e-79d126f0cf11-kube-api-access-7rn64\") pod \"cert-manager-cainjector-559cdd8c87-4j2wf\" (UID: \"983ac4fd-6751-4204-823e-79d126f0cf11\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-4j2wf" Jul 15 19:29:38.338781 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:38.338706 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/983ac4fd-6751-4204-823e-79d126f0cf11-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-4j2wf\" (UID: \"983ac4fd-6751-4204-823e-79d126f0cf11\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-4j2wf" Jul 15 19:29:38.367565 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:38.363350 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rn64\" (UniqueName: \"kubernetes.io/projected/983ac4fd-6751-4204-823e-79d126f0cf11-kube-api-access-7rn64\") pod \"cert-manager-cainjector-559cdd8c87-4j2wf\" (UID: \"983ac4fd-6751-4204-823e-79d126f0cf11\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-4j2wf" Jul 15 19:29:38.367565 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:38.365489 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/983ac4fd-6751-4204-823e-79d126f0cf11-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-4j2wf\" (UID: \"983ac4fd-6751-4204-823e-79d126f0cf11\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-4j2wf" Jul 15 19:29:38.481272 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:38.481240 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-4j2wf" Jul 15 19:29:38.628715 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:38.628687 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-4j2wf"] Jul 15 19:29:38.632082 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:29:38.632050 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod983ac4fd_6751_4204_823e_79d126f0cf11.slice/crio-8f48b7e9c335c00b15d424749deb82f16f319c52fc48a5660f62abcdc46eef7c WatchSource:0}: Error finding container 8f48b7e9c335c00b15d424749deb82f16f319c52fc48a5660f62abcdc46eef7c: Status 404 returned error can't find the container with id 8f48b7e9c335c00b15d424749deb82f16f319c52fc48a5660f62abcdc46eef7c Jul 15 19:29:38.737071 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:38.737035 2577 generic.go:356] "Generic (PLEG): container finished" podID="aaf90a54-99a1-43fb-8d89-34028366e276" containerID="960e66da3e5fb762c8560a8d38f3f6106b54ed84e249840d9fedeb1079ecf817" exitCode=0 Jul 15 19:29:38.737543 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:38.737126 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwmjkj" event={"ID":"aaf90a54-99a1-43fb-8d89-34028366e276","Type":"ContainerDied","Data":"960e66da3e5fb762c8560a8d38f3f6106b54ed84e249840d9fedeb1079ecf817"} Jul 15 19:29:38.737543 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:38.737214 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwmjkj" event={"ID":"aaf90a54-99a1-43fb-8d89-34028366e276","Type":"ContainerStarted","Data":"339db059be2150e4c137b99760a501b14c0b2f97fd370807ed27fa2907b92854"} Jul 15 19:29:38.738620 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:38.738588 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-4j2wf" event={"ID":"983ac4fd-6751-4204-823e-79d126f0cf11","Type":"ContainerStarted","Data":"8f48b7e9c335c00b15d424749deb82f16f319c52fc48a5660f62abcdc46eef7c"} Jul 15 19:29:40.750797 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:40.750751 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-4j2wf" event={"ID":"983ac4fd-6751-4204-823e-79d126f0cf11","Type":"ContainerStarted","Data":"d525e19f8c14abf3d528563f8a3e41b4fbc6488cc9cedd1b3a50c01cac86c23c"} Jul 15 19:29:40.752475 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:40.752447 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-9vh8b" event={"ID":"5582772f-8b84-4fac-a53e-cf632d3695d6","Type":"ContainerStarted","Data":"27ae641248a55d985af91ce74bb711f4e2447185648c2e30be0e70f72cab9b81"} Jul 15 19:29:40.752627 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:40.752571 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-9vh8b" Jul 15 19:29:40.767418 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:40.767376 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-4j2wf" podStartSLOduration=1.444024814 podStartE2EDuration="2.767360422s" totalImagesPullingTime="1.323335608s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:29:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:29:40.766134021 +0000 UTC m=+494.343135693" watchObservedRunningTime="2026-07-15 19:29:40.767360422 +0000 UTC m=+494.344362094" Jul 15 19:29:40.784014 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:40.783956 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-9vh8b" podStartSLOduration=1.95607094 podStartE2EDuration="4.783938934s" totalImagesPullingTime="2.827867994s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:29:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:29:40.781156419 +0000 UTC m=+494.358158091" watchObservedRunningTime="2026-07-15 19:29:40.783938934 +0000 UTC m=+494.360940605" Jul 15 19:29:41.758303 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:41.758263 2577 generic.go:356] "Generic (PLEG): container finished" podID="aaf90a54-99a1-43fb-8d89-34028366e276" containerID="72568e69e774b12c6044f5e58ce8d685189f12e7f60362486506166693ca5355" exitCode=0 Jul 15 19:29:41.758691 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:41.758351 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwmjkj" event={"ID":"aaf90a54-99a1-43fb-8d89-34028366e276","Type":"ContainerDied","Data":"72568e69e774b12c6044f5e58ce8d685189f12e7f60362486506166693ca5355"} Jul 15 19:29:42.764564 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:42.764531 2577 generic.go:356] "Generic (PLEG): container finished" podID="aaf90a54-99a1-43fb-8d89-34028366e276" containerID="f25ffcc55a6fbc25cfbb2c87cc13424afe47c84718d5394d45ebc949c1d73ebd" exitCode=0 Jul 15 19:29:42.764923 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:42.764623 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwmjkj" event={"ID":"aaf90a54-99a1-43fb-8d89-34028366e276","Type":"ContainerDied","Data":"f25ffcc55a6fbc25cfbb2c87cc13424afe47c84718d5394d45ebc949c1d73ebd"} Jul 15 19:29:43.891263 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:43.891238 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwmjkj" Jul 15 19:29:43.984223 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:43.984196 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7g589\" (UniqueName: \"kubernetes.io/projected/aaf90a54-99a1-43fb-8d89-34028366e276-kube-api-access-7g589\") pod \"aaf90a54-99a1-43fb-8d89-34028366e276\" (UID: \"aaf90a54-99a1-43fb-8d89-34028366e276\") " Jul 15 19:29:43.984400 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:43.984237 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aaf90a54-99a1-43fb-8d89-34028366e276-util\") pod \"aaf90a54-99a1-43fb-8d89-34028366e276\" (UID: \"aaf90a54-99a1-43fb-8d89-34028366e276\") " Jul 15 19:29:43.984400 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:43.984261 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aaf90a54-99a1-43fb-8d89-34028366e276-bundle\") pod \"aaf90a54-99a1-43fb-8d89-34028366e276\" (UID: \"aaf90a54-99a1-43fb-8d89-34028366e276\") " Jul 15 19:29:43.984684 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:43.984660 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aaf90a54-99a1-43fb-8d89-34028366e276-bundle" (OuterVolumeSpecName: "bundle") pod "aaf90a54-99a1-43fb-8d89-34028366e276" (UID: "aaf90a54-99a1-43fb-8d89-34028366e276"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:29:43.986302 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:43.986283 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aaf90a54-99a1-43fb-8d89-34028366e276-kube-api-access-7g589" (OuterVolumeSpecName: "kube-api-access-7g589") pod "aaf90a54-99a1-43fb-8d89-34028366e276" (UID: "aaf90a54-99a1-43fb-8d89-34028366e276"). InnerVolumeSpecName "kube-api-access-7g589". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:29:43.988829 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:43.988808 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aaf90a54-99a1-43fb-8d89-34028366e276-util" (OuterVolumeSpecName: "util") pod "aaf90a54-99a1-43fb-8d89-34028366e276" (UID: "aaf90a54-99a1-43fb-8d89-34028366e276"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:29:44.085228 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:44.085205 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aaf90a54-99a1-43fb-8d89-34028366e276-bundle\") on node \"ip-10-0-141-62.ec2.internal\" DevicePath \"\"" Jul 15 19:29:44.085228 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:44.085227 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7g589\" (UniqueName: \"kubernetes.io/projected/aaf90a54-99a1-43fb-8d89-34028366e276-kube-api-access-7g589\") on node \"ip-10-0-141-62.ec2.internal\" DevicePath \"\"" Jul 15 19:29:44.085380 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:44.085242 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aaf90a54-99a1-43fb-8d89-34028366e276-util\") on node \"ip-10-0-141-62.ec2.internal\" DevicePath \"\"" Jul 15 19:29:44.776128 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:44.776095 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwmjkj" event={"ID":"aaf90a54-99a1-43fb-8d89-34028366e276","Type":"ContainerDied","Data":"339db059be2150e4c137b99760a501b14c0b2f97fd370807ed27fa2907b92854"} Jul 15 19:29:44.776128 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:44.776129 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fwmjkj" Jul 15 19:29:44.776384 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:44.776141 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="339db059be2150e4c137b99760a501b14c0b2f97fd370807ed27fa2907b92854" Jul 15 19:29:46.760973 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:29:46.760944 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-9vh8b" Jul 15 19:30:01.855174 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:01.855140 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4bc8n"] Jul 15 19:30:01.855662 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:01.855580 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aaf90a54-99a1-43fb-8d89-34028366e276" containerName="util" Jul 15 19:30:01.855662 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:01.855598 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="aaf90a54-99a1-43fb-8d89-34028366e276" containerName="util" Jul 15 19:30:01.855662 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:01.855616 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aaf90a54-99a1-43fb-8d89-34028366e276" containerName="extract" Jul 15 19:30:01.855662 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:01.855624 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="aaf90a54-99a1-43fb-8d89-34028366e276" containerName="extract" Jul 15 19:30:01.855662 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:01.855636 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aaf90a54-99a1-43fb-8d89-34028366e276" containerName="pull" Jul 15 19:30:01.855662 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:01.855642 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="aaf90a54-99a1-43fb-8d89-34028366e276" containerName="pull" Jul 15 19:30:01.855868 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:01.855700 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="aaf90a54-99a1-43fb-8d89-34028366e276" containerName="extract" Jul 15 19:30:01.857844 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:01.857827 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4bc8n" Jul 15 19:30:01.860230 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:01.860203 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 15 19:30:01.860379 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:01.860204 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 15 19:30:01.860850 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:01.860824 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-zzfn2\"" Jul 15 19:30:01.868276 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:01.868250 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4bc8n"] Jul 15 19:30:01.916973 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:01.916940 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtz98\" (UniqueName: \"kubernetes.io/projected/9030baa3-821e-400f-a9e8-5c95a00e0238-kube-api-access-qtz98\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4bc8n\" (UID: \"9030baa3-821e-400f-a9e8-5c95a00e0238\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4bc8n" Jul 15 19:30:01.917149 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:01.916985 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9030baa3-821e-400f-a9e8-5c95a00e0238-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4bc8n\" (UID: \"9030baa3-821e-400f-a9e8-5c95a00e0238\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4bc8n" Jul 15 19:30:01.917149 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:01.917012 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9030baa3-821e-400f-a9e8-5c95a00e0238-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4bc8n\" (UID: \"9030baa3-821e-400f-a9e8-5c95a00e0238\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4bc8n" Jul 15 19:30:02.017891 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:02.017856 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9030baa3-821e-400f-a9e8-5c95a00e0238-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4bc8n\" (UID: \"9030baa3-821e-400f-a9e8-5c95a00e0238\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4bc8n" Jul 15 19:30:02.018067 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:02.017926 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtz98\" (UniqueName: \"kubernetes.io/projected/9030baa3-821e-400f-a9e8-5c95a00e0238-kube-api-access-qtz98\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4bc8n\" (UID: \"9030baa3-821e-400f-a9e8-5c95a00e0238\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4bc8n" Jul 15 19:30:02.018067 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:02.017963 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9030baa3-821e-400f-a9e8-5c95a00e0238-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4bc8n\" (UID: \"9030baa3-821e-400f-a9e8-5c95a00e0238\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4bc8n" Jul 15 19:30:02.018246 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:02.018225 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9030baa3-821e-400f-a9e8-5c95a00e0238-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4bc8n\" (UID: \"9030baa3-821e-400f-a9e8-5c95a00e0238\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4bc8n" Jul 15 19:30:02.018345 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:02.018306 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9030baa3-821e-400f-a9e8-5c95a00e0238-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4bc8n\" (UID: \"9030baa3-821e-400f-a9e8-5c95a00e0238\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4bc8n" Jul 15 19:30:02.027455 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:02.027420 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtz98\" (UniqueName: \"kubernetes.io/projected/9030baa3-821e-400f-a9e8-5c95a00e0238-kube-api-access-qtz98\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4bc8n\" (UID: \"9030baa3-821e-400f-a9e8-5c95a00e0238\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4bc8n" Jul 15 19:30:02.167285 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:02.167199 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4bc8n" Jul 15 19:30:02.296066 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:02.296021 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4bc8n"] Jul 15 19:30:02.299406 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:30:02.299377 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9030baa3_821e_400f_a9e8_5c95a00e0238.slice/crio-a81879724ed623b1132bf007e042cb8716087833f6c14ab3a98b9837751947d4 WatchSource:0}: Error finding container a81879724ed623b1132bf007e042cb8716087833f6c14ab3a98b9837751947d4: Status 404 returned error can't find the container with id a81879724ed623b1132bf007e042cb8716087833f6c14ab3a98b9837751947d4 Jul 15 19:30:02.857703 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:02.857669 2577 generic.go:356] "Generic (PLEG): container finished" podID="9030baa3-821e-400f-a9e8-5c95a00e0238" containerID="8eb4532abe91b962c92edee8ed9949702b70395b02f9c16b4c4951e47bd683e6" exitCode=0 Jul 15 19:30:02.858141 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:02.857751 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4bc8n" event={"ID":"9030baa3-821e-400f-a9e8-5c95a00e0238","Type":"ContainerDied","Data":"8eb4532abe91b962c92edee8ed9949702b70395b02f9c16b4c4951e47bd683e6"} Jul 15 19:30:02.858141 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:02.857800 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4bc8n" event={"ID":"9030baa3-821e-400f-a9e8-5c95a00e0238","Type":"ContainerStarted","Data":"a81879724ed623b1132bf007e042cb8716087833f6c14ab3a98b9837751947d4"} Jul 15 19:30:05.876895 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:05.876863 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4bc8n" event={"ID":"9030baa3-821e-400f-a9e8-5c95a00e0238","Type":"ContainerStarted","Data":"3ae6c3c5edba150e247fa34685bc596c2e7f982480cf9b3cd5452183e8cfc621"} Jul 15 19:30:06.773492 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:06.773459 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6788c996f8-phgb4"] Jul 15 19:30:06.776429 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:06.776413 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6788c996f8-phgb4" Jul 15 19:30:06.779614 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:06.779588 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-5qbqr\"" Jul 15 19:30:06.779753 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:06.779618 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 15 19:30:06.779753 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:06.779696 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 15 19:30:06.779753 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:06.779706 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 15 19:30:06.779753 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:06.779717 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 15 19:30:06.780143 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:06.780125 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 15 19:30:06.788893 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:06.788871 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6788c996f8-phgb4"] Jul 15 19:30:06.860496 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:06.860457 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9bbe494d-9024-4dfb-a6c0-b5a74466304b-manager-config\") pod \"lws-controller-manager-6788c996f8-phgb4\" (UID: \"9bbe494d-9024-4dfb-a6c0-b5a74466304b\") " pod="openshift-lws-operator/lws-controller-manager-6788c996f8-phgb4" Jul 15 19:30:06.860700 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:06.860536 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9bbe494d-9024-4dfb-a6c0-b5a74466304b-cert\") pod \"lws-controller-manager-6788c996f8-phgb4\" (UID: \"9bbe494d-9024-4dfb-a6c0-b5a74466304b\") " pod="openshift-lws-operator/lws-controller-manager-6788c996f8-phgb4" Jul 15 19:30:06.860700 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:06.860569 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9bbe494d-9024-4dfb-a6c0-b5a74466304b-metrics-cert\") pod \"lws-controller-manager-6788c996f8-phgb4\" (UID: \"9bbe494d-9024-4dfb-a6c0-b5a74466304b\") " pod="openshift-lws-operator/lws-controller-manager-6788c996f8-phgb4" Jul 15 19:30:06.860700 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:06.860614 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46ttc\" (UniqueName: \"kubernetes.io/projected/9bbe494d-9024-4dfb-a6c0-b5a74466304b-kube-api-access-46ttc\") pod \"lws-controller-manager-6788c996f8-phgb4\" (UID: \"9bbe494d-9024-4dfb-a6c0-b5a74466304b\") " pod="openshift-lws-operator/lws-controller-manager-6788c996f8-phgb4" Jul 15 19:30:06.884378 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:06.884342 2577 generic.go:356] "Generic (PLEG): container finished" podID="9030baa3-821e-400f-a9e8-5c95a00e0238" containerID="3ae6c3c5edba150e247fa34685bc596c2e7f982480cf9b3cd5452183e8cfc621" exitCode=0 Jul 15 19:30:06.884941 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:06.884429 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4bc8n" event={"ID":"9030baa3-821e-400f-a9e8-5c95a00e0238","Type":"ContainerDied","Data":"3ae6c3c5edba150e247fa34685bc596c2e7f982480cf9b3cd5452183e8cfc621"} Jul 15 19:30:06.961993 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:06.961957 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46ttc\" (UniqueName: \"kubernetes.io/projected/9bbe494d-9024-4dfb-a6c0-b5a74466304b-kube-api-access-46ttc\") pod \"lws-controller-manager-6788c996f8-phgb4\" (UID: \"9bbe494d-9024-4dfb-a6c0-b5a74466304b\") " pod="openshift-lws-operator/lws-controller-manager-6788c996f8-phgb4" Jul 15 19:30:06.962132 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:06.962041 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9bbe494d-9024-4dfb-a6c0-b5a74466304b-manager-config\") pod \"lws-controller-manager-6788c996f8-phgb4\" (UID: \"9bbe494d-9024-4dfb-a6c0-b5a74466304b\") " pod="openshift-lws-operator/lws-controller-manager-6788c996f8-phgb4" Jul 15 19:30:06.962132 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:06.962092 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9bbe494d-9024-4dfb-a6c0-b5a74466304b-cert\") pod \"lws-controller-manager-6788c996f8-phgb4\" (UID: \"9bbe494d-9024-4dfb-a6c0-b5a74466304b\") " pod="openshift-lws-operator/lws-controller-manager-6788c996f8-phgb4" Jul 15 19:30:06.962132 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:06.962118 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9bbe494d-9024-4dfb-a6c0-b5a74466304b-metrics-cert\") pod \"lws-controller-manager-6788c996f8-phgb4\" (UID: \"9bbe494d-9024-4dfb-a6c0-b5a74466304b\") " pod="openshift-lws-operator/lws-controller-manager-6788c996f8-phgb4" Jul 15 19:30:06.963390 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:06.963362 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9bbe494d-9024-4dfb-a6c0-b5a74466304b-manager-config\") pod \"lws-controller-manager-6788c996f8-phgb4\" (UID: \"9bbe494d-9024-4dfb-a6c0-b5a74466304b\") " pod="openshift-lws-operator/lws-controller-manager-6788c996f8-phgb4" Jul 15 19:30:06.966384 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:06.965290 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9bbe494d-9024-4dfb-a6c0-b5a74466304b-cert\") pod \"lws-controller-manager-6788c996f8-phgb4\" (UID: \"9bbe494d-9024-4dfb-a6c0-b5a74466304b\") " pod="openshift-lws-operator/lws-controller-manager-6788c996f8-phgb4" Jul 15 19:30:06.966384 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:06.965832 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9bbe494d-9024-4dfb-a6c0-b5a74466304b-metrics-cert\") pod \"lws-controller-manager-6788c996f8-phgb4\" (UID: \"9bbe494d-9024-4dfb-a6c0-b5a74466304b\") " pod="openshift-lws-operator/lws-controller-manager-6788c996f8-phgb4" Jul 15 19:30:06.971535 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:06.971508 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-46ttc\" (UniqueName: \"kubernetes.io/projected/9bbe494d-9024-4dfb-a6c0-b5a74466304b-kube-api-access-46ttc\") pod \"lws-controller-manager-6788c996f8-phgb4\" (UID: \"9bbe494d-9024-4dfb-a6c0-b5a74466304b\") " pod="openshift-lws-operator/lws-controller-manager-6788c996f8-phgb4" Jul 15 19:30:07.085801 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:07.085781 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6788c996f8-phgb4" Jul 15 19:30:07.222976 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:07.222940 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6788c996f8-phgb4"] Jul 15 19:30:07.227386 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:30:07.227361 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bbe494d_9024_4dfb_a6c0_b5a74466304b.slice/crio-0083e84aa4587eba06ae5f42e0cc040fe5b5867213a56a1e9d39aa68b9cfdbc4 WatchSource:0}: Error finding container 0083e84aa4587eba06ae5f42e0cc040fe5b5867213a56a1e9d39aa68b9cfdbc4: Status 404 returned error can't find the container with id 0083e84aa4587eba06ae5f42e0cc040fe5b5867213a56a1e9d39aa68b9cfdbc4 Jul 15 19:30:07.891585 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:07.891550 2577 generic.go:356] "Generic (PLEG): container finished" podID="9030baa3-821e-400f-a9e8-5c95a00e0238" containerID="d65f893c9c2a7c206d5ce7a5e51a98dc1ab4420dd3dda2ee55b800035d0e8dd2" exitCode=0 Jul 15 19:30:07.891998 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:07.891630 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4bc8n" event={"ID":"9030baa3-821e-400f-a9e8-5c95a00e0238","Type":"ContainerDied","Data":"d65f893c9c2a7c206d5ce7a5e51a98dc1ab4420dd3dda2ee55b800035d0e8dd2"} Jul 15 19:30:07.892808 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:07.892785 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6788c996f8-phgb4" event={"ID":"9bbe494d-9024-4dfb-a6c0-b5a74466304b","Type":"ContainerStarted","Data":"0083e84aa4587eba06ae5f42e0cc040fe5b5867213a56a1e9d39aa68b9cfdbc4"} Jul 15 19:30:09.024171 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:09.024147 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4bc8n" Jul 15 19:30:09.079197 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:09.079162 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qtz98\" (UniqueName: \"kubernetes.io/projected/9030baa3-821e-400f-a9e8-5c95a00e0238-kube-api-access-qtz98\") pod \"9030baa3-821e-400f-a9e8-5c95a00e0238\" (UID: \"9030baa3-821e-400f-a9e8-5c95a00e0238\") " Jul 15 19:30:09.079403 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:09.079217 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9030baa3-821e-400f-a9e8-5c95a00e0238-util\") pod \"9030baa3-821e-400f-a9e8-5c95a00e0238\" (UID: \"9030baa3-821e-400f-a9e8-5c95a00e0238\") " Jul 15 19:30:09.079403 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:09.079242 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9030baa3-821e-400f-a9e8-5c95a00e0238-bundle\") pod \"9030baa3-821e-400f-a9e8-5c95a00e0238\" (UID: \"9030baa3-821e-400f-a9e8-5c95a00e0238\") " Jul 15 19:30:09.079956 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:09.079931 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9030baa3-821e-400f-a9e8-5c95a00e0238-bundle" (OuterVolumeSpecName: "bundle") pod "9030baa3-821e-400f-a9e8-5c95a00e0238" (UID: "9030baa3-821e-400f-a9e8-5c95a00e0238"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:30:09.081211 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:09.081180 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9030baa3-821e-400f-a9e8-5c95a00e0238-kube-api-access-qtz98" (OuterVolumeSpecName: "kube-api-access-qtz98") pod "9030baa3-821e-400f-a9e8-5c95a00e0238" (UID: "9030baa3-821e-400f-a9e8-5c95a00e0238"). InnerVolumeSpecName "kube-api-access-qtz98". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:30:09.086029 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:09.086003 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9030baa3-821e-400f-a9e8-5c95a00e0238-util" (OuterVolumeSpecName: "util") pod "9030baa3-821e-400f-a9e8-5c95a00e0238" (UID: "9030baa3-821e-400f-a9e8-5c95a00e0238"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:30:09.180454 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:09.180374 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qtz98\" (UniqueName: \"kubernetes.io/projected/9030baa3-821e-400f-a9e8-5c95a00e0238-kube-api-access-qtz98\") on node \"ip-10-0-141-62.ec2.internal\" DevicePath \"\"" Jul 15 19:30:09.180454 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:09.180401 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9030baa3-821e-400f-a9e8-5c95a00e0238-util\") on node \"ip-10-0-141-62.ec2.internal\" DevicePath \"\"" Jul 15 19:30:09.180454 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:09.180410 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9030baa3-821e-400f-a9e8-5c95a00e0238-bundle\") on node \"ip-10-0-141-62.ec2.internal\" DevicePath \"\"" Jul 15 19:30:09.905040 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:09.905009 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4bc8n" Jul 15 19:30:09.905226 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:09.904995 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4bc8n" event={"ID":"9030baa3-821e-400f-a9e8-5c95a00e0238","Type":"ContainerDied","Data":"a81879724ed623b1132bf007e042cb8716087833f6c14ab3a98b9837751947d4"} Jul 15 19:30:09.905226 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:09.905185 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a81879724ed623b1132bf007e042cb8716087833f6c14ab3a98b9837751947d4" Jul 15 19:30:10.911118 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:10.911078 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6788c996f8-phgb4" event={"ID":"9bbe494d-9024-4dfb-a6c0-b5a74466304b","Type":"ContainerStarted","Data":"a92e1df93abaaafca98800c2ee3c68b2199c9f34da9cbbf2f334b3243512cedb"} Jul 15 19:30:10.911485 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:10.911207 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6788c996f8-phgb4" Jul 15 19:30:10.928330 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:10.928276 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6788c996f8-phgb4" podStartSLOduration=1.6553598379999999 podStartE2EDuration="4.928262077s" totalImagesPullingTime="3.272902239s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:30:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:30:10.92736536 +0000 UTC m=+524.504367031" watchObservedRunningTime="2026-07-15 19:30:10.928262077 +0000 UTC m=+524.505263748" Jul 15 19:30:21.510417 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:21.510334 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-df875cfdf-bp4cf"] Jul 15 19:30:21.510949 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:21.510815 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9030baa3-821e-400f-a9e8-5c95a00e0238" containerName="util" Jul 15 19:30:21.510949 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:21.510868 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9030baa3-821e-400f-a9e8-5c95a00e0238" containerName="util" Jul 15 19:30:21.510949 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:21.510886 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9030baa3-821e-400f-a9e8-5c95a00e0238" containerName="pull" Jul 15 19:30:21.510949 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:21.510896 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9030baa3-821e-400f-a9e8-5c95a00e0238" containerName="pull" Jul 15 19:30:21.510949 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:21.510905 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9030baa3-821e-400f-a9e8-5c95a00e0238" containerName="extract" Jul 15 19:30:21.510949 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:21.510914 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9030baa3-821e-400f-a9e8-5c95a00e0238" containerName="extract" Jul 15 19:30:21.511273 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:21.511049 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9030baa3-821e-400f-a9e8-5c95a00e0238" containerName="extract" Jul 15 19:30:21.513151 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:21.513128 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-df875cfdf-bp4cf" Jul 15 19:30:21.524470 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:21.524450 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 15 19:30:21.524573 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:21.524478 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 15 19:30:21.525097 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:21.525082 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 15 19:30:21.526302 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:21.526289 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 15 19:30:21.526851 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:21.526837 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-k7zdf\"" Jul 15 19:30:21.538347 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:21.538326 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-df875cfdf-bp4cf"] Jul 15 19:30:21.587599 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:21.587558 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ce2beebd-634e-404f-83ba-d169ed6f278e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-df875cfdf-bp4cf\" (UID: \"ce2beebd-634e-404f-83ba-d169ed6f278e\") " pod="opendatahub/opendatahub-operator-controller-manager-df875cfdf-bp4cf" Jul 15 19:30:21.587717 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:21.587644 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ce2beebd-634e-404f-83ba-d169ed6f278e-webhook-cert\") pod \"opendatahub-operator-controller-manager-df875cfdf-bp4cf\" (UID: \"ce2beebd-634e-404f-83ba-d169ed6f278e\") " pod="opendatahub/opendatahub-operator-controller-manager-df875cfdf-bp4cf" Jul 15 19:30:21.587717 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:21.587682 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ce2beebd-634e-404f-83ba-d169ed6f278e-manifests\") pod \"opendatahub-operator-controller-manager-df875cfdf-bp4cf\" (UID: \"ce2beebd-634e-404f-83ba-d169ed6f278e\") " pod="opendatahub/opendatahub-operator-controller-manager-df875cfdf-bp4cf" Jul 15 19:30:21.587717 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:21.587697 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ce2beebd-634e-404f-83ba-d169ed6f278e-tmp\") pod \"opendatahub-operator-controller-manager-df875cfdf-bp4cf\" (UID: \"ce2beebd-634e-404f-83ba-d169ed6f278e\") " pod="opendatahub/opendatahub-operator-controller-manager-df875cfdf-bp4cf" Jul 15 19:30:21.587817 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:21.587764 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chnc5\" (UniqueName: \"kubernetes.io/projected/ce2beebd-634e-404f-83ba-d169ed6f278e-kube-api-access-chnc5\") pod \"opendatahub-operator-controller-manager-df875cfdf-bp4cf\" (UID: \"ce2beebd-634e-404f-83ba-d169ed6f278e\") " pod="opendatahub/opendatahub-operator-controller-manager-df875cfdf-bp4cf" Jul 15 19:30:21.688508 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:21.688480 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ce2beebd-634e-404f-83ba-d169ed6f278e-manifests\") pod \"opendatahub-operator-controller-manager-df875cfdf-bp4cf\" (UID: \"ce2beebd-634e-404f-83ba-d169ed6f278e\") " pod="opendatahub/opendatahub-operator-controller-manager-df875cfdf-bp4cf" Jul 15 19:30:21.688508 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:21.688509 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ce2beebd-634e-404f-83ba-d169ed6f278e-tmp\") pod \"opendatahub-operator-controller-manager-df875cfdf-bp4cf\" (UID: \"ce2beebd-634e-404f-83ba-d169ed6f278e\") " pod="opendatahub/opendatahub-operator-controller-manager-df875cfdf-bp4cf" Jul 15 19:30:21.688673 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:21.688544 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chnc5\" (UniqueName: \"kubernetes.io/projected/ce2beebd-634e-404f-83ba-d169ed6f278e-kube-api-access-chnc5\") pod \"opendatahub-operator-controller-manager-df875cfdf-bp4cf\" (UID: \"ce2beebd-634e-404f-83ba-d169ed6f278e\") " pod="opendatahub/opendatahub-operator-controller-manager-df875cfdf-bp4cf" Jul 15 19:30:21.688673 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:21.688587 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ce2beebd-634e-404f-83ba-d169ed6f278e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-df875cfdf-bp4cf\" (UID: \"ce2beebd-634e-404f-83ba-d169ed6f278e\") " pod="opendatahub/opendatahub-operator-controller-manager-df875cfdf-bp4cf" Jul 15 19:30:21.688673 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:21.688635 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ce2beebd-634e-404f-83ba-d169ed6f278e-webhook-cert\") pod \"opendatahub-operator-controller-manager-df875cfdf-bp4cf\" (UID: \"ce2beebd-634e-404f-83ba-d169ed6f278e\") " pod="opendatahub/opendatahub-operator-controller-manager-df875cfdf-bp4cf" Jul 15 19:30:21.688918 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:21.688895 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ce2beebd-634e-404f-83ba-d169ed6f278e-manifests\") pod \"opendatahub-operator-controller-manager-df875cfdf-bp4cf\" (UID: \"ce2beebd-634e-404f-83ba-d169ed6f278e\") " pod="opendatahub/opendatahub-operator-controller-manager-df875cfdf-bp4cf" Jul 15 19:30:21.690919 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:21.690891 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ce2beebd-634e-404f-83ba-d169ed6f278e-tmp\") pod \"opendatahub-operator-controller-manager-df875cfdf-bp4cf\" (UID: \"ce2beebd-634e-404f-83ba-d169ed6f278e\") " pod="opendatahub/opendatahub-operator-controller-manager-df875cfdf-bp4cf" Jul 15 19:30:21.691040 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:21.691022 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ce2beebd-634e-404f-83ba-d169ed6f278e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-df875cfdf-bp4cf\" (UID: \"ce2beebd-634e-404f-83ba-d169ed6f278e\") " pod="opendatahub/opendatahub-operator-controller-manager-df875cfdf-bp4cf" Jul 15 19:30:21.691082 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:21.691026 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ce2beebd-634e-404f-83ba-d169ed6f278e-webhook-cert\") pod \"opendatahub-operator-controller-manager-df875cfdf-bp4cf\" (UID: \"ce2beebd-634e-404f-83ba-d169ed6f278e\") " pod="opendatahub/opendatahub-operator-controller-manager-df875cfdf-bp4cf" Jul 15 19:30:21.697545 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:21.697529 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chnc5\" (UniqueName: \"kubernetes.io/projected/ce2beebd-634e-404f-83ba-d169ed6f278e-kube-api-access-chnc5\") pod \"opendatahub-operator-controller-manager-df875cfdf-bp4cf\" (UID: \"ce2beebd-634e-404f-83ba-d169ed6f278e\") " pod="opendatahub/opendatahub-operator-controller-manager-df875cfdf-bp4cf" Jul 15 19:30:21.822242 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:21.822211 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-df875cfdf-bp4cf" Jul 15 19:30:21.918151 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:21.918060 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6788c996f8-phgb4" Jul 15 19:30:21.994094 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:21.994071 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-df875cfdf-bp4cf"] Jul 15 19:30:21.997095 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:30:21.997067 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce2beebd_634e_404f_83ba_d169ed6f278e.slice/crio-4c2727de43555cbfd253a0f8cd1290c5be01fdab0ef4fa2508b8acf77cdd8dc7 WatchSource:0}: Error finding container 4c2727de43555cbfd253a0f8cd1290c5be01fdab0ef4fa2508b8acf77cdd8dc7: Status 404 returned error can't find the container with id 4c2727de43555cbfd253a0f8cd1290c5be01fdab0ef4fa2508b8acf77cdd8dc7 Jul 15 19:30:22.967452 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:22.967411 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-df875cfdf-bp4cf" event={"ID":"ce2beebd-634e-404f-83ba-d169ed6f278e","Type":"ContainerStarted","Data":"4c2727de43555cbfd253a0f8cd1290c5be01fdab0ef4fa2508b8acf77cdd8dc7"} Jul 15 19:30:25.984392 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:25.984349 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-df875cfdf-bp4cf" event={"ID":"ce2beebd-634e-404f-83ba-d169ed6f278e","Type":"ContainerStarted","Data":"2a282a8d3cd128d54577dffab64b48071b79e5e820986ff2440541ec45d7b27c"} Jul 15 19:30:26.990543 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:26.990512 2577 generic.go:356] "Generic (PLEG): container finished" podID="ce2beebd-634e-404f-83ba-d169ed6f278e" containerID="2a282a8d3cd128d54577dffab64b48071b79e5e820986ff2440541ec45d7b27c" exitCode=0 Jul 15 19:30:26.990882 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:26.990598 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-df875cfdf-bp4cf" event={"ID":"ce2beebd-634e-404f-83ba-d169ed6f278e","Type":"ContainerDied","Data":"2a282a8d3cd128d54577dffab64b48071b79e5e820986ff2440541ec45d7b27c"} Jul 15 19:30:27.996826 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:27.996789 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-df875cfdf-bp4cf" event={"ID":"ce2beebd-634e-404f-83ba-d169ed6f278e","Type":"ContainerStarted","Data":"23db788b44b506eae1f5f1944fdb4eebcf5f756c86ed6f5a78371f66bb00778e"} Jul 15 19:30:27.997235 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:27.996902 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-df875cfdf-bp4cf" Jul 15 19:30:28.017753 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:28.017654 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-df875cfdf-bp4cf" podStartSLOduration=2.34450474 podStartE2EDuration="7.017636631s" totalImagesPullingTime="3.581401515s" totalInitContainerRuntime="1.091730376s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:30:21 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:30:28.017427538 +0000 UTC m=+541.594429216" watchObservedRunningTime="2026-07-15 19:30:28.017636631 +0000 UTC m=+541.594638302" Jul 15 19:30:39.003759 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:39.003728 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-df875cfdf-bp4cf" Jul 15 19:30:41.984503 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:41.984470 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6d4465d888-q94gr"] Jul 15 19:30:41.995004 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:41.994981 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6d4465d888-q94gr" Jul 15 19:30:41.997784 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:41.997760 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-ckp6x\"" Jul 15 19:30:41.997912 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:41.997763 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 15 19:30:41.998110 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:41.998096 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 15 19:30:42.009212 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:42.009186 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6d4465d888-q94gr"] Jul 15 19:30:42.053488 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:42.053465 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/44825d03-933d-4f48-8c7d-986f153def49-tmp\") pod \"kube-auth-proxy-6d4465d888-q94gr\" (UID: \"44825d03-933d-4f48-8c7d-986f153def49\") " pod="openshift-ingress/kube-auth-proxy-6d4465d888-q94gr" Jul 15 19:30:42.053602 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:42.053501 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lctx2\" (UniqueName: \"kubernetes.io/projected/44825d03-933d-4f48-8c7d-986f153def49-kube-api-access-lctx2\") pod \"kube-auth-proxy-6d4465d888-q94gr\" (UID: \"44825d03-933d-4f48-8c7d-986f153def49\") " pod="openshift-ingress/kube-auth-proxy-6d4465d888-q94gr" Jul 15 19:30:42.053651 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:42.053636 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44825d03-933d-4f48-8c7d-986f153def49-tls-certs\") pod \"kube-auth-proxy-6d4465d888-q94gr\" (UID: \"44825d03-933d-4f48-8c7d-986f153def49\") " pod="openshift-ingress/kube-auth-proxy-6d4465d888-q94gr" Jul 15 19:30:42.154246 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:42.154219 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44825d03-933d-4f48-8c7d-986f153def49-tls-certs\") pod \"kube-auth-proxy-6d4465d888-q94gr\" (UID: \"44825d03-933d-4f48-8c7d-986f153def49\") " pod="openshift-ingress/kube-auth-proxy-6d4465d888-q94gr" Jul 15 19:30:42.154450 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:42.154259 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/44825d03-933d-4f48-8c7d-986f153def49-tmp\") pod \"kube-auth-proxy-6d4465d888-q94gr\" (UID: \"44825d03-933d-4f48-8c7d-986f153def49\") " pod="openshift-ingress/kube-auth-proxy-6d4465d888-q94gr" Jul 15 19:30:42.154450 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:42.154289 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lctx2\" (UniqueName: \"kubernetes.io/projected/44825d03-933d-4f48-8c7d-986f153def49-kube-api-access-lctx2\") pod \"kube-auth-proxy-6d4465d888-q94gr\" (UID: \"44825d03-933d-4f48-8c7d-986f153def49\") " pod="openshift-ingress/kube-auth-proxy-6d4465d888-q94gr" Jul 15 19:30:42.156621 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:42.156595 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/44825d03-933d-4f48-8c7d-986f153def49-tmp\") pod \"kube-auth-proxy-6d4465d888-q94gr\" (UID: \"44825d03-933d-4f48-8c7d-986f153def49\") " pod="openshift-ingress/kube-auth-proxy-6d4465d888-q94gr" Jul 15 19:30:42.156826 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:42.156807 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/44825d03-933d-4f48-8c7d-986f153def49-tls-certs\") pod \"kube-auth-proxy-6d4465d888-q94gr\" (UID: \"44825d03-933d-4f48-8c7d-986f153def49\") " pod="openshift-ingress/kube-auth-proxy-6d4465d888-q94gr" Jul 15 19:30:42.163687 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:42.163665 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lctx2\" (UniqueName: \"kubernetes.io/projected/44825d03-933d-4f48-8c7d-986f153def49-kube-api-access-lctx2\") pod \"kube-auth-proxy-6d4465d888-q94gr\" (UID: \"44825d03-933d-4f48-8c7d-986f153def49\") " pod="openshift-ingress/kube-auth-proxy-6d4465d888-q94gr" Jul 15 19:30:42.305185 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:42.305158 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6d4465d888-q94gr" Jul 15 19:30:42.442984 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:42.442960 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6d4465d888-q94gr"] Jul 15 19:30:42.445237 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:30:42.445205 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44825d03_933d_4f48_8c7d_986f153def49.slice/crio-9b134f2ad44b61ce8bd0bcbcbb5c449a323adf3babc0b5609177fb55ffc92574 WatchSource:0}: Error finding container 9b134f2ad44b61ce8bd0bcbcbb5c449a323adf3babc0b5609177fb55ffc92574: Status 404 returned error can't find the container with id 9b134f2ad44b61ce8bd0bcbcbb5c449a323adf3babc0b5609177fb55ffc92574 Jul 15 19:30:43.074974 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:43.074939 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6d4465d888-q94gr" event={"ID":"44825d03-933d-4f48-8c7d-986f153def49","Type":"ContainerStarted","Data":"9b134f2ad44b61ce8bd0bcbcbb5c449a323adf3babc0b5609177fb55ffc92574"} Jul 15 19:30:47.094565 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:47.094535 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6d4465d888-q94gr" event={"ID":"44825d03-933d-4f48-8c7d-986f153def49","Type":"ContainerStarted","Data":"037a7c7f1eb3e8fb581f8f478e8a0763a033b90d7903376c436098e139a1f1d2"} Jul 15 19:30:47.116034 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:30:47.115985 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6d4465d888-q94gr" podStartSLOduration=2.500368347 podStartE2EDuration="6.115969675s" totalImagesPullingTime="3.615601328s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:30:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:30:47.114639109 +0000 UTC m=+560.691640780" watchObservedRunningTime="2026-07-15 19:30:47.115969675 +0000 UTC m=+560.692971355" Jul 15 19:31:26.895667 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:26.895637 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-j2rcv_a2ded0a5-37e2-48fd-98b6-3d64ae279896/console-operator/1.log" Jul 15 19:31:26.896191 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:26.895861 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-j2rcv_a2ded0a5-37e2-48fd-98b6-3d64ae279896/console-operator/1.log" Jul 15 19:31:26.902625 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:26.902606 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fw6vl_89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21/ovn-acl-logging/0.log" Jul 15 19:31:26.903254 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:26.903234 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fw6vl_89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21/ovn-acl-logging/0.log" Jul 15 19:31:58.045138 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:58.045067 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l46z4"] Jul 15 19:31:58.047566 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:58.047548 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l46z4" Jul 15 19:31:58.049736 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:58.049714 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 15 19:31:58.049909 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:58.049892 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-drc2n\"" Jul 15 19:31:58.050592 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:58.050576 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 15 19:31:58.056960 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:58.056934 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l46z4"] Jul 15 19:31:58.148153 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:58.148118 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/35defb98-7604-4ea6-ac0c-193c0cb0bed0-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l46z4\" (UID: \"35defb98-7604-4ea6-ac0c-193c0cb0bed0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l46z4" Jul 15 19:31:58.148304 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:58.148192 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmf6p\" (UniqueName: \"kubernetes.io/projected/35defb98-7604-4ea6-ac0c-193c0cb0bed0-kube-api-access-lmf6p\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l46z4\" (UID: \"35defb98-7604-4ea6-ac0c-193c0cb0bed0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l46z4" Jul 15 19:31:58.148304 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:58.148218 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/35defb98-7604-4ea6-ac0c-193c0cb0bed0-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l46z4\" (UID: \"35defb98-7604-4ea6-ac0c-193c0cb0bed0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l46z4" Jul 15 19:31:58.249371 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:58.249308 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/35defb98-7604-4ea6-ac0c-193c0cb0bed0-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l46z4\" (UID: \"35defb98-7604-4ea6-ac0c-193c0cb0bed0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l46z4" Jul 15 19:31:58.249530 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:58.249405 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmf6p\" (UniqueName: \"kubernetes.io/projected/35defb98-7604-4ea6-ac0c-193c0cb0bed0-kube-api-access-lmf6p\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l46z4\" (UID: \"35defb98-7604-4ea6-ac0c-193c0cb0bed0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l46z4" Jul 15 19:31:58.249530 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:58.249439 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/35defb98-7604-4ea6-ac0c-193c0cb0bed0-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l46z4\" (UID: \"35defb98-7604-4ea6-ac0c-193c0cb0bed0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l46z4" Jul 15 19:31:58.249705 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:58.249686 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/35defb98-7604-4ea6-ac0c-193c0cb0bed0-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l46z4\" (UID: \"35defb98-7604-4ea6-ac0c-193c0cb0bed0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l46z4" Jul 15 19:31:58.249806 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:58.249789 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/35defb98-7604-4ea6-ac0c-193c0cb0bed0-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l46z4\" (UID: \"35defb98-7604-4ea6-ac0c-193c0cb0bed0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l46z4" Jul 15 19:31:58.258534 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:58.258507 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmf6p\" (UniqueName: \"kubernetes.io/projected/35defb98-7604-4ea6-ac0c-193c0cb0bed0-kube-api-access-lmf6p\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l46z4\" (UID: \"35defb98-7604-4ea6-ac0c-193c0cb0bed0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l46z4" Jul 15 19:31:58.356979 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:58.356954 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l46z4" Jul 15 19:31:58.488237 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:58.488211 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l46z4"] Jul 15 19:31:58.490609 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:31:58.490580 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35defb98_7604_4ea6_ac0c_193c0cb0bed0.slice/crio-2583e51a16131b75c71e06a0cc076a9c75d2f5e99466c120757d5406c904c3bc WatchSource:0}: Error finding container 2583e51a16131b75c71e06a0cc076a9c75d2f5e99466c120757d5406c904c3bc: Status 404 returned error can't find the container with id 2583e51a16131b75c71e06a0cc076a9c75d2f5e99466c120757d5406c904c3bc Jul 15 19:31:58.648343 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:58.648271 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cpkdr"] Jul 15 19:31:58.650824 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:58.650808 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cpkdr" Jul 15 19:31:58.661797 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:58.661774 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cpkdr"] Jul 15 19:31:58.754009 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:58.753983 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7841fdc5-da16-4565-bb38-324b24b3916c-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cpkdr\" (UID: \"7841fdc5-da16-4565-bb38-324b24b3916c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cpkdr" Jul 15 19:31:58.754123 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:58.754025 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpckl\" (UniqueName: \"kubernetes.io/projected/7841fdc5-da16-4565-bb38-324b24b3916c-kube-api-access-cpckl\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cpkdr\" (UID: \"7841fdc5-da16-4565-bb38-324b24b3916c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cpkdr" Jul 15 19:31:58.754123 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:58.754094 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7841fdc5-da16-4565-bb38-324b24b3916c-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cpkdr\" (UID: \"7841fdc5-da16-4565-bb38-324b24b3916c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cpkdr" Jul 15 19:31:58.854980 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:58.854960 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7841fdc5-da16-4565-bb38-324b24b3916c-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cpkdr\" (UID: \"7841fdc5-da16-4565-bb38-324b24b3916c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cpkdr" Jul 15 19:31:58.855099 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:58.854993 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpckl\" (UniqueName: \"kubernetes.io/projected/7841fdc5-da16-4565-bb38-324b24b3916c-kube-api-access-cpckl\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cpkdr\" (UID: \"7841fdc5-da16-4565-bb38-324b24b3916c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cpkdr" Jul 15 19:31:58.855154 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:58.855120 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7841fdc5-da16-4565-bb38-324b24b3916c-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cpkdr\" (UID: \"7841fdc5-da16-4565-bb38-324b24b3916c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cpkdr" Jul 15 19:31:58.855374 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:58.855355 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7841fdc5-da16-4565-bb38-324b24b3916c-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cpkdr\" (UID: \"7841fdc5-da16-4565-bb38-324b24b3916c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cpkdr" Jul 15 19:31:58.855422 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:58.855406 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7841fdc5-da16-4565-bb38-324b24b3916c-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cpkdr\" (UID: \"7841fdc5-da16-4565-bb38-324b24b3916c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cpkdr" Jul 15 19:31:58.864415 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:58.864389 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpckl\" (UniqueName: \"kubernetes.io/projected/7841fdc5-da16-4565-bb38-324b24b3916c-kube-api-access-cpckl\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cpkdr\" (UID: \"7841fdc5-da16-4565-bb38-324b24b3916c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cpkdr" Jul 15 19:31:58.973730 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:58.973676 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cpkdr" Jul 15 19:31:59.042834 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:59.042806 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqnwm"] Jul 15 19:31:59.045522 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:59.045501 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqnwm" Jul 15 19:31:59.056706 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:59.056683 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqnwm"] Jul 15 19:31:59.107756 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:59.107731 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cpkdr"] Jul 15 19:31:59.157467 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:59.157446 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4ec38921-1795-454a-9617-7a9b40cf7ab1-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqnwm\" (UID: \"4ec38921-1795-454a-9617-7a9b40cf7ab1\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqnwm" Jul 15 19:31:59.157558 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:59.157477 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgsdq\" (UniqueName: \"kubernetes.io/projected/4ec38921-1795-454a-9617-7a9b40cf7ab1-kube-api-access-vgsdq\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqnwm\" (UID: \"4ec38921-1795-454a-9617-7a9b40cf7ab1\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqnwm" Jul 15 19:31:59.157558 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:59.157545 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4ec38921-1795-454a-9617-7a9b40cf7ab1-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqnwm\" (UID: \"4ec38921-1795-454a-9617-7a9b40cf7ab1\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqnwm" Jul 15 19:31:59.258782 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:59.258721 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4ec38921-1795-454a-9617-7a9b40cf7ab1-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqnwm\" (UID: \"4ec38921-1795-454a-9617-7a9b40cf7ab1\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqnwm" Jul 15 19:31:59.258782 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:59.258760 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgsdq\" (UniqueName: \"kubernetes.io/projected/4ec38921-1795-454a-9617-7a9b40cf7ab1-kube-api-access-vgsdq\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqnwm\" (UID: \"4ec38921-1795-454a-9617-7a9b40cf7ab1\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqnwm" Jul 15 19:31:59.258924 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:59.258815 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4ec38921-1795-454a-9617-7a9b40cf7ab1-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqnwm\" (UID: \"4ec38921-1795-454a-9617-7a9b40cf7ab1\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqnwm" Jul 15 19:31:59.259164 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:59.259140 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4ec38921-1795-454a-9617-7a9b40cf7ab1-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqnwm\" (UID: \"4ec38921-1795-454a-9617-7a9b40cf7ab1\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqnwm" Jul 15 19:31:59.259229 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:59.259197 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4ec38921-1795-454a-9617-7a9b40cf7ab1-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqnwm\" (UID: \"4ec38921-1795-454a-9617-7a9b40cf7ab1\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqnwm" Jul 15 19:31:59.268621 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:59.268594 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgsdq\" (UniqueName: \"kubernetes.io/projected/4ec38921-1795-454a-9617-7a9b40cf7ab1-kube-api-access-vgsdq\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqnwm\" (UID: \"4ec38921-1795-454a-9617-7a9b40cf7ab1\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqnwm" Jul 15 19:31:59.355262 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:59.355237 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqnwm" Jul 15 19:31:59.420078 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:59.420046 2577 generic.go:356] "Generic (PLEG): container finished" podID="7841fdc5-da16-4565-bb38-324b24b3916c" containerID="23a42df612b3d833cb1cc7b89b07a6fa46200dfcd5220ec5a4bb422fac093b3b" exitCode=0 Jul 15 19:31:59.420209 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:59.420077 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cpkdr" event={"ID":"7841fdc5-da16-4565-bb38-324b24b3916c","Type":"ContainerDied","Data":"23a42df612b3d833cb1cc7b89b07a6fa46200dfcd5220ec5a4bb422fac093b3b"} Jul 15 19:31:59.420209 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:59.420128 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cpkdr" event={"ID":"7841fdc5-da16-4565-bb38-324b24b3916c","Type":"ContainerStarted","Data":"1ff3cf44ed6c1584746aafa1813216a13302d9242c3999cbbd2a7d42c88618fc"} Jul 15 19:31:59.420635 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:59.420619 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 19:31:59.421855 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:59.421832 2577 generic.go:356] "Generic (PLEG): container finished" podID="35defb98-7604-4ea6-ac0c-193c0cb0bed0" containerID="4d7b932d03731b18322d52f6ddef7cef685e2a1cc9a630f44bc35129867c479e" exitCode=0 Jul 15 19:31:59.421977 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:59.421914 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l46z4" event={"ID":"35defb98-7604-4ea6-ac0c-193c0cb0bed0","Type":"ContainerDied","Data":"4d7b932d03731b18322d52f6ddef7cef685e2a1cc9a630f44bc35129867c479e"} Jul 15 19:31:59.421977 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:59.421953 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l46z4" event={"ID":"35defb98-7604-4ea6-ac0c-193c0cb0bed0","Type":"ContainerStarted","Data":"2583e51a16131b75c71e06a0cc076a9c75d2f5e99466c120757d5406c904c3bc"} Jul 15 19:31:59.454740 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:59.454713 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t86m4"] Jul 15 19:31:59.458107 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:59.458086 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t86m4" Jul 15 19:31:59.471684 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:59.471597 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t86m4"] Jul 15 19:31:59.491782 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:59.491683 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqnwm"] Jul 15 19:31:59.494223 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:31:59.494198 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ec38921_1795_454a_9617_7a9b40cf7ab1.slice/crio-36e80bdf17a9530ccea724b8d92d024fb4682d396584bcf4818174f65344bbd6 WatchSource:0}: Error finding container 36e80bdf17a9530ccea724b8d92d024fb4682d396584bcf4818174f65344bbd6: Status 404 returned error can't find the container with id 36e80bdf17a9530ccea724b8d92d024fb4682d396584bcf4818174f65344bbd6 Jul 15 19:31:59.561808 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:59.561780 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8ad06c01-d62c-4d59-b1b4-c77854454d60-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t86m4\" (UID: \"8ad06c01-d62c-4d59-b1b4-c77854454d60\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t86m4" Jul 15 19:31:59.561909 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:59.561834 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6hss\" (UniqueName: \"kubernetes.io/projected/8ad06c01-d62c-4d59-b1b4-c77854454d60-kube-api-access-x6hss\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t86m4\" (UID: \"8ad06c01-d62c-4d59-b1b4-c77854454d60\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t86m4" Jul 15 19:31:59.561909 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:59.561866 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8ad06c01-d62c-4d59-b1b4-c77854454d60-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t86m4\" (UID: \"8ad06c01-d62c-4d59-b1b4-c77854454d60\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t86m4" Jul 15 19:31:59.663044 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:59.663013 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8ad06c01-d62c-4d59-b1b4-c77854454d60-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t86m4\" (UID: \"8ad06c01-d62c-4d59-b1b4-c77854454d60\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t86m4" Jul 15 19:31:59.663160 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:59.663066 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6hss\" (UniqueName: \"kubernetes.io/projected/8ad06c01-d62c-4d59-b1b4-c77854454d60-kube-api-access-x6hss\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t86m4\" (UID: \"8ad06c01-d62c-4d59-b1b4-c77854454d60\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t86m4" Jul 15 19:31:59.663160 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:59.663114 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8ad06c01-d62c-4d59-b1b4-c77854454d60-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t86m4\" (UID: \"8ad06c01-d62c-4d59-b1b4-c77854454d60\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t86m4" Jul 15 19:31:59.663415 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:59.663394 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8ad06c01-d62c-4d59-b1b4-c77854454d60-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t86m4\" (UID: \"8ad06c01-d62c-4d59-b1b4-c77854454d60\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t86m4" Jul 15 19:31:59.663490 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:59.663445 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8ad06c01-d62c-4d59-b1b4-c77854454d60-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t86m4\" (UID: \"8ad06c01-d62c-4d59-b1b4-c77854454d60\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t86m4" Jul 15 19:31:59.671490 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:59.671472 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6hss\" (UniqueName: \"kubernetes.io/projected/8ad06c01-d62c-4d59-b1b4-c77854454d60-kube-api-access-x6hss\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t86m4\" (UID: \"8ad06c01-d62c-4d59-b1b4-c77854454d60\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t86m4" Jul 15 19:31:59.768441 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:59.768409 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t86m4" Jul 15 19:31:59.896401 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:31:59.896372 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t86m4"] Jul 15 19:31:59.898825 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:31:59.898794 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ad06c01_d62c_4d59_b1b4_c77854454d60.slice/crio-4b43a94187ed42a7c7531f647bd120b28e1239c049be4ea87bd3767898ac13e4 WatchSource:0}: Error finding container 4b43a94187ed42a7c7531f647bd120b28e1239c049be4ea87bd3767898ac13e4: Status 404 returned error can't find the container with id 4b43a94187ed42a7c7531f647bd120b28e1239c049be4ea87bd3767898ac13e4 Jul 15 19:32:00.427812 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:00.427781 2577 generic.go:356] "Generic (PLEG): container finished" podID="8ad06c01-d62c-4d59-b1b4-c77854454d60" containerID="fedd098f73919727841e832daea6958702baff3fc7c3faac950a5443f0f993a0" exitCode=0 Jul 15 19:32:00.428216 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:00.427857 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t86m4" event={"ID":"8ad06c01-d62c-4d59-b1b4-c77854454d60","Type":"ContainerDied","Data":"fedd098f73919727841e832daea6958702baff3fc7c3faac950a5443f0f993a0"} Jul 15 19:32:00.428216 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:00.427902 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t86m4" event={"ID":"8ad06c01-d62c-4d59-b1b4-c77854454d60","Type":"ContainerStarted","Data":"4b43a94187ed42a7c7531f647bd120b28e1239c049be4ea87bd3767898ac13e4"} Jul 15 19:32:00.429456 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:00.429433 2577 generic.go:356] "Generic (PLEG): container finished" podID="4ec38921-1795-454a-9617-7a9b40cf7ab1" containerID="b9c7b16ee6beed4e356f55bba41766b100b9edddb030590aad863e5237f89ae3" exitCode=0 Jul 15 19:32:00.429524 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:00.429498 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqnwm" event={"ID":"4ec38921-1795-454a-9617-7a9b40cf7ab1","Type":"ContainerDied","Data":"b9c7b16ee6beed4e356f55bba41766b100b9edddb030590aad863e5237f89ae3"} Jul 15 19:32:00.429561 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:00.429533 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqnwm" event={"ID":"4ec38921-1795-454a-9617-7a9b40cf7ab1","Type":"ContainerStarted","Data":"36e80bdf17a9530ccea724b8d92d024fb4682d396584bcf4818174f65344bbd6"} Jul 15 19:32:01.436152 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:01.436120 2577 generic.go:356] "Generic (PLEG): container finished" podID="35defb98-7604-4ea6-ac0c-193c0cb0bed0" containerID="4070dfefe55a5b3bdcfeaa9f5afd3f140cc8954f23fa7d0c3fe9c5332671c2fc" exitCode=0 Jul 15 19:32:01.436566 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:01.436233 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l46z4" event={"ID":"35defb98-7604-4ea6-ac0c-193c0cb0bed0","Type":"ContainerDied","Data":"4070dfefe55a5b3bdcfeaa9f5afd3f140cc8954f23fa7d0c3fe9c5332671c2fc"} Jul 15 19:32:01.439156 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:01.439135 2577 generic.go:356] "Generic (PLEG): container finished" podID="7841fdc5-da16-4565-bb38-324b24b3916c" containerID="c9a92250c388d1d673707d0f14e17e931c1e5e7a8cc478f301e36f6681dc98df" exitCode=0 Jul 15 19:32:01.439263 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:01.439172 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cpkdr" event={"ID":"7841fdc5-da16-4565-bb38-324b24b3916c","Type":"ContainerDied","Data":"c9a92250c388d1d673707d0f14e17e931c1e5e7a8cc478f301e36f6681dc98df"} Jul 15 19:32:02.449083 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:02.449050 2577 generic.go:356] "Generic (PLEG): container finished" podID="7841fdc5-da16-4565-bb38-324b24b3916c" containerID="b7198b5c8100e44694c1b1ee7f015968f9e2e02b4034e1c41a16347d2a561b89" exitCode=0 Jul 15 19:32:02.449472 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:02.449130 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cpkdr" event={"ID":"7841fdc5-da16-4565-bb38-324b24b3916c","Type":"ContainerDied","Data":"b7198b5c8100e44694c1b1ee7f015968f9e2e02b4034e1c41a16347d2a561b89"} Jul 15 19:32:02.450988 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:02.450964 2577 generic.go:356] "Generic (PLEG): container finished" podID="35defb98-7604-4ea6-ac0c-193c0cb0bed0" containerID="60242ad10b6e155fa9aa085bf999eda169c5c2f479151f5a157aadb3ae92c80e" exitCode=0 Jul 15 19:32:02.451112 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:02.451047 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l46z4" event={"ID":"35defb98-7604-4ea6-ac0c-193c0cb0bed0","Type":"ContainerDied","Data":"60242ad10b6e155fa9aa085bf999eda169c5c2f479151f5a157aadb3ae92c80e"} Jul 15 19:32:02.452566 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:02.452549 2577 generic.go:356] "Generic (PLEG): container finished" podID="8ad06c01-d62c-4d59-b1b4-c77854454d60" containerID="3f7abc3d2eb6ab2ab936931f558af4427a31e36d576b9ce04ec8652a822156da" exitCode=0 Jul 15 19:32:02.452646 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:02.452593 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t86m4" event={"ID":"8ad06c01-d62c-4d59-b1b4-c77854454d60","Type":"ContainerDied","Data":"3f7abc3d2eb6ab2ab936931f558af4427a31e36d576b9ce04ec8652a822156da"} Jul 15 19:32:02.454360 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:02.454337 2577 generic.go:356] "Generic (PLEG): container finished" podID="4ec38921-1795-454a-9617-7a9b40cf7ab1" containerID="2a8449c291a19111cb81fff05ba1d2c7dede989138cd577908ee3a1d727fbca5" exitCode=0 Jul 15 19:32:02.454469 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:02.454359 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqnwm" event={"ID":"4ec38921-1795-454a-9617-7a9b40cf7ab1","Type":"ContainerDied","Data":"2a8449c291a19111cb81fff05ba1d2c7dede989138cd577908ee3a1d727fbca5"} Jul 15 19:32:03.461454 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:03.461412 2577 generic.go:356] "Generic (PLEG): container finished" podID="8ad06c01-d62c-4d59-b1b4-c77854454d60" containerID="654065afebe7814bfc4b4f1554086372e898f3d3026f274ecaa6418663756fe8" exitCode=0 Jul 15 19:32:03.461863 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:03.461496 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t86m4" event={"ID":"8ad06c01-d62c-4d59-b1b4-c77854454d60","Type":"ContainerDied","Data":"654065afebe7814bfc4b4f1554086372e898f3d3026f274ecaa6418663756fe8"} Jul 15 19:32:03.463386 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:03.463365 2577 generic.go:356] "Generic (PLEG): container finished" podID="4ec38921-1795-454a-9617-7a9b40cf7ab1" containerID="f503c7a2dc040c930076c388a804d08c8a6a98dc34c70939792d380a1da9d8eb" exitCode=0 Jul 15 19:32:03.463483 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:03.463453 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqnwm" event={"ID":"4ec38921-1795-454a-9617-7a9b40cf7ab1","Type":"ContainerDied","Data":"f503c7a2dc040c930076c388a804d08c8a6a98dc34c70939792d380a1da9d8eb"} Jul 15 19:32:03.623366 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:03.623342 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cpkdr" Jul 15 19:32:03.626657 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:03.626637 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l46z4" Jul 15 19:32:03.696571 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:03.696544 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/35defb98-7604-4ea6-ac0c-193c0cb0bed0-bundle\") pod \"35defb98-7604-4ea6-ac0c-193c0cb0bed0\" (UID: \"35defb98-7604-4ea6-ac0c-193c0cb0bed0\") " Jul 15 19:32:03.696571 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:03.696574 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7841fdc5-da16-4565-bb38-324b24b3916c-bundle\") pod \"7841fdc5-da16-4565-bb38-324b24b3916c\" (UID: \"7841fdc5-da16-4565-bb38-324b24b3916c\") " Jul 15 19:32:03.696751 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:03.696594 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7841fdc5-da16-4565-bb38-324b24b3916c-util\") pod \"7841fdc5-da16-4565-bb38-324b24b3916c\" (UID: \"7841fdc5-da16-4565-bb38-324b24b3916c\") " Jul 15 19:32:03.696751 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:03.696621 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cpckl\" (UniqueName: \"kubernetes.io/projected/7841fdc5-da16-4565-bb38-324b24b3916c-kube-api-access-cpckl\") pod \"7841fdc5-da16-4565-bb38-324b24b3916c\" (UID: \"7841fdc5-da16-4565-bb38-324b24b3916c\") " Jul 15 19:32:03.696751 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:03.696680 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lmf6p\" (UniqueName: \"kubernetes.io/projected/35defb98-7604-4ea6-ac0c-193c0cb0bed0-kube-api-access-lmf6p\") pod \"35defb98-7604-4ea6-ac0c-193c0cb0bed0\" (UID: \"35defb98-7604-4ea6-ac0c-193c0cb0bed0\") " Jul 15 19:32:03.696751 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:03.696707 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/35defb98-7604-4ea6-ac0c-193c0cb0bed0-util\") pod \"35defb98-7604-4ea6-ac0c-193c0cb0bed0\" (UID: \"35defb98-7604-4ea6-ac0c-193c0cb0bed0\") " Jul 15 19:32:03.697209 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:03.697121 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7841fdc5-da16-4565-bb38-324b24b3916c-bundle" (OuterVolumeSpecName: "bundle") pod "7841fdc5-da16-4565-bb38-324b24b3916c" (UID: "7841fdc5-da16-4565-bb38-324b24b3916c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:32:03.697209 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:03.697149 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35defb98-7604-4ea6-ac0c-193c0cb0bed0-bundle" (OuterVolumeSpecName: "bundle") pod "35defb98-7604-4ea6-ac0c-193c0cb0bed0" (UID: "35defb98-7604-4ea6-ac0c-193c0cb0bed0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:32:03.698859 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:03.698830 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7841fdc5-da16-4565-bb38-324b24b3916c-kube-api-access-cpckl" (OuterVolumeSpecName: "kube-api-access-cpckl") pod "7841fdc5-da16-4565-bb38-324b24b3916c" (UID: "7841fdc5-da16-4565-bb38-324b24b3916c"). InnerVolumeSpecName "kube-api-access-cpckl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:32:03.699140 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:03.699115 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35defb98-7604-4ea6-ac0c-193c0cb0bed0-kube-api-access-lmf6p" (OuterVolumeSpecName: "kube-api-access-lmf6p") pod "35defb98-7604-4ea6-ac0c-193c0cb0bed0" (UID: "35defb98-7604-4ea6-ac0c-193c0cb0bed0"). InnerVolumeSpecName "kube-api-access-lmf6p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:32:03.702955 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:03.702932 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7841fdc5-da16-4565-bb38-324b24b3916c-util" (OuterVolumeSpecName: "util") pod "7841fdc5-da16-4565-bb38-324b24b3916c" (UID: "7841fdc5-da16-4565-bb38-324b24b3916c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:32:03.703059 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:03.702953 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35defb98-7604-4ea6-ac0c-193c0cb0bed0-util" (OuterVolumeSpecName: "util") pod "35defb98-7604-4ea6-ac0c-193c0cb0bed0" (UID: "35defb98-7604-4ea6-ac0c-193c0cb0bed0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:32:03.798195 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:03.798124 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/35defb98-7604-4ea6-ac0c-193c0cb0bed0-bundle\") on node \"ip-10-0-141-62.ec2.internal\" DevicePath \"\"" Jul 15 19:32:03.798195 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:03.798153 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7841fdc5-da16-4565-bb38-324b24b3916c-bundle\") on node \"ip-10-0-141-62.ec2.internal\" DevicePath \"\"" Jul 15 19:32:03.798195 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:03.798164 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7841fdc5-da16-4565-bb38-324b24b3916c-util\") on node \"ip-10-0-141-62.ec2.internal\" DevicePath \"\"" Jul 15 19:32:03.798195 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:03.798176 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cpckl\" (UniqueName: \"kubernetes.io/projected/7841fdc5-da16-4565-bb38-324b24b3916c-kube-api-access-cpckl\") on node \"ip-10-0-141-62.ec2.internal\" DevicePath \"\"" Jul 15 19:32:03.798195 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:03.798191 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lmf6p\" (UniqueName: \"kubernetes.io/projected/35defb98-7604-4ea6-ac0c-193c0cb0bed0-kube-api-access-lmf6p\") on node \"ip-10-0-141-62.ec2.internal\" DevicePath \"\"" Jul 15 19:32:03.798480 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:03.798204 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/35defb98-7604-4ea6-ac0c-193c0cb0bed0-util\") on node \"ip-10-0-141-62.ec2.internal\" DevicePath \"\"" Jul 15 19:32:04.470474 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:04.470440 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l46z4" Jul 15 19:32:04.470474 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:04.470467 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l46z4" event={"ID":"35defb98-7604-4ea6-ac0c-193c0cb0bed0","Type":"ContainerDied","Data":"2583e51a16131b75c71e06a0cc076a9c75d2f5e99466c120757d5406c904c3bc"} Jul 15 19:32:04.470923 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:04.470511 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2583e51a16131b75c71e06a0cc076a9c75d2f5e99466c120757d5406c904c3bc" Jul 15 19:32:04.472296 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:04.472273 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cpkdr" Jul 15 19:32:04.472428 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:04.472301 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cpkdr" event={"ID":"7841fdc5-da16-4565-bb38-324b24b3916c","Type":"ContainerDied","Data":"1ff3cf44ed6c1584746aafa1813216a13302d9242c3999cbbd2a7d42c88618fc"} Jul 15 19:32:04.472428 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:04.472347 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1ff3cf44ed6c1584746aafa1813216a13302d9242c3999cbbd2a7d42c88618fc" Jul 15 19:32:04.621670 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:04.621648 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t86m4" Jul 15 19:32:04.623633 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:04.623616 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqnwm" Jul 15 19:32:04.703883 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:04.703852 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8ad06c01-d62c-4d59-b1b4-c77854454d60-util\") pod \"8ad06c01-d62c-4d59-b1b4-c77854454d60\" (UID: \"8ad06c01-d62c-4d59-b1b4-c77854454d60\") " Jul 15 19:32:04.704047 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:04.703912 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8ad06c01-d62c-4d59-b1b4-c77854454d60-bundle\") pod \"8ad06c01-d62c-4d59-b1b4-c77854454d60\" (UID: \"8ad06c01-d62c-4d59-b1b4-c77854454d60\") " Jul 15 19:32:04.704047 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:04.703971 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4ec38921-1795-454a-9617-7a9b40cf7ab1-bundle\") pod \"4ec38921-1795-454a-9617-7a9b40cf7ab1\" (UID: \"4ec38921-1795-454a-9617-7a9b40cf7ab1\") " Jul 15 19:32:04.704047 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:04.704030 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vgsdq\" (UniqueName: \"kubernetes.io/projected/4ec38921-1795-454a-9617-7a9b40cf7ab1-kube-api-access-vgsdq\") pod \"4ec38921-1795-454a-9617-7a9b40cf7ab1\" (UID: \"4ec38921-1795-454a-9617-7a9b40cf7ab1\") " Jul 15 19:32:04.704217 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:04.704102 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4ec38921-1795-454a-9617-7a9b40cf7ab1-util\") pod \"4ec38921-1795-454a-9617-7a9b40cf7ab1\" (UID: \"4ec38921-1795-454a-9617-7a9b40cf7ab1\") " Jul 15 19:32:04.704217 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:04.704139 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x6hss\" (UniqueName: \"kubernetes.io/projected/8ad06c01-d62c-4d59-b1b4-c77854454d60-kube-api-access-x6hss\") pod \"8ad06c01-d62c-4d59-b1b4-c77854454d60\" (UID: \"8ad06c01-d62c-4d59-b1b4-c77854454d60\") " Jul 15 19:32:04.704662 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:04.704625 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ec38921-1795-454a-9617-7a9b40cf7ab1-bundle" (OuterVolumeSpecName: "bundle") pod "4ec38921-1795-454a-9617-7a9b40cf7ab1" (UID: "4ec38921-1795-454a-9617-7a9b40cf7ab1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:32:04.704840 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:04.704815 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ad06c01-d62c-4d59-b1b4-c77854454d60-bundle" (OuterVolumeSpecName: "bundle") pod "8ad06c01-d62c-4d59-b1b4-c77854454d60" (UID: "8ad06c01-d62c-4d59-b1b4-c77854454d60"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:32:04.706360 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:04.706333 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ec38921-1795-454a-9617-7a9b40cf7ab1-kube-api-access-vgsdq" (OuterVolumeSpecName: "kube-api-access-vgsdq") pod "4ec38921-1795-454a-9617-7a9b40cf7ab1" (UID: "4ec38921-1795-454a-9617-7a9b40cf7ab1"). InnerVolumeSpecName "kube-api-access-vgsdq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:32:04.706719 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:04.706693 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ad06c01-d62c-4d59-b1b4-c77854454d60-kube-api-access-x6hss" (OuterVolumeSpecName: "kube-api-access-x6hss") pod "8ad06c01-d62c-4d59-b1b4-c77854454d60" (UID: "8ad06c01-d62c-4d59-b1b4-c77854454d60"). InnerVolumeSpecName "kube-api-access-x6hss". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:32:04.710514 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:04.710488 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ec38921-1795-454a-9617-7a9b40cf7ab1-util" (OuterVolumeSpecName: "util") pod "4ec38921-1795-454a-9617-7a9b40cf7ab1" (UID: "4ec38921-1795-454a-9617-7a9b40cf7ab1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:32:04.710638 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:04.710558 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ad06c01-d62c-4d59-b1b4-c77854454d60-util" (OuterVolumeSpecName: "util") pod "8ad06c01-d62c-4d59-b1b4-c77854454d60" (UID: "8ad06c01-d62c-4d59-b1b4-c77854454d60"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:32:04.804889 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:04.804862 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vgsdq\" (UniqueName: \"kubernetes.io/projected/4ec38921-1795-454a-9617-7a9b40cf7ab1-kube-api-access-vgsdq\") on node \"ip-10-0-141-62.ec2.internal\" DevicePath \"\"" Jul 15 19:32:04.804889 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:04.804887 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4ec38921-1795-454a-9617-7a9b40cf7ab1-util\") on node \"ip-10-0-141-62.ec2.internal\" DevicePath \"\"" Jul 15 19:32:04.805037 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:04.804897 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6hss\" (UniqueName: \"kubernetes.io/projected/8ad06c01-d62c-4d59-b1b4-c77854454d60-kube-api-access-x6hss\") on node \"ip-10-0-141-62.ec2.internal\" DevicePath \"\"" Jul 15 19:32:04.805037 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:04.804905 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8ad06c01-d62c-4d59-b1b4-c77854454d60-util\") on node \"ip-10-0-141-62.ec2.internal\" DevicePath \"\"" Jul 15 19:32:04.805037 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:04.804914 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8ad06c01-d62c-4d59-b1b4-c77854454d60-bundle\") on node \"ip-10-0-141-62.ec2.internal\" DevicePath \"\"" Jul 15 19:32:04.805037 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:04.804921 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4ec38921-1795-454a-9617-7a9b40cf7ab1-bundle\") on node \"ip-10-0-141-62.ec2.internal\" DevicePath \"\"" Jul 15 19:32:05.478980 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:05.478903 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t86m4" Jul 15 19:32:05.479381 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:05.478893 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t86m4" event={"ID":"8ad06c01-d62c-4d59-b1b4-c77854454d60","Type":"ContainerDied","Data":"4b43a94187ed42a7c7531f647bd120b28e1239c049be4ea87bd3767898ac13e4"} Jul 15 19:32:05.479381 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:05.479033 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4b43a94187ed42a7c7531f647bd120b28e1239c049be4ea87bd3767898ac13e4" Jul 15 19:32:05.480953 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:05.480924 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqnwm" event={"ID":"4ec38921-1795-454a-9617-7a9b40cf7ab1","Type":"ContainerDied","Data":"36e80bdf17a9530ccea724b8d92d024fb4682d396584bcf4818174f65344bbd6"} Jul 15 19:32:05.480953 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:05.480964 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="36e80bdf17a9530ccea724b8d92d024fb4682d396584bcf4818174f65344bbd6" Jul 15 19:32:05.481154 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:05.480973 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqnwm" Jul 15 19:32:14.916869 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:14.916839 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c65576f67-dwfqz"] Jul 15 19:32:32.961545 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:32.961512 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-hc7tq"] Jul 15 19:32:32.961985 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:32.961814 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7841fdc5-da16-4565-bb38-324b24b3916c" containerName="pull" Jul 15 19:32:32.961985 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:32.961824 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7841fdc5-da16-4565-bb38-324b24b3916c" containerName="pull" Jul 15 19:32:32.961985 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:32.961834 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ad06c01-d62c-4d59-b1b4-c77854454d60" containerName="pull" Jul 15 19:32:32.961985 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:32.961840 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ad06c01-d62c-4d59-b1b4-c77854454d60" containerName="pull" Jul 15 19:32:32.961985 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:32.961846 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7841fdc5-da16-4565-bb38-324b24b3916c" containerName="extract" Jul 15 19:32:32.961985 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:32.961852 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7841fdc5-da16-4565-bb38-324b24b3916c" containerName="extract" Jul 15 19:32:32.961985 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:32.961858 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35defb98-7604-4ea6-ac0c-193c0cb0bed0" containerName="pull" Jul 15 19:32:32.961985 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:32.961863 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="35defb98-7604-4ea6-ac0c-193c0cb0bed0" containerName="pull" Jul 15 19:32:32.961985 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:32.961874 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ec38921-1795-454a-9617-7a9b40cf7ab1" containerName="util" Jul 15 19:32:32.961985 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:32.961878 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ec38921-1795-454a-9617-7a9b40cf7ab1" containerName="util" Jul 15 19:32:32.961985 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:32.961887 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35defb98-7604-4ea6-ac0c-193c0cb0bed0" containerName="extract" Jul 15 19:32:32.961985 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:32.961892 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="35defb98-7604-4ea6-ac0c-193c0cb0bed0" containerName="extract" Jul 15 19:32:32.961985 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:32.961898 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ec38921-1795-454a-9617-7a9b40cf7ab1" containerName="extract" Jul 15 19:32:32.961985 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:32.961903 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ec38921-1795-454a-9617-7a9b40cf7ab1" containerName="extract" Jul 15 19:32:32.961985 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:32.961909 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ad06c01-d62c-4d59-b1b4-c77854454d60" containerName="extract" Jul 15 19:32:32.961985 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:32.961914 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ad06c01-d62c-4d59-b1b4-c77854454d60" containerName="extract" Jul 15 19:32:32.961985 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:32.961921 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35defb98-7604-4ea6-ac0c-193c0cb0bed0" containerName="util" Jul 15 19:32:32.961985 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:32.961925 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="35defb98-7604-4ea6-ac0c-193c0cb0bed0" containerName="util" Jul 15 19:32:32.961985 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:32.961932 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7841fdc5-da16-4565-bb38-324b24b3916c" containerName="util" Jul 15 19:32:32.961985 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:32.961936 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7841fdc5-da16-4565-bb38-324b24b3916c" containerName="util" Jul 15 19:32:32.961985 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:32.961945 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ad06c01-d62c-4d59-b1b4-c77854454d60" containerName="util" Jul 15 19:32:32.961985 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:32.961950 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ad06c01-d62c-4d59-b1b4-c77854454d60" containerName="util" Jul 15 19:32:32.961985 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:32.961956 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ec38921-1795-454a-9617-7a9b40cf7ab1" containerName="pull" Jul 15 19:32:32.961985 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:32.961961 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ec38921-1795-454a-9617-7a9b40cf7ab1" containerName="pull" Jul 15 19:32:32.962693 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:32.962008 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ec38921-1795-454a-9617-7a9b40cf7ab1" containerName="extract" Jul 15 19:32:32.962693 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:32.962016 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7841fdc5-da16-4565-bb38-324b24b3916c" containerName="extract" Jul 15 19:32:32.962693 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:32.962022 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ad06c01-d62c-4d59-b1b4-c77854454d60" containerName="extract" Jul 15 19:32:32.962693 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:32.962030 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="35defb98-7604-4ea6-ac0c-193c0cb0bed0" containerName="extract" Jul 15 19:32:32.966388 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:32.966372 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hc7tq" Jul 15 19:32:32.969363 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:32.969299 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 15 19:32:32.969363 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:32.969340 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 15 19:32:32.969553 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:32.969376 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 15 19:32:32.969553 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:32.969393 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-drc2n\"" Jul 15 19:32:32.969553 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:32.969399 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 15 19:32:32.975101 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:32.975081 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-hc7tq"] Jul 15 19:32:33.128508 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:33.128479 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/807d4176-fe09-46d9-8f69-c26648484006-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-hc7tq\" (UID: \"807d4176-fe09-46d9-8f69-c26648484006\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hc7tq" Jul 15 19:32:33.128647 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:33.128515 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vnhf\" (UniqueName: \"kubernetes.io/projected/807d4176-fe09-46d9-8f69-c26648484006-kube-api-access-8vnhf\") pod \"kuadrant-console-plugin-66b5774985-hc7tq\" (UID: \"807d4176-fe09-46d9-8f69-c26648484006\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hc7tq" Jul 15 19:32:33.128647 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:33.128561 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/807d4176-fe09-46d9-8f69-c26648484006-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-hc7tq\" (UID: \"807d4176-fe09-46d9-8f69-c26648484006\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hc7tq" Jul 15 19:32:33.229387 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:33.229302 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/807d4176-fe09-46d9-8f69-c26648484006-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-hc7tq\" (UID: \"807d4176-fe09-46d9-8f69-c26648484006\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hc7tq" Jul 15 19:32:33.229387 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:33.229356 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vnhf\" (UniqueName: \"kubernetes.io/projected/807d4176-fe09-46d9-8f69-c26648484006-kube-api-access-8vnhf\") pod \"kuadrant-console-plugin-66b5774985-hc7tq\" (UID: \"807d4176-fe09-46d9-8f69-c26648484006\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hc7tq" Jul 15 19:32:33.229581 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:33.229397 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/807d4176-fe09-46d9-8f69-c26648484006-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-hc7tq\" (UID: \"807d4176-fe09-46d9-8f69-c26648484006\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hc7tq" Jul 15 19:32:33.229900 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:33.229881 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/807d4176-fe09-46d9-8f69-c26648484006-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-hc7tq\" (UID: \"807d4176-fe09-46d9-8f69-c26648484006\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hc7tq" Jul 15 19:32:33.231799 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:33.231779 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/807d4176-fe09-46d9-8f69-c26648484006-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-hc7tq\" (UID: \"807d4176-fe09-46d9-8f69-c26648484006\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hc7tq" Jul 15 19:32:33.236906 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:33.236889 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vnhf\" (UniqueName: \"kubernetes.io/projected/807d4176-fe09-46d9-8f69-c26648484006-kube-api-access-8vnhf\") pod \"kuadrant-console-plugin-66b5774985-hc7tq\" (UID: \"807d4176-fe09-46d9-8f69-c26648484006\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hc7tq" Jul 15 19:32:33.276244 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:33.276220 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hc7tq" Jul 15 19:32:33.407283 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:33.407260 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-hc7tq"] Jul 15 19:32:33.409469 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:32:33.409442 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod807d4176_fe09_46d9_8f69_c26648484006.slice/crio-b21374d2534977f1fe596ce01af0112d963efd487995e58bae52dd289d624520 WatchSource:0}: Error finding container b21374d2534977f1fe596ce01af0112d963efd487995e58bae52dd289d624520: Status 404 returned error can't find the container with id b21374d2534977f1fe596ce01af0112d963efd487995e58bae52dd289d624520 Jul 15 19:32:33.613224 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:33.613192 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hc7tq" event={"ID":"807d4176-fe09-46d9-8f69-c26648484006","Type":"ContainerStarted","Data":"b21374d2534977f1fe596ce01af0112d963efd487995e58bae52dd289d624520"} Jul 15 19:32:39.937176 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:39.937105 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7c65576f67-dwfqz" podUID="9ec7c062-7d57-45f4-92fa-248c7f9f96c2" containerName="console" containerID="cri-o://613a7e0716c35388c204b92820cc40bd97865a070f6048f7f8d9e4a5cb05d5fa" gracePeriod=15 Jul 15 19:32:40.201996 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:40.201939 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c65576f67-dwfqz" Jul 15 19:32:40.287087 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:40.287058 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9ec7c062-7d57-45f4-92fa-248c7f9f96c2-console-serving-cert\") pod \"9ec7c062-7d57-45f4-92fa-248c7f9f96c2\" (UID: \"9ec7c062-7d57-45f4-92fa-248c7f9f96c2\") " Jul 15 19:32:40.287265 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:40.287106 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9ec7c062-7d57-45f4-92fa-248c7f9f96c2-console-config\") pod \"9ec7c062-7d57-45f4-92fa-248c7f9f96c2\" (UID: \"9ec7c062-7d57-45f4-92fa-248c7f9f96c2\") " Jul 15 19:32:40.287265 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:40.287159 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9ec7c062-7d57-45f4-92fa-248c7f9f96c2-service-ca\") pod \"9ec7c062-7d57-45f4-92fa-248c7f9f96c2\" (UID: \"9ec7c062-7d57-45f4-92fa-248c7f9f96c2\") " Jul 15 19:32:40.287265 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:40.287192 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cxmrq\" (UniqueName: \"kubernetes.io/projected/9ec7c062-7d57-45f4-92fa-248c7f9f96c2-kube-api-access-cxmrq\") pod \"9ec7c062-7d57-45f4-92fa-248c7f9f96c2\" (UID: \"9ec7c062-7d57-45f4-92fa-248c7f9f96c2\") " Jul 15 19:32:40.287265 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:40.287228 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9ec7c062-7d57-45f4-92fa-248c7f9f96c2-console-oauth-config\") pod \"9ec7c062-7d57-45f4-92fa-248c7f9f96c2\" (UID: \"9ec7c062-7d57-45f4-92fa-248c7f9f96c2\") " Jul 15 19:32:40.287446 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:40.287267 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9ec7c062-7d57-45f4-92fa-248c7f9f96c2-oauth-serving-cert\") pod \"9ec7c062-7d57-45f4-92fa-248c7f9f96c2\" (UID: \"9ec7c062-7d57-45f4-92fa-248c7f9f96c2\") " Jul 15 19:32:40.287446 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:40.287340 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ec7c062-7d57-45f4-92fa-248c7f9f96c2-trusted-ca-bundle\") pod \"9ec7c062-7d57-45f4-92fa-248c7f9f96c2\" (UID: \"9ec7c062-7d57-45f4-92fa-248c7f9f96c2\") " Jul 15 19:32:40.287911 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:40.287865 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ec7c062-7d57-45f4-92fa-248c7f9f96c2-service-ca" (OuterVolumeSpecName: "service-ca") pod "9ec7c062-7d57-45f4-92fa-248c7f9f96c2" (UID: "9ec7c062-7d57-45f4-92fa-248c7f9f96c2"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:32:40.287911 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:40.287889 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ec7c062-7d57-45f4-92fa-248c7f9f96c2-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "9ec7c062-7d57-45f4-92fa-248c7f9f96c2" (UID: "9ec7c062-7d57-45f4-92fa-248c7f9f96c2"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:32:40.288198 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:40.288017 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ec7c062-7d57-45f4-92fa-248c7f9f96c2-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "9ec7c062-7d57-45f4-92fa-248c7f9f96c2" (UID: "9ec7c062-7d57-45f4-92fa-248c7f9f96c2"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:32:40.288198 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:40.288174 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ec7c062-7d57-45f4-92fa-248c7f9f96c2-console-config" (OuterVolumeSpecName: "console-config") pod "9ec7c062-7d57-45f4-92fa-248c7f9f96c2" (UID: "9ec7c062-7d57-45f4-92fa-248c7f9f96c2"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:32:40.289782 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:40.289759 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ec7c062-7d57-45f4-92fa-248c7f9f96c2-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "9ec7c062-7d57-45f4-92fa-248c7f9f96c2" (UID: "9ec7c062-7d57-45f4-92fa-248c7f9f96c2"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:32:40.289891 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:40.289832 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ec7c062-7d57-45f4-92fa-248c7f9f96c2-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "9ec7c062-7d57-45f4-92fa-248c7f9f96c2" (UID: "9ec7c062-7d57-45f4-92fa-248c7f9f96c2"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:32:40.290188 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:40.290152 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ec7c062-7d57-45f4-92fa-248c7f9f96c2-kube-api-access-cxmrq" (OuterVolumeSpecName: "kube-api-access-cxmrq") pod "9ec7c062-7d57-45f4-92fa-248c7f9f96c2" (UID: "9ec7c062-7d57-45f4-92fa-248c7f9f96c2"). InnerVolumeSpecName "kube-api-access-cxmrq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:32:40.388678 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:40.388655 2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9ec7c062-7d57-45f4-92fa-248c7f9f96c2-oauth-serving-cert\") on node \"ip-10-0-141-62.ec2.internal\" DevicePath \"\"" Jul 15 19:32:40.388678 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:40.388677 2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ec7c062-7d57-45f4-92fa-248c7f9f96c2-trusted-ca-bundle\") on node \"ip-10-0-141-62.ec2.internal\" DevicePath \"\"" Jul 15 19:32:40.388822 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:40.388687 2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9ec7c062-7d57-45f4-92fa-248c7f9f96c2-console-serving-cert\") on node \"ip-10-0-141-62.ec2.internal\" DevicePath \"\"" Jul 15 19:32:40.388822 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:40.388696 2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9ec7c062-7d57-45f4-92fa-248c7f9f96c2-console-config\") on node \"ip-10-0-141-62.ec2.internal\" DevicePath \"\"" Jul 15 19:32:40.388822 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:40.388705 2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9ec7c062-7d57-45f4-92fa-248c7f9f96c2-service-ca\") on node \"ip-10-0-141-62.ec2.internal\" DevicePath \"\"" Jul 15 19:32:40.388822 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:40.388713 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cxmrq\" (UniqueName: \"kubernetes.io/projected/9ec7c062-7d57-45f4-92fa-248c7f9f96c2-kube-api-access-cxmrq\") on node \"ip-10-0-141-62.ec2.internal\" DevicePath \"\"" Jul 15 19:32:40.388822 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:40.388722 2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9ec7c062-7d57-45f4-92fa-248c7f9f96c2-console-oauth-config\") on node \"ip-10-0-141-62.ec2.internal\" DevicePath \"\"" Jul 15 19:32:40.654065 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:40.654036 2577 generic.go:356] "Generic (PLEG): container finished" podID="9ec7c062-7d57-45f4-92fa-248c7f9f96c2" containerID="613a7e0716c35388c204b92820cc40bd97865a070f6048f7f8d9e4a5cb05d5fa" exitCode=0 Jul 15 19:32:40.654222 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:40.654072 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c65576f67-dwfqz" event={"ID":"9ec7c062-7d57-45f4-92fa-248c7f9f96c2","Type":"ContainerDied","Data":"613a7e0716c35388c204b92820cc40bd97865a070f6048f7f8d9e4a5cb05d5fa"} Jul 15 19:32:40.654222 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:40.654078 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c65576f67-dwfqz" Jul 15 19:32:40.654222 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:40.654107 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c65576f67-dwfqz" event={"ID":"9ec7c062-7d57-45f4-92fa-248c7f9f96c2","Type":"ContainerDied","Data":"b317d3432e4ec9f4ee7365cf2dd301895b1b82bd39dd3f9deefead09657d6721"} Jul 15 19:32:40.654222 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:40.654127 2577 scope.go:117] "RemoveContainer" containerID="613a7e0716c35388c204b92820cc40bd97865a070f6048f7f8d9e4a5cb05d5fa" Jul 15 19:32:40.663560 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:40.663544 2577 scope.go:117] "RemoveContainer" containerID="613a7e0716c35388c204b92820cc40bd97865a070f6048f7f8d9e4a5cb05d5fa" Jul 15 19:32:40.663815 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:32:40.663800 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"613a7e0716c35388c204b92820cc40bd97865a070f6048f7f8d9e4a5cb05d5fa\": container with ID starting with 613a7e0716c35388c204b92820cc40bd97865a070f6048f7f8d9e4a5cb05d5fa not found: ID does not exist" containerID="613a7e0716c35388c204b92820cc40bd97865a070f6048f7f8d9e4a5cb05d5fa" Jul 15 19:32:40.663878 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:40.663825 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"613a7e0716c35388c204b92820cc40bd97865a070f6048f7f8d9e4a5cb05d5fa"} err="failed to get container status \"613a7e0716c35388c204b92820cc40bd97865a070f6048f7f8d9e4a5cb05d5fa\": rpc error: code = NotFound desc = could not find container \"613a7e0716c35388c204b92820cc40bd97865a070f6048f7f8d9e4a5cb05d5fa\": container with ID starting with 613a7e0716c35388c204b92820cc40bd97865a070f6048f7f8d9e4a5cb05d5fa not found: ID does not exist" Jul 15 19:32:40.676877 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:40.676854 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c65576f67-dwfqz"] Jul 15 19:32:40.685210 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:40.685183 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7c65576f67-dwfqz"] Jul 15 19:32:41.093752 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:41.093721 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ec7c062-7d57-45f4-92fa-248c7f9f96c2" path="/var/lib/kubelet/pods/9ec7c062-7d57-45f4-92fa-248c7f9f96c2/volumes" Jul 15 19:32:58.759055 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:58.759020 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hc7tq" event={"ID":"807d4176-fe09-46d9-8f69-c26648484006","Type":"ContainerStarted","Data":"557f330a0c886147508e3d04d4f9716f3d738881cde92dceef3a47dc40dda501"} Jul 15 19:32:58.777630 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:32:58.777581 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hc7tq" podStartSLOduration=2.466364732 podStartE2EDuration="26.777565281s" totalImagesPullingTime="24.311200549s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:32:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:32:58.775536885 +0000 UTC m=+692.352538557" watchObservedRunningTime="2026-07-15 19:32:58.777565281 +0000 UTC m=+692.354566951" Jul 15 19:33:24.953636 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:33:24.953556 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pngz"] Jul 15 19:33:24.954069 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:33:24.953866 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ec7c062-7d57-45f4-92fa-248c7f9f96c2" containerName="console" Jul 15 19:33:24.954069 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:33:24.953877 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ec7c062-7d57-45f4-92fa-248c7f9f96c2" containerName="console" Jul 15 19:33:24.954069 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:33:24.953939 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ec7c062-7d57-45f4-92fa-248c7f9f96c2" containerName="console" Jul 15 19:33:24.972859 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:33:24.972835 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pngz"] Jul 15 19:33:24.973008 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:33:24.972960 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-8pngz" Jul 15 19:33:24.975143 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:33:24.975124 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 15 19:33:25.008355 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:33:25.008327 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pngz"] Jul 15 19:33:25.018932 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:33:25.018908 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdsws\" (UniqueName: \"kubernetes.io/projected/65ef11f8-41ff-41ac-a0cc-e85de469abdb-kube-api-access-mdsws\") pod \"limitador-limitador-54d6d988d5-8pngz\" (UID: \"65ef11f8-41ff-41ac-a0cc-e85de469abdb\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-8pngz" Jul 15 19:33:25.019057 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:33:25.018944 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/65ef11f8-41ff-41ac-a0cc-e85de469abdb-config-file\") pod \"limitador-limitador-54d6d988d5-8pngz\" (UID: \"65ef11f8-41ff-41ac-a0cc-e85de469abdb\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-8pngz" Jul 15 19:33:25.119989 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:33:25.119945 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdsws\" (UniqueName: \"kubernetes.io/projected/65ef11f8-41ff-41ac-a0cc-e85de469abdb-kube-api-access-mdsws\") pod \"limitador-limitador-54d6d988d5-8pngz\" (UID: \"65ef11f8-41ff-41ac-a0cc-e85de469abdb\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-8pngz" Jul 15 19:33:25.119989 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:33:25.119984 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/65ef11f8-41ff-41ac-a0cc-e85de469abdb-config-file\") pod \"limitador-limitador-54d6d988d5-8pngz\" (UID: \"65ef11f8-41ff-41ac-a0cc-e85de469abdb\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-8pngz" Jul 15 19:33:25.120616 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:33:25.120597 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/65ef11f8-41ff-41ac-a0cc-e85de469abdb-config-file\") pod \"limitador-limitador-54d6d988d5-8pngz\" (UID: \"65ef11f8-41ff-41ac-a0cc-e85de469abdb\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-8pngz" Jul 15 19:33:25.128615 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:33:25.128590 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdsws\" (UniqueName: \"kubernetes.io/projected/65ef11f8-41ff-41ac-a0cc-e85de469abdb-kube-api-access-mdsws\") pod \"limitador-limitador-54d6d988d5-8pngz\" (UID: \"65ef11f8-41ff-41ac-a0cc-e85de469abdb\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-8pngz" Jul 15 19:33:25.283979 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:33:25.283897 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-8pngz" Jul 15 19:33:25.475928 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:33:25.475901 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pngz"] Jul 15 19:33:25.478623 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:33:25.478596 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65ef11f8_41ff_41ac_a0cc_e85de469abdb.slice/crio-f84d318bb89ae82ea027b58ae640abb83e50c342e2b743e9bea9adccfd5d9751 WatchSource:0}: Error finding container f84d318bb89ae82ea027b58ae640abb83e50c342e2b743e9bea9adccfd5d9751: Status 404 returned error can't find the container with id f84d318bb89ae82ea027b58ae640abb83e50c342e2b743e9bea9adccfd5d9751 Jul 15 19:33:25.518417 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:33:25.518393 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-9lfm7"] Jul 15 19:33:25.546553 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:33:25.546501 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-9lfm7" Jul 15 19:33:25.546654 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:33:25.546507 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-9lfm7"] Jul 15 19:33:25.548635 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:33:25.548599 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-4j5jf\"" Jul 15 19:33:25.623363 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:33:25.623331 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztnv2\" (UniqueName: \"kubernetes.io/projected/df68813f-bc57-468f-93a9-d1d58b874f4a-kube-api-access-ztnv2\") pod \"authorino-75d556bd77-9lfm7\" (UID: \"df68813f-bc57-468f-93a9-d1d58b874f4a\") " pod="kuadrant-system/authorino-75d556bd77-9lfm7" Jul 15 19:33:25.724636 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:33:25.724609 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztnv2\" (UniqueName: \"kubernetes.io/projected/df68813f-bc57-468f-93a9-d1d58b874f4a-kube-api-access-ztnv2\") pod \"authorino-75d556bd77-9lfm7\" (UID: \"df68813f-bc57-468f-93a9-d1d58b874f4a\") " pod="kuadrant-system/authorino-75d556bd77-9lfm7" Jul 15 19:33:25.733761 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:33:25.733738 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztnv2\" (UniqueName: \"kubernetes.io/projected/df68813f-bc57-468f-93a9-d1d58b874f4a-kube-api-access-ztnv2\") pod \"authorino-75d556bd77-9lfm7\" (UID: \"df68813f-bc57-468f-93a9-d1d58b874f4a\") " pod="kuadrant-system/authorino-75d556bd77-9lfm7" Jul 15 19:33:25.855994 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:33:25.855966 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-9lfm7" Jul 15 19:33:25.891869 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:33:25.891830 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-8pngz" event={"ID":"65ef11f8-41ff-41ac-a0cc-e85de469abdb","Type":"ContainerStarted","Data":"f84d318bb89ae82ea027b58ae640abb83e50c342e2b743e9bea9adccfd5d9751"} Jul 15 19:33:26.016825 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:33:26.016800 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-9lfm7"] Jul 15 19:33:26.018630 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:33:26.018601 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddf68813f_bc57_468f_93a9_d1d58b874f4a.slice/crio-1bada2a776ae3a5caad675cfa14d62153bfabd76ad2c8a1ddf16746bffc0aa3c WatchSource:0}: Error finding container 1bada2a776ae3a5caad675cfa14d62153bfabd76ad2c8a1ddf16746bffc0aa3c: Status 404 returned error can't find the container with id 1bada2a776ae3a5caad675cfa14d62153bfabd76ad2c8a1ddf16746bffc0aa3c Jul 15 19:33:26.901902 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:33:26.901868 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-9lfm7" event={"ID":"df68813f-bc57-468f-93a9-d1d58b874f4a","Type":"ContainerStarted","Data":"1bada2a776ae3a5caad675cfa14d62153bfabd76ad2c8a1ddf16746bffc0aa3c"} Jul 15 19:33:30.924299 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:33:30.924255 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-8pngz" event={"ID":"65ef11f8-41ff-41ac-a0cc-e85de469abdb","Type":"ContainerStarted","Data":"6695e76612cdcc33d23f6019958c4a29fb8c20cd099cae248a05e4e613577e63"} Jul 15 19:33:30.924790 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:33:30.924344 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-8pngz" Jul 15 19:33:30.925707 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:33:30.925681 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-9lfm7" event={"ID":"df68813f-bc57-468f-93a9-d1d58b874f4a","Type":"ContainerStarted","Data":"62b75183ce691a33ae45cafefba09c6adf1a488aa8dcc2775fb3a5dc11512daf"} Jul 15 19:33:30.944392 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:33:30.944350 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-8pngz" podStartSLOduration=2.412266227 podStartE2EDuration="6.944337619s" totalImagesPullingTime="4.532071392s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:33:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:33:30.94194484 +0000 UTC m=+724.518946512" watchObservedRunningTime="2026-07-15 19:33:30.944337619 +0000 UTC m=+724.521339290" Jul 15 19:33:30.960138 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:33:30.960091 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-9lfm7" podStartSLOduration=1.965698357 podStartE2EDuration="5.960080042s" totalImagesPullingTime="3.994381685s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:33:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:33:30.958335898 +0000 UTC m=+724.535337582" watchObservedRunningTime="2026-07-15 19:33:30.960080042 +0000 UTC m=+724.537081713" Jul 15 19:33:41.930953 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:33:41.930924 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-8pngz" Jul 15 19:34:02.081917 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:34:02.081884 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-9lfm7"] Jul 15 19:34:02.082513 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:34:02.082049 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-9lfm7" podUID="df68813f-bc57-468f-93a9-d1d58b874f4a" containerName="authorino" containerID="cri-o://62b75183ce691a33ae45cafefba09c6adf1a488aa8dcc2775fb3a5dc11512daf" gracePeriod=30 Jul 15 19:34:02.334835 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:34:02.334782 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-9lfm7" Jul 15 19:34:02.511564 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:34:02.511534 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ztnv2\" (UniqueName: \"kubernetes.io/projected/df68813f-bc57-468f-93a9-d1d58b874f4a-kube-api-access-ztnv2\") pod \"df68813f-bc57-468f-93a9-d1d58b874f4a\" (UID: \"df68813f-bc57-468f-93a9-d1d58b874f4a\") " Jul 15 19:34:02.513554 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:34:02.513532 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df68813f-bc57-468f-93a9-d1d58b874f4a-kube-api-access-ztnv2" (OuterVolumeSpecName: "kube-api-access-ztnv2") pod "df68813f-bc57-468f-93a9-d1d58b874f4a" (UID: "df68813f-bc57-468f-93a9-d1d58b874f4a"). InnerVolumeSpecName "kube-api-access-ztnv2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:34:02.612838 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:34:02.612773 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ztnv2\" (UniqueName: \"kubernetes.io/projected/df68813f-bc57-468f-93a9-d1d58b874f4a-kube-api-access-ztnv2\") on node \"ip-10-0-141-62.ec2.internal\" DevicePath \"\"" Jul 15 19:34:03.080225 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:34:03.080193 2577 generic.go:356] "Generic (PLEG): container finished" podID="df68813f-bc57-468f-93a9-d1d58b874f4a" containerID="62b75183ce691a33ae45cafefba09c6adf1a488aa8dcc2775fb3a5dc11512daf" exitCode=0 Jul 15 19:34:03.080422 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:34:03.080236 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-9lfm7" Jul 15 19:34:03.080422 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:34:03.080280 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-9lfm7" event={"ID":"df68813f-bc57-468f-93a9-d1d58b874f4a","Type":"ContainerDied","Data":"62b75183ce691a33ae45cafefba09c6adf1a488aa8dcc2775fb3a5dc11512daf"} Jul 15 19:34:03.080422 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:34:03.080339 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-9lfm7" event={"ID":"df68813f-bc57-468f-93a9-d1d58b874f4a","Type":"ContainerDied","Data":"1bada2a776ae3a5caad675cfa14d62153bfabd76ad2c8a1ddf16746bffc0aa3c"} Jul 15 19:34:03.080422 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:34:03.080361 2577 scope.go:117] "RemoveContainer" containerID="62b75183ce691a33ae45cafefba09c6adf1a488aa8dcc2775fb3a5dc11512daf" Jul 15 19:34:03.089401 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:34:03.089190 2577 scope.go:117] "RemoveContainer" containerID="62b75183ce691a33ae45cafefba09c6adf1a488aa8dcc2775fb3a5dc11512daf" Jul 15 19:34:03.089621 ip-10-0-141-62 kubenswrapper[2577]: E0715 19:34:03.089466 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62b75183ce691a33ae45cafefba09c6adf1a488aa8dcc2775fb3a5dc11512daf\": container with ID starting with 62b75183ce691a33ae45cafefba09c6adf1a488aa8dcc2775fb3a5dc11512daf not found: ID does not exist" containerID="62b75183ce691a33ae45cafefba09c6adf1a488aa8dcc2775fb3a5dc11512daf" Jul 15 19:34:03.089621 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:34:03.089493 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62b75183ce691a33ae45cafefba09c6adf1a488aa8dcc2775fb3a5dc11512daf"} err="failed to get container status \"62b75183ce691a33ae45cafefba09c6adf1a488aa8dcc2775fb3a5dc11512daf\": rpc error: code = NotFound desc = could not find container \"62b75183ce691a33ae45cafefba09c6adf1a488aa8dcc2775fb3a5dc11512daf\": container with ID starting with 62b75183ce691a33ae45cafefba09c6adf1a488aa8dcc2775fb3a5dc11512daf not found: ID does not exist" Jul 15 19:34:03.103133 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:34:03.103108 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-9lfm7"] Jul 15 19:34:03.108839 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:34:03.108820 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-9lfm7"] Jul 15 19:34:05.092531 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:34:05.092501 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df68813f-bc57-468f-93a9-d1d58b874f4a" path="/var/lib/kubelet/pods/df68813f-bc57-468f-93a9-d1d58b874f4a/volumes" Jul 15 19:34:44.707648 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:34:44.707617 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pngz"] Jul 15 19:35:03.811402 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:03.811307 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pngz"] Jul 15 19:35:15.378822 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:15.378785 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pngz"] Jul 15 19:35:21.232585 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:21.232546 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pngz"] Jul 15 19:35:30.573360 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:30.573326 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pngz"] Jul 15 19:35:38.499499 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:38.499468 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-5t6xx"] Jul 15 19:35:38.499945 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:38.499789 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df68813f-bc57-468f-93a9-d1d58b874f4a" containerName="authorino" Jul 15 19:35:38.499945 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:38.499800 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="df68813f-bc57-468f-93a9-d1d58b874f4a" containerName="authorino" Jul 15 19:35:38.499945 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:38.499861 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="df68813f-bc57-468f-93a9-d1d58b874f4a" containerName="authorino" Jul 15 19:35:38.501788 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:38.501772 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5t6xx" Jul 15 19:35:38.504632 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:38.504612 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 15 19:35:38.504632 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:38.504627 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 15 19:35:38.505444 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:38.505426 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 15 19:35:38.505784 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:38.505770 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-56lms\"" Jul 15 19:35:38.516262 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:38.516243 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-5t6xx"] Jul 15 19:35:38.541458 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:38.541435 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/68addbce-40e8-4f9e-b460-64f68720a1a6-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5t6xx\" (UID: \"68addbce-40e8-4f9e-b460-64f68720a1a6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5t6xx" Jul 15 19:35:38.541577 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:38.541465 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68addbce-40e8-4f9e-b460-64f68720a1a6-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5t6xx\" (UID: \"68addbce-40e8-4f9e-b460-64f68720a1a6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5t6xx" Jul 15 19:35:38.541577 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:38.541501 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nj5wc\" (UniqueName: \"kubernetes.io/projected/68addbce-40e8-4f9e-b460-64f68720a1a6-kube-api-access-nj5wc\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5t6xx\" (UID: \"68addbce-40e8-4f9e-b460-64f68720a1a6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5t6xx" Jul 15 19:35:38.541577 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:38.541563 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/68addbce-40e8-4f9e-b460-64f68720a1a6-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5t6xx\" (UID: \"68addbce-40e8-4f9e-b460-64f68720a1a6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5t6xx" Jul 15 19:35:38.541736 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:38.541585 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/68addbce-40e8-4f9e-b460-64f68720a1a6-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5t6xx\" (UID: \"68addbce-40e8-4f9e-b460-64f68720a1a6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5t6xx" Jul 15 19:35:38.541736 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:38.541641 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/68addbce-40e8-4f9e-b460-64f68720a1a6-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5t6xx\" (UID: \"68addbce-40e8-4f9e-b460-64f68720a1a6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5t6xx" Jul 15 19:35:38.642937 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:38.642910 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nj5wc\" (UniqueName: \"kubernetes.io/projected/68addbce-40e8-4f9e-b460-64f68720a1a6-kube-api-access-nj5wc\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5t6xx\" (UID: \"68addbce-40e8-4f9e-b460-64f68720a1a6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5t6xx" Jul 15 19:35:38.643081 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:38.642946 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/68addbce-40e8-4f9e-b460-64f68720a1a6-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5t6xx\" (UID: \"68addbce-40e8-4f9e-b460-64f68720a1a6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5t6xx" Jul 15 19:35:38.643081 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:38.642963 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/68addbce-40e8-4f9e-b460-64f68720a1a6-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5t6xx\" (UID: \"68addbce-40e8-4f9e-b460-64f68720a1a6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5t6xx" Jul 15 19:35:38.643081 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:38.642987 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/68addbce-40e8-4f9e-b460-64f68720a1a6-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5t6xx\" (UID: \"68addbce-40e8-4f9e-b460-64f68720a1a6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5t6xx" Jul 15 19:35:38.643081 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:38.643020 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/68addbce-40e8-4f9e-b460-64f68720a1a6-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5t6xx\" (UID: \"68addbce-40e8-4f9e-b460-64f68720a1a6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5t6xx" Jul 15 19:35:38.643081 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:38.643045 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68addbce-40e8-4f9e-b460-64f68720a1a6-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5t6xx\" (UID: \"68addbce-40e8-4f9e-b460-64f68720a1a6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5t6xx" Jul 15 19:35:38.643399 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:38.643377 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/68addbce-40e8-4f9e-b460-64f68720a1a6-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5t6xx\" (UID: \"68addbce-40e8-4f9e-b460-64f68720a1a6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5t6xx" Jul 15 19:35:38.643470 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:38.643449 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/68addbce-40e8-4f9e-b460-64f68720a1a6-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5t6xx\" (UID: \"68addbce-40e8-4f9e-b460-64f68720a1a6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5t6xx" Jul 15 19:35:38.643525 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:38.643488 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68addbce-40e8-4f9e-b460-64f68720a1a6-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5t6xx\" (UID: \"68addbce-40e8-4f9e-b460-64f68720a1a6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5t6xx" Jul 15 19:35:38.645367 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:38.645345 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/68addbce-40e8-4f9e-b460-64f68720a1a6-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5t6xx\" (UID: \"68addbce-40e8-4f9e-b460-64f68720a1a6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5t6xx" Jul 15 19:35:38.645493 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:38.645477 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/68addbce-40e8-4f9e-b460-64f68720a1a6-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5t6xx\" (UID: \"68addbce-40e8-4f9e-b460-64f68720a1a6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5t6xx" Jul 15 19:35:38.651721 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:38.651698 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nj5wc\" (UniqueName: \"kubernetes.io/projected/68addbce-40e8-4f9e-b460-64f68720a1a6-kube-api-access-nj5wc\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5t6xx\" (UID: \"68addbce-40e8-4f9e-b460-64f68720a1a6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5t6xx" Jul 15 19:35:38.811602 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:38.811575 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5t6xx" Jul 15 19:35:38.951156 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:38.951126 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-5t6xx"] Jul 15 19:35:38.954436 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:35:38.954407 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68addbce_40e8_4f9e_b460_64f68720a1a6.slice/crio-6d5ffa203f57dc22855c9b851839ba0e9629713c9e3596fc494d5bd1658a830c WatchSource:0}: Error finding container 6d5ffa203f57dc22855c9b851839ba0e9629713c9e3596fc494d5bd1658a830c: Status 404 returned error can't find the container with id 6d5ffa203f57dc22855c9b851839ba0e9629713c9e3596fc494d5bd1658a830c Jul 15 19:35:39.555004 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:39.554965 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5t6xx" event={"ID":"68addbce-40e8-4f9e-b460-64f68720a1a6","Type":"ContainerStarted","Data":"6d5ffa203f57dc22855c9b851839ba0e9629713c9e3596fc494d5bd1658a830c"} Jul 15 19:35:40.518270 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:40.518229 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pngz"] Jul 15 19:35:40.561852 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:40.561819 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5t6xx" event={"ID":"68addbce-40e8-4f9e-b460-64f68720a1a6","Type":"ContainerStarted","Data":"f999b1c0c8d8ba455be7ddffdcb5934cf1b3721d23238af4ff234c5121db4ea4"} Jul 15 19:35:40.583664 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:40.583621 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5t6xx" podStartSLOduration=1.498498149 podStartE2EDuration="2.583607859s" totalImagesPullingTime="1.08510971s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:35:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:35:40.582577867 +0000 UTC m=+854.159579537" watchObservedRunningTime="2026-07-15 19:35:40.583607859 +0000 UTC m=+854.160609576" Jul 15 19:35:48.812188 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:48.812152 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5t6xx" Jul 15 19:35:48.812647 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:48.812209 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5t6xx" Jul 15 19:35:48.824793 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:48.824770 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5t6xx" Jul 15 19:35:49.616514 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:49.616487 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5t6xx" Jul 15 19:35:56.274824 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:56.274795 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2lwhz"] Jul 15 19:35:56.277098 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:56.277080 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2lwhz" Jul 15 19:35:56.279354 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:56.279308 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 15 19:35:56.288109 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:56.288089 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2lwhz"] Jul 15 19:35:56.369799 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:56.369768 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71df2989-d480-45fd-8be7-57e444b62aad-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2lwhz\" (UID: \"71df2989-d480-45fd-8be7-57e444b62aad\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2lwhz" Jul 15 19:35:56.369799 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:56.369802 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71df2989-d480-45fd-8be7-57e444b62aad-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2lwhz\" (UID: \"71df2989-d480-45fd-8be7-57e444b62aad\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2lwhz" Jul 15 19:35:56.369978 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:56.369850 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71df2989-d480-45fd-8be7-57e444b62aad-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2lwhz\" (UID: \"71df2989-d480-45fd-8be7-57e444b62aad\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2lwhz" Jul 15 19:35:56.369978 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:56.369889 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2k2jz\" (UniqueName: \"kubernetes.io/projected/71df2989-d480-45fd-8be7-57e444b62aad-kube-api-access-2k2jz\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2lwhz\" (UID: \"71df2989-d480-45fd-8be7-57e444b62aad\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2lwhz" Jul 15 19:35:56.369978 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:56.369938 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71df2989-d480-45fd-8be7-57e444b62aad-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2lwhz\" (UID: \"71df2989-d480-45fd-8be7-57e444b62aad\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2lwhz" Jul 15 19:35:56.369978 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:56.369965 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71df2989-d480-45fd-8be7-57e444b62aad-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2lwhz\" (UID: \"71df2989-d480-45fd-8be7-57e444b62aad\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2lwhz" Jul 15 19:35:56.470463 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:56.470434 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71df2989-d480-45fd-8be7-57e444b62aad-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2lwhz\" (UID: \"71df2989-d480-45fd-8be7-57e444b62aad\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2lwhz" Jul 15 19:35:56.470594 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:56.470471 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71df2989-d480-45fd-8be7-57e444b62aad-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2lwhz\" (UID: \"71df2989-d480-45fd-8be7-57e444b62aad\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2lwhz" Jul 15 19:35:56.470594 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:56.470507 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71df2989-d480-45fd-8be7-57e444b62aad-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2lwhz\" (UID: \"71df2989-d480-45fd-8be7-57e444b62aad\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2lwhz" Jul 15 19:35:56.470594 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:56.470531 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71df2989-d480-45fd-8be7-57e444b62aad-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2lwhz\" (UID: \"71df2989-d480-45fd-8be7-57e444b62aad\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2lwhz" Jul 15 19:35:56.470594 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:56.470562 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71df2989-d480-45fd-8be7-57e444b62aad-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2lwhz\" (UID: \"71df2989-d480-45fd-8be7-57e444b62aad\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2lwhz" Jul 15 19:35:56.470594 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:56.470588 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2k2jz\" (UniqueName: \"kubernetes.io/projected/71df2989-d480-45fd-8be7-57e444b62aad-kube-api-access-2k2jz\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2lwhz\" (UID: \"71df2989-d480-45fd-8be7-57e444b62aad\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2lwhz" Jul 15 19:35:56.470885 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:56.470861 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71df2989-d480-45fd-8be7-57e444b62aad-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2lwhz\" (UID: \"71df2989-d480-45fd-8be7-57e444b62aad\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2lwhz" Jul 15 19:35:56.470946 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:56.470926 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71df2989-d480-45fd-8be7-57e444b62aad-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2lwhz\" (UID: \"71df2989-d480-45fd-8be7-57e444b62aad\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2lwhz" Jul 15 19:35:56.470994 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:56.470982 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71df2989-d480-45fd-8be7-57e444b62aad-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2lwhz\" (UID: \"71df2989-d480-45fd-8be7-57e444b62aad\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2lwhz" Jul 15 19:35:56.472829 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:56.472806 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71df2989-d480-45fd-8be7-57e444b62aad-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2lwhz\" (UID: \"71df2989-d480-45fd-8be7-57e444b62aad\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2lwhz" Jul 15 19:35:56.473191 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:56.473168 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71df2989-d480-45fd-8be7-57e444b62aad-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2lwhz\" (UID: \"71df2989-d480-45fd-8be7-57e444b62aad\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2lwhz" Jul 15 19:35:56.478723 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:56.478701 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2k2jz\" (UniqueName: \"kubernetes.io/projected/71df2989-d480-45fd-8be7-57e444b62aad-kube-api-access-2k2jz\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2lwhz\" (UID: \"71df2989-d480-45fd-8be7-57e444b62aad\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2lwhz" Jul 15 19:35:56.586832 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:56.586804 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2lwhz" Jul 15 19:35:56.714033 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:56.714009 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2lwhz"] Jul 15 19:35:56.716076 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:35:56.716048 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod71df2989_d480_45fd_8be7_57e444b62aad.slice/crio-d043940981127c8c208a74bf8fe92dbf61f35ed8836e828140101fd88965afdd WatchSource:0}: Error finding container d043940981127c8c208a74bf8fe92dbf61f35ed8836e828140101fd88965afdd: Status 404 returned error can't find the container with id d043940981127c8c208a74bf8fe92dbf61f35ed8836e828140101fd88965afdd Jul 15 19:35:57.647810 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:57.647777 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2lwhz" event={"ID":"71df2989-d480-45fd-8be7-57e444b62aad","Type":"ContainerStarted","Data":"ce85a827767416db1fca69a939d8ca2c6b23d843bfdfabbd8432f98d9554c281"} Jul 15 19:35:57.647810 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:57.647816 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2lwhz" event={"ID":"71df2989-d480-45fd-8be7-57e444b62aad","Type":"ContainerStarted","Data":"d043940981127c8c208a74bf8fe92dbf61f35ed8836e828140101fd88965afdd"} Jul 15 19:35:57.666278 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:57.666229 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2lwhz" podStartSLOduration=1.477304824 podStartE2EDuration="1.666215709s" totalImagesPullingTime="188.910885ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:35:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:35:57.663528549 +0000 UTC m=+871.240530232" watchObservedRunningTime="2026-07-15 19:35:57.666215709 +0000 UTC m=+871.243217380" Jul 15 19:35:58.881096 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:35:58.881065 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pngz"] Jul 15 19:36:06.587294 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:36:06.587258 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2lwhz" Jul 15 19:36:06.587294 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:36:06.587300 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2lwhz" Jul 15 19:36:06.599687 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:36:06.599657 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2lwhz" Jul 15 19:36:06.704693 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:36:06.704666 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2lwhz" Jul 15 19:36:26.928853 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:36:26.928783 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-j2rcv_a2ded0a5-37e2-48fd-98b6-3d64ae279896/console-operator/1.log" Jul 15 19:36:26.930012 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:36:26.929990 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-j2rcv_a2ded0a5-37e2-48fd-98b6-3d64ae279896/console-operator/1.log" Jul 15 19:36:26.936081 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:36:26.936066 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fw6vl_89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21/ovn-acl-logging/0.log" Jul 15 19:36:26.937027 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:36:26.937011 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fw6vl_89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21/ovn-acl-logging/0.log" Jul 15 19:36:48.214702 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:36:48.214666 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-74975f694f-vtxzl"] Jul 15 19:36:48.217373 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:36:48.217350 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-74975f694f-vtxzl" Jul 15 19:36:48.219554 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:36:48.219521 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 15 19:36:48.220182 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:36:48.220165 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-4j5jf\"" Jul 15 19:36:48.236671 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:36:48.236649 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-74975f694f-vtxzl"] Jul 15 19:36:48.273609 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:36:48.273580 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/95c013b6-b56d-4613-b67f-724d6764159c-tls-cert\") pod \"authorino-74975f694f-vtxzl\" (UID: \"95c013b6-b56d-4613-b67f-724d6764159c\") " pod="kuadrant-system/authorino-74975f694f-vtxzl" Jul 15 19:36:48.273721 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:36:48.273626 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbkp6\" (UniqueName: \"kubernetes.io/projected/95c013b6-b56d-4613-b67f-724d6764159c-kube-api-access-hbkp6\") pod \"authorino-74975f694f-vtxzl\" (UID: \"95c013b6-b56d-4613-b67f-724d6764159c\") " pod="kuadrant-system/authorino-74975f694f-vtxzl" Jul 15 19:36:48.374975 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:36:48.374948 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/95c013b6-b56d-4613-b67f-724d6764159c-tls-cert\") pod \"authorino-74975f694f-vtxzl\" (UID: \"95c013b6-b56d-4613-b67f-724d6764159c\") " pod="kuadrant-system/authorino-74975f694f-vtxzl" Jul 15 19:36:48.375101 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:36:48.374984 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbkp6\" (UniqueName: \"kubernetes.io/projected/95c013b6-b56d-4613-b67f-724d6764159c-kube-api-access-hbkp6\") pod \"authorino-74975f694f-vtxzl\" (UID: \"95c013b6-b56d-4613-b67f-724d6764159c\") " pod="kuadrant-system/authorino-74975f694f-vtxzl" Jul 15 19:36:48.377334 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:36:48.377302 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/95c013b6-b56d-4613-b67f-724d6764159c-tls-cert\") pod \"authorino-74975f694f-vtxzl\" (UID: \"95c013b6-b56d-4613-b67f-724d6764159c\") " pod="kuadrant-system/authorino-74975f694f-vtxzl" Jul 15 19:36:48.383546 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:36:48.383519 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbkp6\" (UniqueName: \"kubernetes.io/projected/95c013b6-b56d-4613-b67f-724d6764159c-kube-api-access-hbkp6\") pod \"authorino-74975f694f-vtxzl\" (UID: \"95c013b6-b56d-4613-b67f-724d6764159c\") " pod="kuadrant-system/authorino-74975f694f-vtxzl" Jul 15 19:36:48.525900 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:36:48.525835 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-74975f694f-vtxzl" Jul 15 19:36:48.655748 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:36:48.655722 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-74975f694f-vtxzl"] Jul 15 19:36:48.657681 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:36:48.657656 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod95c013b6_b56d_4613_b67f_724d6764159c.slice/crio-58ae1741c5feff2e1f7efb9c467ff9936b1b3fda966ffd0f60df491ac7923aec WatchSource:0}: Error finding container 58ae1741c5feff2e1f7efb9c467ff9936b1b3fda966ffd0f60df491ac7923aec: Status 404 returned error can't find the container with id 58ae1741c5feff2e1f7efb9c467ff9936b1b3fda966ffd0f60df491ac7923aec Jul 15 19:36:48.911042 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:36:48.911011 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-74975f694f-vtxzl" event={"ID":"95c013b6-b56d-4613-b67f-724d6764159c","Type":"ContainerStarted","Data":"58ae1741c5feff2e1f7efb9c467ff9936b1b3fda966ffd0f60df491ac7923aec"} Jul 15 19:36:49.918701 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:36:49.918664 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-74975f694f-vtxzl" event={"ID":"95c013b6-b56d-4613-b67f-724d6764159c","Type":"ContainerStarted","Data":"cf1a108f22f32978cc3f889ae3ed2bec3b789fdec9efe5729798272701b155d8"} Jul 15 19:36:49.946102 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:36:49.946053 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-74975f694f-vtxzl" podStartSLOduration=1.368806586 podStartE2EDuration="1.946037415s" totalImagesPullingTime="577.230829ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:36:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:36:49.944557305 +0000 UTC m=+923.521559006" watchObservedRunningTime="2026-07-15 19:36:49.946037415 +0000 UTC m=+923.523039130" Jul 15 19:37:26.189836 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:37:26.189796 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pngz"] Jul 15 19:37:37.772055 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:37:37.772017 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pngz"] Jul 15 19:37:46.791368 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:37:46.791339 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pngz"] Jul 15 19:37:56.000942 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:37:56.000861 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pngz"] Jul 15 19:38:07.135266 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:38:07.135234 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pngz"] Jul 15 19:38:36.996694 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:38:36.996660 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pngz"] Jul 15 19:38:49.776718 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:38:49.776684 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pngz"] Jul 15 19:39:25.983862 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:39:25.983789 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pngz"] Jul 15 19:39:36.783300 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:39:36.783269 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pngz"] Jul 15 19:39:47.114325 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:39:47.114293 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pngz"] Jul 15 19:39:54.684103 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:39:54.684068 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pngz"] Jul 15 19:40:05.890586 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:40:05.890550 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pngz"] Jul 15 19:40:15.474915 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:40:15.474880 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pngz"] Jul 15 19:40:26.686303 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:40:26.686270 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pngz"] Jul 15 19:40:36.285887 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:40:36.285847 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pngz"] Jul 15 19:41:00.177616 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:41:00.177539 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pngz"] Jul 15 19:41:11.782369 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:41:11.782335 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pngz"] Jul 15 19:41:26.987982 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:41:26.987951 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-j2rcv_a2ded0a5-37e2-48fd-98b6-3d64ae279896/console-operator/1.log" Jul 15 19:41:26.990852 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:41:26.990828 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-j2rcv_a2ded0a5-37e2-48fd-98b6-3d64ae279896/console-operator/1.log" Jul 15 19:41:26.995259 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:41:26.995230 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fw6vl_89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21/ovn-acl-logging/0.log" Jul 15 19:41:26.998082 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:41:26.998063 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fw6vl_89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21/ovn-acl-logging/0.log" Jul 15 19:41:31.590665 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:41:31.590621 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pngz"] Jul 15 19:41:41.990891 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:41:41.990858 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pngz"] Jul 15 19:41:58.689744 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:41:58.689708 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pngz"] Jul 15 19:42:07.785507 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:42:07.785476 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pngz"] Jul 15 19:42:17.187618 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:42:17.187585 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pngz"] Jul 15 19:42:29.189243 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:42:29.189167 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pngz"] Jul 15 19:42:38.695423 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:42:38.695386 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pngz"] Jul 15 19:44:25.517506 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:44:25.517472 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pngz"] Jul 15 19:44:37.873633 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:44:37.873598 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pngz"] Jul 15 19:44:49.323384 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:44:49.323348 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pngz"] Jul 15 19:45:02.634222 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:45:02.634184 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pngz"] Jul 15 19:45:12.822645 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:45:12.822610 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pngz"] Jul 15 19:45:25.296200 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:45:25.296121 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pngz"] Jul 15 19:45:33.295938 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:45:33.295906 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pngz"] Jul 15 19:45:48.920473 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:45:48.920435 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pngz"] Jul 15 19:45:59.297238 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:45:59.297207 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pngz"] Jul 15 19:46:07.999126 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:46:07.999092 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pngz"] Jul 15 19:46:21.903267 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:46:21.903221 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pngz"] Jul 15 19:46:27.020968 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:46:27.020941 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-j2rcv_a2ded0a5-37e2-48fd-98b6-3d64ae279896/console-operator/1.log" Jul 15 19:46:27.025085 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:46:27.025059 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-j2rcv_a2ded0a5-37e2-48fd-98b6-3d64ae279896/console-operator/1.log" Jul 15 19:46:27.027766 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:46:27.027747 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fw6vl_89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21/ovn-acl-logging/0.log" Jul 15 19:46:27.031845 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:46:27.031827 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fw6vl_89ddcb87-9fb2-4dfe-864e-68c5a5bf2d21/ovn-acl-logging/0.log" Jul 15 19:46:30.999418 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:46:30.999378 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pngz"] Jul 15 19:47:11.902272 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:47:11.902198 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pngz"] Jul 15 19:47:21.488929 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:47:21.488895 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pngz"] Jul 15 19:47:31.513392 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:47:31.513356 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pngz"] Jul 15 19:49:38.950063 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:49:38.950024 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pngz"] Jul 15 19:49:50.308211 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:49:50.308132 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pngz"] Jul 15 19:49:59.492944 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:49:59.492903 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pngz"] Jul 15 19:50:11.908773 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:11.908736 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pngz"] Jul 15 19:50:17.137855 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:17.137819 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-74975f694f-vtxzl_95c013b6-b56d-4613-b67f-724d6764159c/authorino/0.log" Jul 15 19:50:21.839451 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:21.839419 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-8pngz"] Jul 15 19:50:23.665173 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:23.665140 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-df875cfdf-bp4cf_ce2beebd-634e-404f-83ba-d169ed6f278e/copy-manifests/0.log" Jul 15 19:50:23.673680 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:23.673659 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-df875cfdf-bp4cf_ce2beebd-634e-404f-83ba-d169ed6f278e/manager/0.log" Jul 15 19:50:25.503898 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:25.503863 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cpkdr_7841fdc5-da16-4565-bb38-324b24b3916c/pull/0.log" Jul 15 19:50:25.510974 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:25.510950 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cpkdr_7841fdc5-da16-4565-bb38-324b24b3916c/extract/0.log" Jul 15 19:50:25.519952 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:25.519932 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cpkdr_7841fdc5-da16-4565-bb38-324b24b3916c/util/0.log" Jul 15 19:50:25.666861 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:25.666829 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqnwm_4ec38921-1795-454a-9617-7a9b40cf7ab1/util/0.log" Jul 15 19:50:25.676096 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:25.676073 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqnwm_4ec38921-1795-454a-9617-7a9b40cf7ab1/pull/0.log" Jul 15 19:50:25.684256 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:25.684235 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqnwm_4ec38921-1795-454a-9617-7a9b40cf7ab1/extract/0.log" Jul 15 19:50:25.814885 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:25.814861 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t86m4_8ad06c01-d62c-4d59-b1b4-c77854454d60/util/0.log" Jul 15 19:50:25.830378 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:25.830356 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t86m4_8ad06c01-d62c-4d59-b1b4-c77854454d60/pull/0.log" Jul 15 19:50:25.850038 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:25.850008 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t86m4_8ad06c01-d62c-4d59-b1b4-c77854454d60/extract/0.log" Jul 15 19:50:25.973983 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:25.973958 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l46z4_35defb98-7604-4ea6-ac0c-193c0cb0bed0/pull/0.log" Jul 15 19:50:25.982358 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:25.982335 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l46z4_35defb98-7604-4ea6-ac0c-193c0cb0bed0/extract/0.log" Jul 15 19:50:25.989780 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:25.989761 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l46z4_35defb98-7604-4ea6-ac0c-193c0cb0bed0/util/0.log" Jul 15 19:50:26.114415 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:26.114332 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-74975f694f-vtxzl_95c013b6-b56d-4613-b67f-724d6764159c/authorino/0.log" Jul 15 19:50:26.469303 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:26.469219 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-hc7tq_807d4176-fe09-46d9-8f69-c26648484006/kuadrant-console-plugin/0.log" Jul 15 19:50:26.851602 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:26.851568 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-8pngz_65ef11f8-41ff-41ac-a0cc-e85de469abdb/limitador/0.log" Jul 15 19:50:27.585338 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:27.585298 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6d4465d888-q94gr_44825d03-933d-4f48-8c7d-986f153def49/kube-auth-proxy/0.log" Jul 15 19:50:28.207463 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:28.207434 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6c44cd97bd-5kbj2_d1aea75b-a17d-49fe-aa61-2969e7e720d1/router/0.log" Jul 15 19:50:29.085212 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:29.085180 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-5t6xx_68addbce-40e8-4f9e-b460-64f68720a1a6/main/0.log" Jul 15 19:50:29.213572 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:29.213544 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-2lwhz_71df2989-d480-45fd-8be7-57e444b62aad/main/0.log" Jul 15 19:50:38.409657 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:38.409623 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-pn46g_05d64bcb-1ac1-47aa-a557-63ba6c2011ab/global-pull-secret-syncer/0.log" Jul 15 19:50:38.591093 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:38.591061 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-xx47b_1820cfdf-910d-42da-bafb-fb076d565b78/konnectivity-agent/0.log" Jul 15 19:50:38.718424 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:38.718341 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-62.ec2.internal_db478e14aba317d5490063a0b6ec4a2a/haproxy/0.log" Jul 15 19:50:43.636465 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:43.636433 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cpkdr_7841fdc5-da16-4565-bb38-324b24b3916c/extract/0.log" Jul 15 19:50:43.703675 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:43.703646 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cpkdr_7841fdc5-da16-4565-bb38-324b24b3916c/util/0.log" Jul 15 19:50:43.742399 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:43.742368 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cpkdr_7841fdc5-da16-4565-bb38-324b24b3916c/pull/0.log" Jul 15 19:50:43.781332 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:43.781280 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqnwm_4ec38921-1795-454a-9617-7a9b40cf7ab1/extract/0.log" Jul 15 19:50:43.809191 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:43.809164 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqnwm_4ec38921-1795-454a-9617-7a9b40cf7ab1/util/0.log" Jul 15 19:50:43.848951 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:43.848927 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tqnwm_4ec38921-1795-454a-9617-7a9b40cf7ab1/pull/0.log" Jul 15 19:50:43.884923 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:43.884894 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t86m4_8ad06c01-d62c-4d59-b1b4-c77854454d60/extract/0.log" Jul 15 19:50:43.923091 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:43.923033 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t86m4_8ad06c01-d62c-4d59-b1b4-c77854454d60/util/0.log" Jul 15 19:50:43.960734 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:43.960712 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t86m4_8ad06c01-d62c-4d59-b1b4-c77854454d60/pull/0.log" Jul 15 19:50:43.995617 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:43.995600 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l46z4_35defb98-7604-4ea6-ac0c-193c0cb0bed0/extract/0.log" Jul 15 19:50:44.030828 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:44.030805 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l46z4_35defb98-7604-4ea6-ac0c-193c0cb0bed0/util/0.log" Jul 15 19:50:44.062133 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:44.062114 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l46z4_35defb98-7604-4ea6-ac0c-193c0cb0bed0/pull/0.log" Jul 15 19:50:44.451264 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:44.451215 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-74975f694f-vtxzl_95c013b6-b56d-4613-b67f-724d6764159c/authorino/0.log" Jul 15 19:50:44.556689 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:44.556662 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-hc7tq_807d4176-fe09-46d9-8f69-c26648484006/kuadrant-console-plugin/0.log" Jul 15 19:50:44.706743 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:44.706664 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-8pngz_65ef11f8-41ff-41ac-a0cc-e85de469abdb/limitador/0.log" Jul 15 19:50:46.630477 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:46.630446 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-fk99v_68f7eee7-04d4-49ff-89d7-f10b984a5f55/kube-state-metrics/0.log" Jul 15 19:50:46.660141 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:46.660116 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-fk99v_68f7eee7-04d4-49ff-89d7-f10b984a5f55/kube-rbac-proxy-main/0.log" Jul 15 19:50:46.685934 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:46.685908 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-fk99v_68f7eee7-04d4-49ff-89d7-f10b984a5f55/kube-rbac-proxy-self/0.log" Jul 15 19:50:46.751158 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:46.751125 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-sx7ln_83282529-556c-418b-98bb-874278f184b7/monitoring-plugin/0.log" Jul 15 19:50:46.861053 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:46.861027 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ptl2w_44d02d02-4986-4a68-8398-894bdd356164/node-exporter/0.log" Jul 15 19:50:46.883295 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:46.883222 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ptl2w_44d02d02-4986-4a68-8398-894bdd356164/kube-rbac-proxy/0.log" Jul 15 19:50:46.909058 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:46.909037 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ptl2w_44d02d02-4986-4a68-8398-894bdd356164/init-textfile/0.log" Jul 15 19:50:47.045564 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:47.045535 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-4strs_7ffe651c-2927-43a4-96f1-f7bf31b7f598/kube-rbac-proxy-main/0.log" Jul 15 19:50:47.076599 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:47.076575 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-4strs_7ffe651c-2927-43a4-96f1-f7bf31b7f598/kube-rbac-proxy-self/0.log" Jul 15 19:50:47.104645 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:47.104607 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-4strs_7ffe651c-2927-43a4-96f1-f7bf31b7f598/openshift-state-metrics/0.log" Jul 15 19:50:47.431578 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:47.431554 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-zqw6v_49f71022-3768-4912-8953-29b19518fe3a/prometheus-operator-admission-webhook/0.log" Jul 15 19:50:47.972012 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:47.971983 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-crt2f/perf-node-gather-daemonset-tqpqw"] Jul 15 19:50:47.975473 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:47.975451 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-crt2f/perf-node-gather-daemonset-tqpqw" Jul 15 19:50:47.982497 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:47.982477 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-crt2f\"/\"openshift-service-ca.crt\"" Jul 15 19:50:47.982997 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:47.982981 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-crt2f\"/\"default-dockercfg-vddw7\"" Jul 15 19:50:47.983256 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:47.983239 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-crt2f\"/\"kube-root-ca.crt\"" Jul 15 19:50:47.999204 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:47.999176 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-crt2f/perf-node-gather-daemonset-tqpqw"] Jul 15 19:50:48.078213 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:48.078189 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/229f4b57-16c7-443b-be8b-38d5c0d4df13-sys\") pod \"perf-node-gather-daemonset-tqpqw\" (UID: \"229f4b57-16c7-443b-be8b-38d5c0d4df13\") " pod="openshift-must-gather-crt2f/perf-node-gather-daemonset-tqpqw" Jul 15 19:50:48.078374 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:48.078219 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/229f4b57-16c7-443b-be8b-38d5c0d4df13-lib-modules\") pod \"perf-node-gather-daemonset-tqpqw\" (UID: \"229f4b57-16c7-443b-be8b-38d5c0d4df13\") " pod="openshift-must-gather-crt2f/perf-node-gather-daemonset-tqpqw" Jul 15 19:50:48.078374 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:48.078247 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/229f4b57-16c7-443b-be8b-38d5c0d4df13-proc\") pod \"perf-node-gather-daemonset-tqpqw\" (UID: \"229f4b57-16c7-443b-be8b-38d5c0d4df13\") " pod="openshift-must-gather-crt2f/perf-node-gather-daemonset-tqpqw" Jul 15 19:50:48.078374 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:48.078352 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmml6\" (UniqueName: \"kubernetes.io/projected/229f4b57-16c7-443b-be8b-38d5c0d4df13-kube-api-access-vmml6\") pod \"perf-node-gather-daemonset-tqpqw\" (UID: \"229f4b57-16c7-443b-be8b-38d5c0d4df13\") " pod="openshift-must-gather-crt2f/perf-node-gather-daemonset-tqpqw" Jul 15 19:50:48.078514 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:48.078383 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/229f4b57-16c7-443b-be8b-38d5c0d4df13-podres\") pod \"perf-node-gather-daemonset-tqpqw\" (UID: \"229f4b57-16c7-443b-be8b-38d5c0d4df13\") " pod="openshift-must-gather-crt2f/perf-node-gather-daemonset-tqpqw" Jul 15 19:50:48.179627 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:48.179597 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/229f4b57-16c7-443b-be8b-38d5c0d4df13-sys\") pod \"perf-node-gather-daemonset-tqpqw\" (UID: \"229f4b57-16c7-443b-be8b-38d5c0d4df13\") " pod="openshift-must-gather-crt2f/perf-node-gather-daemonset-tqpqw" Jul 15 19:50:48.179627 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:48.179629 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/229f4b57-16c7-443b-be8b-38d5c0d4df13-lib-modules\") pod \"perf-node-gather-daemonset-tqpqw\" (UID: \"229f4b57-16c7-443b-be8b-38d5c0d4df13\") " pod="openshift-must-gather-crt2f/perf-node-gather-daemonset-tqpqw" Jul 15 19:50:48.179844 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:48.179663 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/229f4b57-16c7-443b-be8b-38d5c0d4df13-proc\") pod \"perf-node-gather-daemonset-tqpqw\" (UID: \"229f4b57-16c7-443b-be8b-38d5c0d4df13\") " pod="openshift-must-gather-crt2f/perf-node-gather-daemonset-tqpqw" Jul 15 19:50:48.179844 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:48.179706 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmml6\" (UniqueName: \"kubernetes.io/projected/229f4b57-16c7-443b-be8b-38d5c0d4df13-kube-api-access-vmml6\") pod \"perf-node-gather-daemonset-tqpqw\" (UID: \"229f4b57-16c7-443b-be8b-38d5c0d4df13\") " pod="openshift-must-gather-crt2f/perf-node-gather-daemonset-tqpqw" Jul 15 19:50:48.179844 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:48.179723 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/229f4b57-16c7-443b-be8b-38d5c0d4df13-podres\") pod \"perf-node-gather-daemonset-tqpqw\" (UID: \"229f4b57-16c7-443b-be8b-38d5c0d4df13\") " pod="openshift-must-gather-crt2f/perf-node-gather-daemonset-tqpqw" Jul 15 19:50:48.179844 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:48.179724 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/229f4b57-16c7-443b-be8b-38d5c0d4df13-sys\") pod \"perf-node-gather-daemonset-tqpqw\" (UID: \"229f4b57-16c7-443b-be8b-38d5c0d4df13\") " pod="openshift-must-gather-crt2f/perf-node-gather-daemonset-tqpqw" Jul 15 19:50:48.179844 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:48.179775 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/229f4b57-16c7-443b-be8b-38d5c0d4df13-lib-modules\") pod \"perf-node-gather-daemonset-tqpqw\" (UID: \"229f4b57-16c7-443b-be8b-38d5c0d4df13\") " pod="openshift-must-gather-crt2f/perf-node-gather-daemonset-tqpqw" Jul 15 19:50:48.179844 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:48.179799 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/229f4b57-16c7-443b-be8b-38d5c0d4df13-proc\") pod \"perf-node-gather-daemonset-tqpqw\" (UID: \"229f4b57-16c7-443b-be8b-38d5c0d4df13\") " pod="openshift-must-gather-crt2f/perf-node-gather-daemonset-tqpqw" Jul 15 19:50:48.180107 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:48.179855 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/229f4b57-16c7-443b-be8b-38d5c0d4df13-podres\") pod \"perf-node-gather-daemonset-tqpqw\" (UID: \"229f4b57-16c7-443b-be8b-38d5c0d4df13\") " pod="openshift-must-gather-crt2f/perf-node-gather-daemonset-tqpqw" Jul 15 19:50:48.188911 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:48.188888 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmml6\" (UniqueName: \"kubernetes.io/projected/229f4b57-16c7-443b-be8b-38d5c0d4df13-kube-api-access-vmml6\") pod \"perf-node-gather-daemonset-tqpqw\" (UID: \"229f4b57-16c7-443b-be8b-38d5c0d4df13\") " pod="openshift-must-gather-crt2f/perf-node-gather-daemonset-tqpqw" Jul 15 19:50:48.284918 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:48.284833 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-crt2f/perf-node-gather-daemonset-tqpqw" Jul 15 19:50:48.415230 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:48.415207 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-crt2f/perf-node-gather-daemonset-tqpqw"] Jul 15 19:50:48.417000 ip-10-0-141-62 kubenswrapper[2577]: W0715 19:50:48.416971 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod229f4b57_16c7_443b_be8b_38d5c0d4df13.slice/crio-080505e2864d4eca253b54b10e51f0e942836fd2e8a7dfecd30f7797ad66d9a0 WatchSource:0}: Error finding container 080505e2864d4eca253b54b10e51f0e942836fd2e8a7dfecd30f7797ad66d9a0: Status 404 returned error can't find the container with id 080505e2864d4eca253b54b10e51f0e942836fd2e8a7dfecd30f7797ad66d9a0 Jul 15 19:50:49.022172 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:49.022135 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-w4b96_152d1d6b-5ee1-40c2-80a8-388091a57c58/networking-console-plugin/0.log" Jul 15 19:50:49.171665 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:49.171627 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-crt2f/perf-node-gather-daemonset-tqpqw" event={"ID":"229f4b57-16c7-443b-be8b-38d5c0d4df13","Type":"ContainerStarted","Data":"db40eefc373df94a6f127a6cc65c0b8f665e06da5220c717e3463332414500b8"} Jul 15 19:50:49.171830 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:49.171676 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-crt2f/perf-node-gather-daemonset-tqpqw" event={"ID":"229f4b57-16c7-443b-be8b-38d5c0d4df13","Type":"ContainerStarted","Data":"080505e2864d4eca253b54b10e51f0e942836fd2e8a7dfecd30f7797ad66d9a0"} Jul 15 19:50:49.171830 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:49.171807 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-crt2f/perf-node-gather-daemonset-tqpqw" Jul 15 19:50:49.189219 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:49.189175 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-crt2f/perf-node-gather-daemonset-tqpqw" podStartSLOduration=2.189163324 podStartE2EDuration="2.189163324s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:50:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:50:49.187486841 +0000 UTC m=+1762.764488532" watchObservedRunningTime="2026-07-15 19:50:49.189163324 +0000 UTC m=+1762.766164995" Jul 15 19:50:49.568392 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:49.568345 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-j2rcv_a2ded0a5-37e2-48fd-98b6-3d64ae279896/console-operator/1.log" Jul 15 19:50:49.581493 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:49.581460 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-j2rcv_a2ded0a5-37e2-48fd-98b6-3d64ae279896/console-operator/2.log" Jul 15 19:50:50.708566 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:50.708538 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-wnl4r_41ea5a11-d522-4aa7-a948-c9eba82c1bc6/volume-data-source-validator/0.log" Jul 15 19:50:51.999063 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:51.999032 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5zrgr_56c7bcac-8a82-4c32-8658-1db0bad8d228/dns/0.log" Jul 15 19:50:52.030910 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:52.030888 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5zrgr_56c7bcac-8a82-4c32-8658-1db0bad8d228/kube-rbac-proxy/0.log" Jul 15 19:50:52.222381 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:52.222356 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9vzln_299d8c27-2943-40f5-8810-67bcfcbbb3a2/dns-node-resolver/0.log" Jul 15 19:50:52.998574 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:52.998549 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-c8jtk_b6d81326-425e-4a61-a7db-8790d035bd50/node-ca/0.log" Jul 15 19:50:54.237286 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:54.237254 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6d4465d888-q94gr_44825d03-933d-4f48-8c7d-986f153def49/kube-auth-proxy/0.log" Jul 15 19:50:54.428178 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:54.428120 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6c44cd97bd-5kbj2_d1aea75b-a17d-49fe-aa61-2969e7e720d1/router/0.log" Jul 15 19:50:55.136223 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:55.136188 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rdqmw_cf8428ad-0fb7-4108-a41f-8fd01ec35c5c/serve-healthcheck-canary/0.log" Jul 15 19:50:55.186261 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:55.186237 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-crt2f/perf-node-gather-daemonset-tqpqw" Jul 15 19:50:55.859354 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:55.859304 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-q6q5q_3c70c39c-c761-4bf8-a95c-f260eebd731c/kube-rbac-proxy/0.log" Jul 15 19:50:55.885743 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:55.885718 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-q6q5q_3c70c39c-c761-4bf8-a95c-f260eebd731c/exporter/0.log" Jul 15 19:50:55.917242 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:55.917215 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-q6q5q_3c70c39c-c761-4bf8-a95c-f260eebd731c/extractor/0.log" Jul 15 19:50:58.694650 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:58.694614 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6d4465d888-q94gr_44825d03-933d-4f48-8c7d-986f153def49/kube-auth-proxy/0.log" Jul 15 19:50:58.893945 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:58.893913 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6c44cd97bd-5kbj2_d1aea75b-a17d-49fe-aa61-2969e7e720d1/router/0.log" Jul 15 19:50:59.036676 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:59.036596 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-df875cfdf-bp4cf_ce2beebd-634e-404f-83ba-d169ed6f278e/manager/0.log" Jul 15 19:50:59.073118 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:50:59.073092 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-df875cfdf-bp4cf_ce2beebd-634e-404f-83ba-d169ed6f278e/copy-manifests/0.log" Jul 15 19:51:00.152653 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:51:00.152618 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6d4465d888-q94gr_44825d03-933d-4f48-8c7d-986f153def49/kube-auth-proxy/0.log" Jul 15 19:51:00.288675 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:51:00.288642 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6c44cd97bd-5kbj2_d1aea75b-a17d-49fe-aa61-2969e7e720d1/router/0.log" Jul 15 19:51:00.613001 ip-10-0-141-62 kubenswrapper[2577]: I0715 19:51:00.612973 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6788c996f8-phgb4_9bbe494d-9024-4dfb-a6c0-b5a74466304b/manager/0.log"