Jul 12 18:17:06.746913 ip-10-0-141-133 systemd[1]: Starting Kubernetes Kubelet... Jul 12 18:17:08.475974 ip-10-0-141-133 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 12 18:17:08.475974 ip-10-0-141-133 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 12 18:17:08.475974 ip-10-0-141-133 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 12 18:17:08.475974 ip-10-0-141-133 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 12 18:17:08.475974 ip-10-0-141-133 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 12 18:17:08.475974 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.436302 2577 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 12 18:17:08.475974 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447026 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 18:17:08.475974 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447046 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 18:17:08.475974 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447050 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 18:17:08.475974 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447054 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 18:17:08.475974 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447058 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 18:17:08.475974 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447062 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 18:17:08.475974 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447065 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 18:17:08.475974 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447068 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 18:17:08.476905 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447071 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 18:17:08.476905 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447074 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 18:17:08.476905 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447077 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 18:17:08.476905 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447081 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 18:17:08.476905 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447084 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 18:17:08.476905 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447087 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 18:17:08.476905 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447090 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 18:17:08.476905 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447093 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 18:17:08.476905 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447096 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 18:17:08.476905 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447098 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 18:17:08.476905 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447101 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 18:17:08.476905 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447106 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 18:17:08.476905 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447109 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 18:17:08.476905 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447114 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 18:17:08.476905 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447117 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 18:17:08.476905 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447120 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 18:17:08.476905 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447123 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 18:17:08.476905 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447126 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 18:17:08.476905 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447129 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 18:17:08.477586 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447136 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 18:17:08.477586 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447139 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 18:17:08.477586 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447142 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 18:17:08.477586 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447145 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 18:17:08.477586 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447148 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 18:17:08.477586 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447150 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 18:17:08.477586 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447153 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 18:17:08.477586 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447156 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 18:17:08.477586 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447159 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 18:17:08.477586 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447161 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 18:17:08.477586 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447164 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 18:17:08.477586 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447167 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 18:17:08.477586 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447170 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 18:17:08.477586 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447173 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 18:17:08.477586 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447176 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 18:17:08.477586 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447178 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 18:17:08.477586 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447181 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 18:17:08.477586 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447184 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 18:17:08.477586 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447187 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 18:17:08.478213 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447189 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 18:17:08.478213 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447192 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 18:17:08.478213 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447195 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 18:17:08.478213 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447198 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 18:17:08.478213 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447209 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 18:17:08.478213 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447213 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 18:17:08.478213 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447217 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 18:17:08.478213 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447220 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 18:17:08.478213 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447224 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 18:17:08.478213 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447227 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 18:17:08.478213 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447230 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 18:17:08.478213 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447232 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 18:17:08.478213 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447235 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 18:17:08.478213 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447238 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 18:17:08.478213 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447242 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 18:17:08.478213 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447245 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 18:17:08.478213 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447248 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 18:17:08.478213 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447251 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 18:17:08.478213 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447254 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 18:17:08.479108 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447257 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 18:17:08.479108 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447260 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 18:17:08.479108 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447262 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 18:17:08.479108 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447266 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 18:17:08.479108 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447269 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 18:17:08.479108 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447272 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 18:17:08.479108 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447275 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 18:17:08.479108 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447277 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 18:17:08.479108 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447280 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 18:17:08.479108 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447284 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 18:17:08.479108 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447288 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 18:17:08.479108 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447291 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 18:17:08.479108 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447294 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 18:17:08.479108 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447297 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 18:17:08.479108 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447300 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 18:17:08.479108 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447302 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 18:17:08.479108 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447306 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 18:17:08.479108 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447308 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 18:17:08.479108 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447318 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 18:17:08.479108 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447321 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 18:17:08.479723 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447324 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 12 18:17:08.479723 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447327 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 18:17:08.479723 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447330 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 18:17:08.479723 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447332 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 18:17:08.479723 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447335 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 18:17:08.479723 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447338 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 18:17:08.479723 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447341 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 18:17:08.479723 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447344 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 18:17:08.479723 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447347 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 18:17:08.479723 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447350 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 18:17:08.479723 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447352 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 18:17:08.479723 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447355 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 18:17:08.479723 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447358 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 18:17:08.479723 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.447361 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 18:17:08.479723 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448277 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 18:17:08.479723 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448286 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 18:17:08.479723 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448290 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 18:17:08.479723 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448293 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 18:17:08.479723 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448296 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 18:17:08.479723 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448299 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 18:17:08.480494 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448302 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 12 18:17:08.480494 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448305 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 18:17:08.480494 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448308 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 18:17:08.480494 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448310 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 18:17:08.480494 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448313 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 18:17:08.480494 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448316 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 18:17:08.480494 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448320 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 18:17:08.480494 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448322 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 18:17:08.480494 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448326 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 18:17:08.480494 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448328 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 18:17:08.480494 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448331 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 18:17:08.480494 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448341 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 18:17:08.480494 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448344 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 18:17:08.480494 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448346 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 18:17:08.480494 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448349 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 18:17:08.480494 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448354 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 18:17:08.480494 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448358 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 18:17:08.480494 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448361 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 18:17:08.480494 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448364 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 18:17:08.480494 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448366 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 18:17:08.481453 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448369 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 18:17:08.481453 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448372 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 18:17:08.481453 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448375 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 18:17:08.481453 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448378 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 18:17:08.481453 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448381 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 18:17:08.481453 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448384 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 18:17:08.481453 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448386 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 18:17:08.481453 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448389 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 18:17:08.481453 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448392 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 18:17:08.481453 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448396 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 18:17:08.481453 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448400 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 18:17:08.481453 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448403 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 18:17:08.481453 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448406 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 18:17:08.481453 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448409 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 18:17:08.481453 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448412 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 18:17:08.481453 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448414 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 18:17:08.481453 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448417 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 18:17:08.481453 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448419 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 18:17:08.481453 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448423 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 18:17:08.482467 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448425 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 18:17:08.482467 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448429 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 18:17:08.482467 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448432 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 18:17:08.482467 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448435 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 18:17:08.482467 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448438 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 18:17:08.482467 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448447 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 18:17:08.482467 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448450 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 18:17:08.482467 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448453 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 18:17:08.482467 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448456 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 18:17:08.482467 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448458 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 18:17:08.482467 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448462 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 18:17:08.482467 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448465 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 18:17:08.482467 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448467 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 18:17:08.482467 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448470 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 18:17:08.482467 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448473 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 18:17:08.482467 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448476 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 18:17:08.482467 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448478 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 18:17:08.482467 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448481 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 18:17:08.482467 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448484 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 18:17:08.482467 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448486 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 18:17:08.483114 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448489 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 18:17:08.483114 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448492 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 18:17:08.483114 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448495 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 18:17:08.483114 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448498 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 18:17:08.483114 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448500 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 18:17:08.483114 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448503 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 18:17:08.483114 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448506 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 18:17:08.483114 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448508 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 18:17:08.483114 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448511 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 18:17:08.483114 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448514 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 18:17:08.483114 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448516 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 18:17:08.483114 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448519 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 18:17:08.483114 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448522 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 18:17:08.483114 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448525 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 18:17:08.483114 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448527 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 18:17:08.483114 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448530 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 18:17:08.483114 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448533 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 18:17:08.483114 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448537 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 18:17:08.483114 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448546 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 18:17:08.483114 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448549 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 18:17:08.484050 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448553 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 18:17:08.484050 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448556 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 18:17:08.484050 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448567 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 18:17:08.484050 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448572 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 18:17:08.484050 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448575 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 18:17:08.484050 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448578 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 18:17:08.484050 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448581 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 18:17:08.484050 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448584 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 18:17:08.484050 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448587 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 18:17:08.484050 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448589 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 18:17:08.484050 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448592 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 18:17:08.484050 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448595 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 18:17:08.484050 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448597 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 18:17:08.484050 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.448600 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 18:17:08.484050 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449092 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 18:17:08.484050 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449097 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 18:17:08.484050 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449100 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 18:17:08.484050 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449104 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 18:17:08.484050 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449109 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 18:17:08.484050 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449112 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 18:17:08.484803 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449114 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 18:17:08.484803 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449117 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 18:17:08.484803 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449120 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 18:17:08.484803 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449124 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 18:17:08.484803 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449126 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 18:17:08.484803 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449129 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 18:17:08.484803 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449132 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 18:17:08.484803 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449134 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 18:17:08.484803 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449137 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 18:17:08.484803 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449140 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 18:17:08.484803 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449149 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 18:17:08.484803 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449152 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 18:17:08.484803 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449155 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 18:17:08.484803 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449157 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 18:17:08.484803 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449160 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 18:17:08.484803 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449163 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 18:17:08.484803 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449166 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 18:17:08.484803 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449174 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 18:17:08.484803 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449177 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 18:17:08.487412 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449180 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 18:17:08.487412 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449184 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 18:17:08.487412 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449188 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 18:17:08.487412 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449190 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 18:17:08.487412 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449193 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 18:17:08.487412 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449196 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 18:17:08.487412 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449198 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 18:17:08.487412 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449201 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 18:17:08.487412 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449204 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 18:17:08.487412 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449206 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 18:17:08.487412 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449209 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 18:17:08.487412 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449213 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 12 18:17:08.487412 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449216 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 18:17:08.487412 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449219 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 18:17:08.487412 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449223 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 18:17:08.487412 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449225 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 18:17:08.487412 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449228 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 18:17:08.487412 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449231 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 18:17:08.487412 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449233 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 18:17:08.487412 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449236 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 18:17:08.488116 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449239 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 18:17:08.488116 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449242 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 18:17:08.488116 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449245 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 18:17:08.488116 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449247 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 18:17:08.488116 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449256 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 18:17:08.488116 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449259 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 18:17:08.488116 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449262 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 18:17:08.488116 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449265 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 18:17:08.488116 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449268 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 18:17:08.488116 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449270 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 18:17:08.488116 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449273 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 18:17:08.488116 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449275 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 18:17:08.488116 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449278 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 18:17:08.488116 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449281 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 18:17:08.488116 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449284 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 18:17:08.488116 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449286 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 18:17:08.488116 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449289 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 18:17:08.488116 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449291 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 18:17:08.488116 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449298 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 18:17:08.488116 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449301 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 18:17:08.576681 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449303 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 18:17:08.576681 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449306 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 18:17:08.576681 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449309 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 18:17:08.576681 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449313 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 18:17:08.576681 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449316 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 18:17:08.576681 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449319 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 18:17:08.576681 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449322 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 18:17:08.576681 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449324 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 18:17:08.576681 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449327 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 18:17:08.576681 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449330 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 18:17:08.576681 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449332 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 18:17:08.576681 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449335 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 18:17:08.576681 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449343 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 18:17:08.576681 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449347 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 18:17:08.576681 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449350 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 18:17:08.576681 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449354 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 18:17:08.576681 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449357 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 18:17:08.576681 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449367 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 18:17:08.576681 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449371 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 18:17:08.537442 ip-10-0-141-133 systemd[1]: Started Kubernetes Kubelet. Jul 12 18:17:08.577957 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449374 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 18:17:08.577957 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449377 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 18:17:08.577957 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449380 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 18:17:08.577957 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449382 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 18:17:08.577957 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449385 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 18:17:08.577957 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449388 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 18:17:08.577957 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449391 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 18:17:08.577957 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449393 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 18:17:08.577957 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449396 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 18:17:08.577957 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449398 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 18:17:08.577957 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449401 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 18:17:08.577957 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449404 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 18:17:08.577957 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449407 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 18:17:08.577957 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449409 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 18:17:08.577957 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.449412 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 18:17:08.577957 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449489 2577 flags.go:64] FLAG: --address="0.0.0.0" Jul 12 18:17:08.577957 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449498 2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 12 18:17:08.577957 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449511 2577 flags.go:64] FLAG: --anonymous-auth="true" Jul 12 18:17:08.577957 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449516 2577 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 12 18:17:08.577957 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449521 2577 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 12 18:17:08.577957 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449524 2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 12 18:17:08.581492 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449529 2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 12 18:17:08.581492 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449534 2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 12 18:17:08.581492 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449537 2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 12 18:17:08.581492 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449540 2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 12 18:17:08.581492 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449545 2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 12 18:17:08.581492 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449549 2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 12 18:17:08.581492 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449552 2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 12 18:17:08.581492 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449557 2577 flags.go:64] FLAG: --cgroup-root="" Jul 12 18:17:08.581492 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449560 2577 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 12 18:17:08.581492 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449563 2577 flags.go:64] FLAG: --client-ca-file="" Jul 12 18:17:08.581492 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449573 2577 flags.go:64] FLAG: --cloud-provider="external" Jul 12 18:17:08.581492 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449577 2577 flags.go:64] FLAG: --cluster-dns="[]" Jul 12 18:17:08.581492 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449584 2577 flags.go:64] FLAG: --cluster-domain="" Jul 12 18:17:08.581492 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449587 2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 12 18:17:08.581492 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449590 2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 12 18:17:08.581492 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449593 2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 12 18:17:08.581492 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449597 2577 flags.go:64] FLAG: --container-log-max-files="5" Jul 12 18:17:08.581492 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449601 2577 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 12 18:17:08.581492 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449604 2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 12 18:17:08.581492 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449607 2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 12 18:17:08.581492 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449610 2577 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 12 18:17:08.581492 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449613 2577 flags.go:64] FLAG: --contention-profiling="false" Jul 12 18:17:08.581492 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449617 2577 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 12 18:17:08.581492 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449620 2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 12 18:17:08.582289 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449623 2577 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 12 18:17:08.582289 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449626 2577 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 12 18:17:08.582289 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449630 2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 12 18:17:08.582289 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449635 2577 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 12 18:17:08.582289 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449639 2577 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 12 18:17:08.582289 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449642 2577 flags.go:64] FLAG: --enable-load-reader="false" Jul 12 18:17:08.582289 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449645 2577 flags.go:64] FLAG: --enable-server="true" Jul 12 18:17:08.582289 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449648 2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 12 18:17:08.582289 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449656 2577 flags.go:64] FLAG: --event-burst="100" Jul 12 18:17:08.582289 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449660 2577 flags.go:64] FLAG: --event-qps="50" Jul 12 18:17:08.582289 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449663 2577 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 12 18:17:08.582289 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449666 2577 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 12 18:17:08.582289 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449671 2577 flags.go:64] FLAG: --eviction-hard="" Jul 12 18:17:08.582289 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449675 2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 12 18:17:08.582289 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449678 2577 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 12 18:17:08.582289 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449681 2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 12 18:17:08.582289 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449684 2577 flags.go:64] FLAG: --eviction-soft="" Jul 12 18:17:08.582289 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449688 2577 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 12 18:17:08.582289 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449691 2577 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 12 18:17:08.582289 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449715 2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 12 18:17:08.582289 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449719 2577 flags.go:64] FLAG: --experimental-mounter-path="" Jul 12 18:17:08.582289 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449722 2577 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 12 18:17:08.582289 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449725 2577 flags.go:64] FLAG: --fail-swap-on="true" Jul 12 18:17:08.582289 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449728 2577 flags.go:64] FLAG: --feature-gates="" Jul 12 18:17:08.582289 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449732 2577 flags.go:64] FLAG: --file-check-frequency="20s" Jul 12 18:17:08.583531 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449735 2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 12 18:17:08.583531 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449739 2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 12 18:17:08.583531 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449742 2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 12 18:17:08.583531 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449745 2577 flags.go:64] FLAG: --healthz-port="10248" Jul 12 18:17:08.583531 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449748 2577 flags.go:64] FLAG: --help="false" Jul 12 18:17:08.583531 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449752 2577 flags.go:64] FLAG: --hostname-override="ip-10-0-141-133.ec2.internal" Jul 12 18:17:08.583531 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449755 2577 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 12 18:17:08.583531 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449758 2577 flags.go:64] FLAG: --http-check-frequency="20s" Jul 12 18:17:08.583531 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449761 2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 12 18:17:08.583531 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449765 2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 12 18:17:08.583531 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449770 2577 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 12 18:17:08.583531 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449773 2577 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 12 18:17:08.583531 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449776 2577 flags.go:64] FLAG: --image-service-endpoint="" Jul 12 18:17:08.583531 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449779 2577 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 12 18:17:08.583531 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449783 2577 flags.go:64] FLAG: --kube-api-burst="100" Jul 12 18:17:08.583531 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449786 2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 12 18:17:08.583531 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449789 2577 flags.go:64] FLAG: --kube-api-qps="50" Jul 12 18:17:08.583531 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449792 2577 flags.go:64] FLAG: --kube-reserved="" Jul 12 18:17:08.583531 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449795 2577 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 12 18:17:08.583531 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449800 2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 12 18:17:08.583531 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449803 2577 flags.go:64] FLAG: --kubelet-cgroups="" Jul 12 18:17:08.583531 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449806 2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 12 18:17:08.583531 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449809 2577 flags.go:64] FLAG: --lock-file="" Jul 12 18:17:08.583531 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449812 2577 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 12 18:17:08.584399 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449816 2577 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 12 18:17:08.584399 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449819 2577 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 12 18:17:08.584399 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449824 2577 flags.go:64] FLAG: --log-json-split-stream="false" Jul 12 18:17:08.584399 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449834 2577 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 12 18:17:08.584399 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449837 2577 flags.go:64] FLAG: --log-text-split-stream="false" Jul 12 18:17:08.584399 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449840 2577 flags.go:64] FLAG: --logging-format="text" Jul 12 18:17:08.584399 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449843 2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 12 18:17:08.584399 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449847 2577 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 12 18:17:08.584399 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449849 2577 flags.go:64] FLAG: --manifest-url="" Jul 12 18:17:08.584399 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449852 2577 flags.go:64] FLAG: --manifest-url-header="" Jul 12 18:17:08.584399 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449857 2577 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 12 18:17:08.584399 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449861 2577 flags.go:64] FLAG: --max-open-files="1000000" Jul 12 18:17:08.584399 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449865 2577 flags.go:64] FLAG: --max-pods="110" Jul 12 18:17:08.584399 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449868 2577 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 12 18:17:08.584399 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449871 2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 12 18:17:08.584399 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449874 2577 flags.go:64] FLAG: --memory-manager-policy="None" Jul 12 18:17:08.584399 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449877 2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 12 18:17:08.584399 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449880 2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 12 18:17:08.584399 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449885 2577 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 12 18:17:08.584399 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449888 2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 12 18:17:08.584399 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449896 2577 flags.go:64] FLAG: --node-status-max-images="50" Jul 12 18:17:08.584399 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449899 2577 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 12 18:17:08.584399 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449903 2577 flags.go:64] FLAG: --oom-score-adj="-999" Jul 12 18:17:08.584399 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449906 2577 flags.go:64] FLAG: --pod-cidr="" Jul 12 18:17:08.585489 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449909 2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 12 18:17:08.585489 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449916 2577 flags.go:64] FLAG: --pod-manifest-path="" Jul 12 18:17:08.585489 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449919 2577 flags.go:64] FLAG: --pod-max-pids="-1" Jul 12 18:17:08.585489 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449922 2577 flags.go:64] FLAG: --pods-per-core="0" Jul 12 18:17:08.585489 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449927 2577 flags.go:64] FLAG: --port="10250" Jul 12 18:17:08.585489 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449930 2577 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 12 18:17:08.585489 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449933 2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0d31a5c57010756f9" Jul 12 18:17:08.585489 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449937 2577 flags.go:64] FLAG: --qos-reserved="" Jul 12 18:17:08.585489 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449940 2577 flags.go:64] FLAG: --read-only-port="10255" Jul 12 18:17:08.585489 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449943 2577 flags.go:64] FLAG: --register-node="true" Jul 12 18:17:08.585489 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449946 2577 flags.go:64] FLAG: --register-with-taints="" Jul 12 18:17:08.585489 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449950 2577 flags.go:64] FLAG: --registry-burst="10" Jul 12 18:17:08.585489 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449959 2577 flags.go:64] FLAG: --registry-qps="5" Jul 12 18:17:08.585489 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449963 2577 flags.go:64] FLAG: --reserved-cpus="" Jul 12 18:17:08.585489 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449966 2577 flags.go:64] FLAG: --reserved-memory="" Jul 12 18:17:08.585489 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449970 2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 12 18:17:08.585489 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449973 2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 12 18:17:08.585489 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449976 2577 flags.go:64] FLAG: --rotate-certificates="false" Jul 12 18:17:08.585489 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449979 2577 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 12 18:17:08.585489 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449982 2577 flags.go:64] FLAG: --runonce="false" Jul 12 18:17:08.585489 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449985 2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 12 18:17:08.585489 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449989 2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 12 18:17:08.585489 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449992 2577 flags.go:64] FLAG: --seccomp-default="false" Jul 12 18:17:08.585489 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449995 2577 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 12 18:17:08.585489 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.449998 2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 12 18:17:08.586385 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.450001 2577 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 12 18:17:08.586385 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.450006 2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 12 18:17:08.586385 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.450009 2577 flags.go:64] FLAG: --storage-driver-password="root" Jul 12 18:17:08.586385 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.450012 2577 flags.go:64] FLAG: --storage-driver-secure="false" Jul 12 18:17:08.586385 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.450016 2577 flags.go:64] FLAG: --storage-driver-table="stats" Jul 12 18:17:08.586385 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.450019 2577 flags.go:64] FLAG: --storage-driver-user="root" Jul 12 18:17:08.586385 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.450022 2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 12 18:17:08.586385 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.450025 2577 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 12 18:17:08.586385 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.450028 2577 flags.go:64] FLAG: --system-cgroups="" Jul 12 18:17:08.586385 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.450032 2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 12 18:17:08.586385 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.450037 2577 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 12 18:17:08.586385 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.450042 2577 flags.go:64] FLAG: --tls-cert-file="" Jul 12 18:17:08.586385 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.450045 2577 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 12 18:17:08.586385 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.450052 2577 flags.go:64] FLAG: --tls-min-version="" Jul 12 18:17:08.586385 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.450055 2577 flags.go:64] FLAG: --tls-private-key-file="" Jul 12 18:17:08.586385 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.450058 2577 flags.go:64] FLAG: --topology-manager-policy="none" Jul 12 18:17:08.586385 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.450061 2577 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 12 18:17:08.586385 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.450064 2577 flags.go:64] FLAG: --topology-manager-scope="container" Jul 12 18:17:08.586385 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.450068 2577 flags.go:64] FLAG: --v="2" Jul 12 18:17:08.586385 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.450072 2577 flags.go:64] FLAG: --version="false" Jul 12 18:17:08.586385 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.450086 2577 flags.go:64] FLAG: --vmodule="" Jul 12 18:17:08.586385 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.450090 2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 12 18:17:08.586385 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.450094 2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 12 18:17:08.586385 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450241 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 18:17:08.586385 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450246 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 18:17:08.587390 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450249 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 18:17:08.587390 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450252 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 18:17:08.587390 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450255 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 18:17:08.587390 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450257 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 18:17:08.587390 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450260 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 18:17:08.587390 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450263 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 18:17:08.587390 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450265 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 18:17:08.587390 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450268 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 18:17:08.587390 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450272 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 18:17:08.587390 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450275 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 18:17:08.587390 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450278 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 18:17:08.587390 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450280 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 18:17:08.587390 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450283 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 18:17:08.587390 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450286 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 18:17:08.587390 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450288 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 18:17:08.587390 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450291 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 18:17:08.587390 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450294 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 18:17:08.587390 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450297 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 18:17:08.587390 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450301 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 18:17:08.587390 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450306 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 18:17:08.588118 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450310 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 18:17:08.588118 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450313 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 18:17:08.588118 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450316 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 18:17:08.588118 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450319 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 18:17:08.588118 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450323 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 18:17:08.588118 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450325 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 18:17:08.588118 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450328 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 18:17:08.588118 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450331 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 18:17:08.588118 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450335 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 18:17:08.588118 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450338 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 18:17:08.588118 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450341 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 18:17:08.588118 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450343 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 18:17:08.588118 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450346 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 18:17:08.588118 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450348 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 18:17:08.588118 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450351 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 18:17:08.588118 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450354 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 18:17:08.588118 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450356 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 18:17:08.588118 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450359 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 18:17:08.588118 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450362 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 18:17:08.588118 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450365 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 18:17:08.589123 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450369 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 18:17:08.589123 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450372 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 18:17:08.589123 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450374 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 18:17:08.589123 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450377 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 18:17:08.589123 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450380 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 18:17:08.589123 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450383 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 18:17:08.589123 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450386 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 18:17:08.589123 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450389 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 18:17:08.589123 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450393 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 18:17:08.589123 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450397 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 18:17:08.589123 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450402 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 18:17:08.589123 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450405 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 18:17:08.589123 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450408 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 18:17:08.589123 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450410 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 18:17:08.589123 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450413 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 18:17:08.589123 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450416 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 18:17:08.589123 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450419 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 18:17:08.589123 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450422 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 18:17:08.589123 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450424 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 18:17:08.590336 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450427 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 18:17:08.590336 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450430 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 18:17:08.590336 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450433 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 18:17:08.590336 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450436 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 18:17:08.590336 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450439 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 18:17:08.590336 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450441 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 18:17:08.590336 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450444 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 18:17:08.590336 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450446 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 18:17:08.590336 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450449 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 18:17:08.590336 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450452 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 18:17:08.590336 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450455 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 18:17:08.590336 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450457 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 18:17:08.590336 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450460 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 18:17:08.590336 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450465 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 18:17:08.590336 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450468 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 18:17:08.590336 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450471 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 18:17:08.590336 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450474 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 18:17:08.590336 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450476 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 18:17:08.590336 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450487 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 18:17:08.590336 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450491 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 18:17:08.683293 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450494 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 18:17:08.683293 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450497 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 18:17:08.683293 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450499 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 18:17:08.683293 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450504 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 18:17:08.683293 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450507 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 18:17:08.683293 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450510 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 12 18:17:08.683293 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450514 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 18:17:08.683293 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450518 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 18:17:08.683293 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450521 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 18:17:08.683293 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450525 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 18:17:08.683293 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450528 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 18:17:08.683293 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450530 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 18:17:08.683293 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450533 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 18:17:08.683293 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450537 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 18:17:08.683293 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450540 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 18:17:08.683293 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450544 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 18:17:08.683293 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450547 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 18:17:08.683293 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.450550 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 18:17:08.684166 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.451527 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 12 18:17:08.684166 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.458883 2577 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 12 18:17:08.684166 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.458902 2577 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 12 18:17:08.684166 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.458920 2577 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 12 18:17:08.684166 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.458930 2577 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 12 18:17:08.684166 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459012 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 18:17:08.684166 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459018 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 18:17:08.684166 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459022 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 18:17:08.684166 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459026 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 18:17:08.684166 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459029 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 18:17:08.684166 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459033 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 18:17:08.684166 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459037 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 18:17:08.684166 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459040 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 18:17:08.684166 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459043 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 18:17:08.685086 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459046 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 18:17:08.685086 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459049 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 18:17:08.685086 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459051 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 18:17:08.685086 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459054 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 18:17:08.685086 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459057 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 18:17:08.685086 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459060 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 18:17:08.685086 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459063 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 18:17:08.685086 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459066 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 18:17:08.685086 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459068 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 18:17:08.685086 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459072 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 18:17:08.685086 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459074 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 18:17:08.685086 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459077 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 18:17:08.685086 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459080 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 18:17:08.685086 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459083 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 18:17:08.685086 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459086 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 18:17:08.685086 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459090 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 18:17:08.685086 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459093 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 18:17:08.685086 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459099 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 18:17:08.685086 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459103 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 18:17:08.690527 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459107 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 18:17:08.690527 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459111 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 12 18:17:08.690527 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459114 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 18:17:08.690527 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459118 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 18:17:08.690527 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459120 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 18:17:08.690527 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459124 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 18:17:08.690527 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459127 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 18:17:08.690527 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459130 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 18:17:08.690527 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459133 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 18:17:08.690527 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459136 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 18:17:08.690527 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459139 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 18:17:08.690527 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459143 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 18:17:08.690527 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459155 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 18:17:08.690527 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459159 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 18:17:08.690527 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459163 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 18:17:08.690527 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459166 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 18:17:08.690527 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459169 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 18:17:08.690527 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459172 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 18:17:08.690527 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459176 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 18:17:08.690527 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459179 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 18:17:08.691450 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459181 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 18:17:08.691450 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459184 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 18:17:08.691450 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459187 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 18:17:08.691450 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459190 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 18:17:08.691450 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459193 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 18:17:08.691450 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459196 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 18:17:08.691450 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459198 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 18:17:08.691450 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459201 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 18:17:08.691450 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459204 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 18:17:08.691450 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459207 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 18:17:08.691450 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459211 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 18:17:08.691450 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459214 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 18:17:08.691450 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459217 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 18:17:08.691450 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459219 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 18:17:08.691450 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459222 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 18:17:08.691450 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459225 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 18:17:08.691450 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459228 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 18:17:08.691450 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459231 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 18:17:08.691450 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459234 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 18:17:08.691450 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459237 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 18:17:08.692363 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459240 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 18:17:08.692363 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459243 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 18:17:08.692363 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459246 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 18:17:08.692363 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459249 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 18:17:08.692363 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459252 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 18:17:08.692363 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459255 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 18:17:08.692363 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459258 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 18:17:08.692363 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459261 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 18:17:08.692363 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459263 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 18:17:08.692363 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459266 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 18:17:08.692363 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459269 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 18:17:08.692363 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459272 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 18:17:08.692363 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459274 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 18:17:08.692363 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459277 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 18:17:08.692363 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459280 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 18:17:08.692363 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459282 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 18:17:08.692363 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459285 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 18:17:08.692363 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459288 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 18:17:08.692363 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459291 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 18:17:08.692363 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459294 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 18:17:08.695073 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459296 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 18:17:08.695073 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459299 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 18:17:08.695073 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459303 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 18:17:08.695073 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459311 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 18:17:08.695073 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459314 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 18:17:08.695073 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459317 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 18:17:08.695073 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459320 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 18:17:08.695073 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459323 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 18:17:08.695073 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459326 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 18:17:08.695073 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459328 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 18:17:08.695073 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459331 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 18:17:08.695073 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.459337 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 12 18:17:08.695073 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459482 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 18:17:08.695073 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459487 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 18:17:08.695073 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459490 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 18:17:08.695985 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459494 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 18:17:08.695985 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459497 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 18:17:08.695985 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459500 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 18:17:08.695985 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459503 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 18:17:08.695985 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459507 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 18:17:08.695985 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459509 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 18:17:08.695985 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459512 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 18:17:08.695985 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459515 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 18:17:08.695985 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459518 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 18:17:08.695985 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459521 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 18:17:08.695985 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459523 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 18:17:08.695985 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459526 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 18:17:08.695985 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459529 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 18:17:08.695985 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459533 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 18:17:08.695985 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459535 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 18:17:08.695985 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459538 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 18:17:08.695985 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459540 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 18:17:08.695985 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459543 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 18:17:08.695985 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459546 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 18:17:08.695985 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459551 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 18:17:08.699559 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459555 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 18:17:08.699559 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459559 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 18:17:08.699559 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459562 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 18:17:08.699559 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459565 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 18:17:08.699559 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459567 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 18:17:08.699559 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459570 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 18:17:08.699559 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459573 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 18:17:08.699559 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459576 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 18:17:08.699559 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459578 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 18:17:08.699559 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459581 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 18:17:08.699559 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459584 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 18:17:08.699559 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459587 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 18:17:08.699559 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459589 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 18:17:08.699559 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459592 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 18:17:08.699559 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459595 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 18:17:08.699559 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459597 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 18:17:08.699559 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459600 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 18:17:08.699559 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459605 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 18:17:08.699559 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459608 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 18:17:08.700475 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459612 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 18:17:08.700475 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459617 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 18:17:08.700475 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459620 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 18:17:08.700475 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459622 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 18:17:08.700475 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459625 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 18:17:08.700475 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459628 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 18:17:08.700475 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459631 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 18:17:08.700475 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459634 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 18:17:08.700475 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459636 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 18:17:08.700475 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459639 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 18:17:08.700475 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459642 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 18:17:08.700475 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459644 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 18:17:08.700475 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459648 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 18:17:08.700475 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459651 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 18:17:08.700475 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459656 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 18:17:08.700475 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459659 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 18:17:08.700475 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459662 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 18:17:08.700475 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459665 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 18:17:08.700475 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459667 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 18:17:08.812607 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459670 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 18:17:08.812607 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459673 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 18:17:08.812607 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459676 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 18:17:08.812607 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459678 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 18:17:08.812607 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459681 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 18:17:08.812607 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459684 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 18:17:08.812607 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459686 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 18:17:08.812607 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459689 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 18:17:08.812607 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459692 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 18:17:08.812607 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459694 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 18:17:08.812607 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459697 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 18:17:08.812607 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459721 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 18:17:08.812607 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459725 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 18:17:08.812607 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459728 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 18:17:08.812607 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459731 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 18:17:08.812607 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459734 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 18:17:08.812607 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459737 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 18:17:08.812607 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459740 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 18:17:08.812607 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459742 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 18:17:08.812607 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459745 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 18:17:08.813541 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459748 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 18:17:08.813541 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459751 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 18:17:08.813541 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459753 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 18:17:08.813541 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459756 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 18:17:08.813541 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459759 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 18:17:08.813541 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459763 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 18:17:08.813541 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459772 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 12 18:17:08.813541 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459777 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 18:17:08.813541 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459781 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 18:17:08.813541 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459784 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 18:17:08.813541 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459787 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 18:17:08.813541 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459789 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 18:17:08.813541 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459793 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 18:17:08.813541 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459797 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 18:17:08.813541 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459800 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 18:17:08.813541 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459803 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 18:17:08.813541 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459806 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 18:17:08.813541 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:08.459809 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 18:17:08.822619 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.459815 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 12 18:17:08.822619 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.459946 2577 server.go:962] "Client rotation is on, will bootstrap in background" Jul 12 18:17:08.822619 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.463016 2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 12 18:17:08.822619 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.464474 2577 server.go:1019] "Starting client certificate rotation" Jul 12 18:17:08.822619 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.464619 2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 12 18:17:08.822619 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.464664 2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 12 18:17:08.822619 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.467810 2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 12 18:17:08.822619 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.469336 2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 12 18:17:08.822619 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.493529 2577 log.go:25] "Validated CRI v1 runtime API" Jul 12 18:17:08.822619 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.500959 2577 log.go:25] "Validated CRI v1 image API" Jul 12 18:17:08.822619 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.501837 2577 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 12 18:17:08.822619 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.509942 2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 12 18:17:08.823511 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.510072 2577 fs.go:135] Filesystem UUIDs: map[030d71f5-a65a-4fb3-8432-4d0e53eed8dd:/dev/nvme0n1p3 1179f875-7897-44af-af7d-38bab7e4c3d8:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jul 12 18:17:08.823511 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.510092 2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 12 18:17:08.933989 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.516896 2577 manager.go:216] Machine: {Timestamp:2026-07-12 18:17:08.515416815 +0000 UTC m=+1.642321613 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3113015 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2bf7b5da4f1d1557557899bfcc43da SystemUUID:ec2bf7b5-da4f-1d15-5755-7899bfcc43da BootID:c7889f24-a747-4ff8-a4ce-dd45c06d92d2 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:52:8a:3b:3d:ed Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:52:8a:3b:3d:ed Speed:0 Mtu:9001} {Name:ovs-system MacAddress:92:d5:b5:44:a6:bc Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 12 18:17:08.933989 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.517012 2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 12 18:17:08.933989 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.517099 2577 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 12 18:17:08.933989 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.520115 2577 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 12 18:17:08.933989 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.520141 2577 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-133.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 12 18:17:08.933989 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.521008 2577 topology_manager.go:138] "Creating topology manager with none policy" Jul 12 18:17:08.933989 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.521019 2577 container_manager_linux.go:309] "Creating device plugin manager" Jul 12 18:17:08.933989 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.521394 2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 12 18:17:08.933989 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.522317 2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 12 18:17:08.933989 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.523641 2577 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 12 18:17:08.933989 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.526932 2577 state_mem.go:36] "Initialized new in-memory state store" Jul 12 18:17:08.933989 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.527063 2577 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 12 18:17:08.933989 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.527096 2577 kubelet.go:486] "Attempting to sync node with API server" Jul 12 18:17:08.933989 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.527109 2577 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 12 18:17:08.933989 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.527124 2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 12 18:17:08.933989 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.527134 2577 kubelet.go:398] "Adding apiserver pod source" Jul 12 18:17:08.933989 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.527144 2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 12 18:17:08.935211 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.527480 2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-z7cwq" Jul 12 18:17:08.935211 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.527551 2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 12 18:17:08.935211 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.527566 2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 12 18:17:08.935211 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.530166 2577 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 12 18:17:08.935211 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.532010 2577 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 12 18:17:08.935211 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.532040 2577 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 12 18:17:08.935211 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.532208 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 12 18:17:08.935211 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.532222 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 12 18:17:08.935211 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.532229 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 12 18:17:08.935211 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.532234 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 12 18:17:08.935211 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.532242 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 12 18:17:08.935211 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.532248 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 12 18:17:08.935211 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.532808 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 12 18:17:08.935211 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.532823 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 12 18:17:08.935211 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.532830 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 12 18:17:08.935211 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.532837 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 12 18:17:08.935211 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.532846 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 12 18:17:08.935211 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.532854 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 12 18:17:08.935211 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.532879 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 12 18:17:08.935211 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.532884 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 12 18:17:08.935211 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.533613 2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-z7cwq" Jul 12 18:17:08.935211 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.536465 2577 server.go:1268] "Started kubelet" Jul 12 18:17:08.935211 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.536596 2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 12 18:17:08.935211 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.536549 2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 12 18:17:08.935211 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.536681 2577 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 12 18:17:08.935211 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.540305 2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-133.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 12 18:17:08.935211 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.540445 2577 server.go:310] "Adding debug handlers to kubelet server" Jul 12 18:17:08.935211 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.540746 2577 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 12 18:17:08.935211 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:08.540972 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 12 18:17:08.935211 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:08.548832 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-133.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 12 18:17:08.935211 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:08.550018 2577 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 12 18:17:08.935211 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.552950 2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 12 18:17:08.935211 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.552980 2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 12 18:17:08.935211 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:08.553086 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-133.ec2.internal\" not found" Jul 12 18:17:08.935211 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.553681 2577 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 12 18:17:08.935211 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.553700 2577 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 12 18:17:08.935211 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.553725 2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 12 18:17:08.935211 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.553804 2577 reconstruct.go:97] "Volume reconstruction finished" Jul 12 18:17:08.935211 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.553815 2577 reconciler.go:29] "Reconciler: start to sync state" Jul 12 18:17:08.935211 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.554382 2577 factory.go:153] Registering CRI-O factory Jul 12 18:17:08.935211 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.554408 2577 factory.go:223] Registration of the crio container factory successfully Jul 12 18:17:08.935211 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.554505 2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 12 18:17:08.935211 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.554515 2577 factory.go:55] Registering systemd factory Jul 12 18:17:08.935211 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.554524 2577 factory.go:223] Registration of the systemd container factory successfully Jul 12 18:17:08.935211 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.554546 2577 factory.go:103] Registering Raw factory Jul 12 18:17:08.935211 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.554563 2577 manager.go:1196] Started watching for new ooms in manager Jul 12 18:17:08.935211 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.555249 2577 manager.go:349] Starting recovery of all containers Jul 12 18:17:08.937890 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.556051 2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 18:17:08.937890 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:08.558735 2577 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-141-133.ec2.internal\" not found" node="ip-10-0-141-133.ec2.internal" Jul 12 18:17:08.937890 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.566613 2577 manager.go:354] Recovery completed Jul 12 18:17:08.937890 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.571468 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 18:17:08.937890 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.573782 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasSufficientMemory" Jul 12 18:17:08.937890 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.573809 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 18:17:08.937890 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.573821 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasSufficientPID" Jul 12 18:17:08.937890 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.574414 2577 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 12 18:17:08.937890 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.574422 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 18:17:08.937890 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.574423 2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 12 18:17:08.937890 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.574471 2577 state_mem.go:36] "Initialized new in-memory state store" Jul 12 18:17:08.937890 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.575371 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasSufficientMemory" Jul 12 18:17:08.937890 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.575402 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 18:17:08.937890 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.575414 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasSufficientPID" Jul 12 18:17:08.937890 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.575591 2577 policy_none.go:49] "None policy: Start" Jul 12 18:17:08.937890 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.575605 2577 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 12 18:17:08.937890 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.575616 2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 12 18:17:08.937890 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.584262 2577 policy_none.go:47] "Start" Jul 12 18:17:08.937890 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.587956 2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 12 18:17:08.937890 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:08.653673 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-133.ec2.internal\" not found" Jul 12 18:17:08.937890 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:08.754003 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-133.ec2.internal\" not found" Jul 12 18:17:08.937890 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:08.854420 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-133.ec2.internal\" not found" Jul 12 18:17:08.954951 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:08.954925 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-133.ec2.internal\" not found" Jul 12 18:17:08.978643 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.978621 2577 manager.go:337] "Starting Device Plugin manager" Jul 12 18:17:08.978756 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:08.978671 2577 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 12 18:17:08.978756 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.978682 2577 server.go:87] "Starting device plugin registration server" Jul 12 18:17:08.978988 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.978974 2577 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 12 18:17:08.979066 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.978993 2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 12 18:17:08.979139 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.979122 2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 12 18:17:08.979239 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.979230 2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 12 18:17:08.979317 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:08.979241 2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 12 18:17:08.979828 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:08.979805 2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 12 18:17:08.979901 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:08.979878 2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-133.ec2.internal\" not found" Jul 12 18:17:09.069278 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.069246 2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 12 18:17:09.070325 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.069289 2577 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 12 18:17:09.070325 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.069315 2577 kubelet.go:2448] "Starting kubelet main sync loop" Jul 12 18:17:09.070325 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:09.069354 2577 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 12 18:17:09.071908 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.071891 2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 18:17:09.079350 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.079334 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 18:17:09.080246 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.080230 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasSufficientMemory" Jul 12 18:17:09.080352 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.080266 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 18:17:09.080352 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.080282 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasSufficientPID" Jul 12 18:17:09.080352 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.080313 2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-133.ec2.internal" Jul 12 18:17:09.090522 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.090497 2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-133.ec2.internal" Jul 12 18:17:09.090614 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:09.090522 2577 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-133.ec2.internal\": node \"ip-10-0-141-133.ec2.internal\" not found" Jul 12 18:17:09.104592 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:09.104572 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-133.ec2.internal\" not found" Jul 12 18:17:09.169661 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.169598 2577 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-133.ec2.internal"] Jul 12 18:17:09.169787 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.169727 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 18:17:09.172320 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.172304 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasSufficientMemory" Jul 12 18:17:09.172400 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.172333 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 18:17:09.172400 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.172344 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasSufficientPID" Jul 12 18:17:09.174738 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.174726 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 18:17:09.174868 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.174842 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal" Jul 12 18:17:09.174916 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.174878 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 18:17:09.178193 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.178176 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasSufficientMemory" Jul 12 18:17:09.178256 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.178195 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasSufficientMemory" Jul 12 18:17:09.178256 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.178205 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 18:17:09.178256 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.178217 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasSufficientPID" Jul 12 18:17:09.178256 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.178220 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 18:17:09.178256 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.178231 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasSufficientPID" Jul 12 18:17:09.179359 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.179345 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-133.ec2.internal" Jul 12 18:17:09.179411 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.179368 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 18:17:09.180116 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.180101 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasSufficientMemory" Jul 12 18:17:09.180173 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.180135 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 18:17:09.180173 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.180151 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasSufficientPID" Jul 12 18:17:09.205771 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:09.205751 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-133.ec2.internal\" not found" Jul 12 18:17:09.210538 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:09.210518 2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-133.ec2.internal\" not found" node="ip-10-0-141-133.ec2.internal" Jul 12 18:17:09.217091 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:09.217072 2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-133.ec2.internal\" not found" node="ip-10-0-141-133.ec2.internal" Jul 12 18:17:09.258529 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.258503 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7d5059b2ae6f7bac7125eda1da58ccd7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal\" (UID: \"7d5059b2ae6f7bac7125eda1da58ccd7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal" Jul 12 18:17:09.258615 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.258535 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7d5059b2ae6f7bac7125eda1da58ccd7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal\" (UID: \"7d5059b2ae6f7bac7125eda1da58ccd7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal" Jul 12 18:17:09.258615 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.258558 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/98d2a84da404683679bff0f03b9d595c-config\") pod \"kube-apiserver-proxy-ip-10-0-141-133.ec2.internal\" (UID: \"98d2a84da404683679bff0f03b9d595c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-133.ec2.internal" Jul 12 18:17:09.306603 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:09.306574 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-133.ec2.internal\" not found" Jul 12 18:17:09.359239 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.359207 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7d5059b2ae6f7bac7125eda1da58ccd7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal\" (UID: \"7d5059b2ae6f7bac7125eda1da58ccd7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal" Jul 12 18:17:09.359298 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.359245 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7d5059b2ae6f7bac7125eda1da58ccd7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal\" (UID: \"7d5059b2ae6f7bac7125eda1da58ccd7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal" Jul 12 18:17:09.359298 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.359265 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/98d2a84da404683679bff0f03b9d595c-config\") pod \"kube-apiserver-proxy-ip-10-0-141-133.ec2.internal\" (UID: \"98d2a84da404683679bff0f03b9d595c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-133.ec2.internal" Jul 12 18:17:09.359361 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.359312 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7d5059b2ae6f7bac7125eda1da58ccd7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal\" (UID: \"7d5059b2ae6f7bac7125eda1da58ccd7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal" Jul 12 18:17:09.359361 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.359344 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7d5059b2ae6f7bac7125eda1da58ccd7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal\" (UID: \"7d5059b2ae6f7bac7125eda1da58ccd7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal" Jul 12 18:17:09.359361 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.359317 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/98d2a84da404683679bff0f03b9d595c-config\") pod \"kube-apiserver-proxy-ip-10-0-141-133.ec2.internal\" (UID: \"98d2a84da404683679bff0f03b9d595c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-133.ec2.internal" Jul 12 18:17:09.410206 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:09.410167 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-133.ec2.internal\" not found" Jul 12 18:17:09.421700 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.421681 2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 18:17:09.455457 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.455425 2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal" Jul 12 18:17:09.463840 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.463819 2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 12 18:17:09.463998 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:09.463974 2577 kubelet.go:3250] "Failed creating a mirror pod" err="Post \"https://a1922e4d2b4ff49a595f639d86a4ba3a-6243e6386330e69f.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/openshift-machine-config-operator/pods\": read tcp 10.0.141.133:34732->52.87.91.13:6443: use of closed network connection" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal" Jul 12 18:17:09.464121 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.464002 2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-133.ec2.internal" Jul 12 18:17:09.464121 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.464000 2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 12 18:17:09.464121 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.464000 2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 12 18:17:09.464121 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.464022 2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 12 18:17:09.484279 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.484258 2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 12 18:17:09.512201 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.512174 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal" Jul 12 18:17:09.520583 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.520562 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-133.ec2.internal" Jul 12 18:17:09.527774 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.527756 2577 apiserver.go:52] "Watching apiserver" Jul 12 18:17:09.536439 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.536420 2577 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 12 18:17:09.536822 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.536803 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-141-133.ec2.internal","openshift-image-registry/node-ca-khqqs","openshift-multus/multus-additional-cni-plugins-29hd5","openshift-multus/multus-t9jvk","openshift-ovn-kubernetes/ovnkube-node-spzqd","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pbwgr","openshift-cluster-node-tuning-operator/tuned-j6pq9","openshift-dns/node-resolver-xwcbp","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal","openshift-multus/network-metrics-daemon-r66xp","openshift-network-diagnostics/network-check-target-lfnkm","openshift-network-operator/iptables-alerter-7f47s","kube-system/global-pull-secret-syncer-2598z","kube-system/konnectivity-agent-hqhkm"] Jul 12 18:17:09.537698 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.537658 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-11 18:12:09 +0000 UTC" deadline="2028-03-18 04:25:26.396466068 +0000 UTC" Jul 12 18:17:09.537698 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.537700 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14746h8m16.858769789s" Jul 12 18:17:09.538181 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.538166 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-khqqs" Jul 12 18:17:09.539484 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.539464 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-29hd5" Jul 12 18:17:09.540700 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.540676 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-t9jvk" Jul 12 18:17:09.540815 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.540724 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 12 18:17:09.540815 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.540799 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-n9rmc\"" Jul 12 18:17:09.540960 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.540941 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 12 18:17:09.541018 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.540949 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 12 18:17:09.541917 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.541894 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 12 18:17:09.542013 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.541927 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 12 18:17:09.542013 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.542000 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.542394 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.542372 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 12 18:17:09.542498 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.542375 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-l7frw\"" Jul 12 18:17:09.542564 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.542541 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 12 18:17:09.542622 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.542598 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 12 18:17:09.542923 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.542908 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-bk9j6\"" Jul 12 18:17:09.543337 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.543319 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pbwgr" Jul 12 18:17:09.543566 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.543531 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 12 18:17:09.544550 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.544529 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 12 18:17:09.544884 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.544595 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-j6pq9" Jul 12 18:17:09.544884 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.544520 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 12 18:17:09.544884 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.544671 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-mv9gc\"" Jul 12 18:17:09.544884 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.544744 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 12 18:17:09.544884 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.544750 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 12 18:17:09.545813 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.545790 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xwcbp" Jul 12 18:17:09.546040 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.545997 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 12 18:17:09.546129 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.546038 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 12 18:17:09.546191 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.546162 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 12 18:17:09.546244 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.546235 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-rcsxx\"" Jul 12 18:17:09.546297 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.546245 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 12 18:17:09.546349 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.546311 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 12 18:17:09.548578 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.547917 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 12 18:17:09.549802 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.549781 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-6ckg2\"" Jul 12 18:17:09.549983 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.549963 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 12 18:17:09.550064 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.550049 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-tjnrl\"" Jul 12 18:17:09.550275 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.550262 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 12 18:17:09.550464 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.550441 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 12 18:17:09.550623 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.550606 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r66xp" Jul 12 18:17:09.550762 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:09.550740 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r66xp" podUID="7d69e388-ddc9-4dd8-b5dc-a81a59dc1698" Jul 12 18:17:09.551610 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.551586 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lfnkm" Jul 12 18:17:09.551730 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:09.551688 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lfnkm" podUID="4d1fc6eb-09af-4f96-8ec8-b193da43354c" Jul 12 18:17:09.552967 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.552947 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7f47s" Jul 12 18:17:09.553163 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.553139 2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 12 18:17:09.555063 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.555044 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2598z" Jul 12 18:17:09.555139 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:09.555120 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2598z" podUID="9f8419c6-07be-4876-9762-d98cd772054b" Jul 12 18:17:09.555724 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.555703 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-7zq8v\"" Jul 12 18:17:09.555784 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.555751 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 12 18:17:09.555853 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.555836 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 12 18:17:09.555853 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.555844 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 12 18:17:09.556518 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.556503 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hqhkm" Jul 12 18:17:09.558865 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.558846 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 12 18:17:09.558964 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.558919 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 12 18:17:09.558964 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.558920 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-kc5mg\"" Jul 12 18:17:09.559486 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.559467 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpzmv\" (UniqueName: \"kubernetes.io/projected/4d1fc6eb-09af-4f96-8ec8-b193da43354c-kube-api-access-tpzmv\") pod \"network-check-target-lfnkm\" (UID: \"4d1fc6eb-09af-4f96-8ec8-b193da43354c\") " pod="openshift-network-diagnostics/network-check-target-lfnkm" Jul 12 18:17:09.559528 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.559499 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cef566f5-8ee3-4192-bc98-fbcbbb9f29b8-iptables-alerter-script\") pod \"iptables-alerter-7f47s\" (UID: \"cef566f5-8ee3-4192-bc98-fbcbbb9f29b8\") " pod="openshift-network-operator/iptables-alerter-7f47s" Jul 12 18:17:09.559528 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.559517 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a58d0c9c-b6c7-4962-90d1-34d10eb4e97c-etc-systemd\") pod \"tuned-j6pq9\" (UID: \"a58d0c9c-b6c7-4962-90d1-34d10eb4e97c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6pq9" Jul 12 18:17:09.559617 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.559535 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1a19952f-4732-46da-98ee-05c476b44908-system-cni-dir\") pod \"multus-additional-cni-plugins-29hd5\" (UID: \"1a19952f-4732-46da-98ee-05c476b44908\") " pod="openshift-multus/multus-additional-cni-plugins-29hd5" Jul 12 18:17:09.559617 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.559560 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1a19952f-4732-46da-98ee-05c476b44908-cni-binary-copy\") pod \"multus-additional-cni-plugins-29hd5\" (UID: \"1a19952f-4732-46da-98ee-05c476b44908\") " pod="openshift-multus/multus-additional-cni-plugins-29hd5" Jul 12 18:17:09.559751 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.559654 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ed115460-b220-476d-b9d5-65f95ec3d059-run-systemd\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.559751 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.559691 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1a19952f-4732-46da-98ee-05c476b44908-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-29hd5\" (UID: \"1a19952f-4732-46da-98ee-05c476b44908\") " pod="openshift-multus/multus-additional-cni-plugins-29hd5" Jul 12 18:17:09.559751 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.559734 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a05fea52-1297-4bd7-82a2-d863030b675b-host-run-netns\") pod \"multus-t9jvk\" (UID: \"a05fea52-1297-4bd7-82a2-d863030b675b\") " pod="openshift-multus/multus-t9jvk" Jul 12 18:17:09.559896 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.559766 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a05fea52-1297-4bd7-82a2-d863030b675b-host-var-lib-cni-bin\") pod \"multus-t9jvk\" (UID: \"a05fea52-1297-4bd7-82a2-d863030b675b\") " pod="openshift-multus/multus-t9jvk" Jul 12 18:17:09.559896 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.559789 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a05fea52-1297-4bd7-82a2-d863030b675b-host-run-multus-certs\") pod \"multus-t9jvk\" (UID: \"a05fea52-1297-4bd7-82a2-d863030b675b\") " pod="openshift-multus/multus-t9jvk" Jul 12 18:17:09.559896 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.559807 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a58d0c9c-b6c7-4962-90d1-34d10eb4e97c-etc-tuned\") pod \"tuned-j6pq9\" (UID: \"a58d0c9c-b6c7-4962-90d1-34d10eb4e97c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6pq9" Jul 12 18:17:09.559896 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.559836 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rc9w4\" (UniqueName: \"kubernetes.io/projected/a58d0c9c-b6c7-4962-90d1-34d10eb4e97c-kube-api-access-rc9w4\") pod \"tuned-j6pq9\" (UID: \"a58d0c9c-b6c7-4962-90d1-34d10eb4e97c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6pq9" Jul 12 18:17:09.559896 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.559862 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a05fea52-1297-4bd7-82a2-d863030b675b-hostroot\") pod \"multus-t9jvk\" (UID: \"a05fea52-1297-4bd7-82a2-d863030b675b\") " pod="openshift-multus/multus-t9jvk" Jul 12 18:17:09.559896 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.559891 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a58d0c9c-b6c7-4962-90d1-34d10eb4e97c-var-lib-kubelet\") pod \"tuned-j6pq9\" (UID: \"a58d0c9c-b6c7-4962-90d1-34d10eb4e97c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6pq9" Jul 12 18:17:09.560076 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.559918 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9f8419c6-07be-4876-9762-d98cd772054b-kubelet-config\") pod \"global-pull-secret-syncer-2598z\" (UID: \"9f8419c6-07be-4876-9762-d98cd772054b\") " pod="kube-system/global-pull-secret-syncer-2598z" Jul 12 18:17:09.560076 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.559938 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9f8419c6-07be-4876-9762-d98cd772054b-original-pull-secret\") pod \"global-pull-secret-syncer-2598z\" (UID: \"9f8419c6-07be-4876-9762-d98cd772054b\") " pod="kube-system/global-pull-secret-syncer-2598z" Jul 12 18:17:09.560076 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.559962 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1a19952f-4732-46da-98ee-05c476b44908-cnibin\") pod \"multus-additional-cni-plugins-29hd5\" (UID: \"1a19952f-4732-46da-98ee-05c476b44908\") " pod="openshift-multus/multus-additional-cni-plugins-29hd5" Jul 12 18:17:09.560076 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.559989 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ed115460-b220-476d-b9d5-65f95ec3d059-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.560076 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.560009 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ed115460-b220-476d-b9d5-65f95ec3d059-ovnkube-config\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.560076 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.560032 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ed115460-b220-476d-b9d5-65f95ec3d059-env-overrides\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.560076 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.560054 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ed115460-b220-476d-b9d5-65f95ec3d059-host-cni-netd\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.560076 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.560074 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a05fea52-1297-4bd7-82a2-d863030b675b-multus-conf-dir\") pod \"multus-t9jvk\" (UID: \"a05fea52-1297-4bd7-82a2-d863030b675b\") " pod="openshift-multus/multus-t9jvk" Jul 12 18:17:09.560310 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.560106 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a58d0c9c-b6c7-4962-90d1-34d10eb4e97c-etc-sysconfig\") pod \"tuned-j6pq9\" (UID: \"a58d0c9c-b6c7-4962-90d1-34d10eb4e97c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6pq9" Jul 12 18:17:09.560310 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.560126 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1a19952f-4732-46da-98ee-05c476b44908-os-release\") pod \"multus-additional-cni-plugins-29hd5\" (UID: \"1a19952f-4732-46da-98ee-05c476b44908\") " pod="openshift-multus/multus-additional-cni-plugins-29hd5" Jul 12 18:17:09.560310 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.560142 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/aaefba09-f06b-4e39-bc1b-a1b70455dc45-device-dir\") pod \"aws-ebs-csi-driver-node-pbwgr\" (UID: \"aaefba09-f06b-4e39-bc1b-a1b70455dc45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pbwgr" Jul 12 18:17:09.560310 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.560158 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd8ab0de-04ec-4aa8-8d33-87aa4bbef597-tmp-dir\") pod \"node-resolver-xwcbp\" (UID: \"cd8ab0de-04ec-4aa8-8d33-87aa4bbef597\") " pod="openshift-dns/node-resolver-xwcbp" Jul 12 18:17:09.560310 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.560185 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a58d0c9c-b6c7-4962-90d1-34d10eb4e97c-etc-modprobe-d\") pod \"tuned-j6pq9\" (UID: \"a58d0c9c-b6c7-4962-90d1-34d10eb4e97c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6pq9" Jul 12 18:17:09.560310 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.560202 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a58d0c9c-b6c7-4962-90d1-34d10eb4e97c-lib-modules\") pod \"tuned-j6pq9\" (UID: \"a58d0c9c-b6c7-4962-90d1-34d10eb4e97c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6pq9" Jul 12 18:17:09.560310 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.560217 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a58d0c9c-b6c7-4962-90d1-34d10eb4e97c-tmp\") pod \"tuned-j6pq9\" (UID: \"a58d0c9c-b6c7-4962-90d1-34d10eb4e97c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6pq9" Jul 12 18:17:09.560310 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.560231 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9f8419c6-07be-4876-9762-d98cd772054b-dbus\") pod \"global-pull-secret-syncer-2598z\" (UID: \"9f8419c6-07be-4876-9762-d98cd772054b\") " pod="kube-system/global-pull-secret-syncer-2598z" Jul 12 18:17:09.560310 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.560244 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1b501a87-6ac0-4f0b-9072-aeecd64d5bc0-host\") pod \"node-ca-khqqs\" (UID: \"1b501a87-6ac0-4f0b-9072-aeecd64d5bc0\") " pod="openshift-image-registry/node-ca-khqqs" Jul 12 18:17:09.560310 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.560268 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ed115460-b220-476d-b9d5-65f95ec3d059-host-kubelet\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.560310 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.560285 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1a19952f-4732-46da-98ee-05c476b44908-tuning-conf-dir\") pod \"multus-additional-cni-plugins-29hd5\" (UID: \"1a19952f-4732-46da-98ee-05c476b44908\") " pod="openshift-multus/multus-additional-cni-plugins-29hd5" Jul 12 18:17:09.560310 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.560301 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a05fea52-1297-4bd7-82a2-d863030b675b-multus-daemon-config\") pod \"multus-t9jvk\" (UID: \"a05fea52-1297-4bd7-82a2-d863030b675b\") " pod="openshift-multus/multus-t9jvk" Jul 12 18:17:09.560762 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.560314 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a58d0c9c-b6c7-4962-90d1-34d10eb4e97c-run\") pod \"tuned-j6pq9\" (UID: \"a58d0c9c-b6c7-4962-90d1-34d10eb4e97c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6pq9" Jul 12 18:17:09.560762 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.560327 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a58d0c9c-b6c7-4962-90d1-34d10eb4e97c-host\") pod \"tuned-j6pq9\" (UID: \"a58d0c9c-b6c7-4962-90d1-34d10eb4e97c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6pq9" Jul 12 18:17:09.560762 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.560371 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1a19952f-4732-46da-98ee-05c476b44908-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-29hd5\" (UID: \"1a19952f-4732-46da-98ee-05c476b44908\") " pod="openshift-multus/multus-additional-cni-plugins-29hd5" Jul 12 18:17:09.560762 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.560392 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ed115460-b220-476d-b9d5-65f95ec3d059-etc-openvswitch\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.560762 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.560408 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ed115460-b220-476d-b9d5-65f95ec3d059-node-log\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.560762 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.560436 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ed115460-b220-476d-b9d5-65f95ec3d059-host-run-ovn-kubernetes\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.560762 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.560452 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a58d0c9c-b6c7-4962-90d1-34d10eb4e97c-sys\") pod \"tuned-j6pq9\" (UID: \"a58d0c9c-b6c7-4962-90d1-34d10eb4e97c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6pq9" Jul 12 18:17:09.560762 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.560466 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1b501a87-6ac0-4f0b-9072-aeecd64d5bc0-serviceca\") pod \"node-ca-khqqs\" (UID: \"1b501a87-6ac0-4f0b-9072-aeecd64d5bc0\") " pod="openshift-image-registry/node-ca-khqqs" Jul 12 18:17:09.560762 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.560482 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ed115460-b220-476d-b9d5-65f95ec3d059-ovnkube-script-lib\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.560762 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.560497 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a05fea52-1297-4bd7-82a2-d863030b675b-system-cni-dir\") pod \"multus-t9jvk\" (UID: \"a05fea52-1297-4bd7-82a2-d863030b675b\") " pod="openshift-multus/multus-t9jvk" Jul 12 18:17:09.560762 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.560515 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a05fea52-1297-4bd7-82a2-d863030b675b-cni-binary-copy\") pod \"multus-t9jvk\" (UID: \"a05fea52-1297-4bd7-82a2-d863030b675b\") " pod="openshift-multus/multus-t9jvk" Jul 12 18:17:09.560762 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.560540 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a05fea52-1297-4bd7-82a2-d863030b675b-host-var-lib-kubelet\") pod \"multus-t9jvk\" (UID: \"a05fea52-1297-4bd7-82a2-d863030b675b\") " pod="openshift-multus/multus-t9jvk" Jul 12 18:17:09.560762 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.560556 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94qmj\" (UniqueName: \"kubernetes.io/projected/a05fea52-1297-4bd7-82a2-d863030b675b-kube-api-access-94qmj\") pod \"multus-t9jvk\" (UID: \"a05fea52-1297-4bd7-82a2-d863030b675b\") " pod="openshift-multus/multus-t9jvk" Jul 12 18:17:09.560762 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.560572 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47zmn\" (UniqueName: \"kubernetes.io/projected/aaefba09-f06b-4e39-bc1b-a1b70455dc45-kube-api-access-47zmn\") pod \"aws-ebs-csi-driver-node-pbwgr\" (UID: \"aaefba09-f06b-4e39-bc1b-a1b70455dc45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pbwgr" Jul 12 18:17:09.560762 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.560597 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ed115460-b220-476d-b9d5-65f95ec3d059-ovn-node-metrics-cert\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.560762 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.560618 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/aaefba09-f06b-4e39-bc1b-a1b70455dc45-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pbwgr\" (UID: \"aaefba09-f06b-4e39-bc1b-a1b70455dc45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pbwgr" Jul 12 18:17:09.561441 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.560652 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a05fea52-1297-4bd7-82a2-d863030b675b-multus-socket-dir-parent\") pod \"multus-t9jvk\" (UID: \"a05fea52-1297-4bd7-82a2-d863030b675b\") " pod="openshift-multus/multus-t9jvk" Jul 12 18:17:09.561441 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.560669 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a05fea52-1297-4bd7-82a2-d863030b675b-os-release\") pod \"multus-t9jvk\" (UID: \"a05fea52-1297-4bd7-82a2-d863030b675b\") " pod="openshift-multus/multus-t9jvk" Jul 12 18:17:09.561441 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.560688 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a58d0c9c-b6c7-4962-90d1-34d10eb4e97c-etc-sysctl-conf\") pod \"tuned-j6pq9\" (UID: \"a58d0c9c-b6c7-4962-90d1-34d10eb4e97c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6pq9" Jul 12 18:17:09.561441 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.560709 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ed115460-b220-476d-b9d5-65f95ec3d059-systemd-units\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.561441 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.560736 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ed115460-b220-476d-b9d5-65f95ec3d059-host-cni-bin\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.561441 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.560758 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/aaefba09-f06b-4e39-bc1b-a1b70455dc45-socket-dir\") pod \"aws-ebs-csi-driver-node-pbwgr\" (UID: \"aaefba09-f06b-4e39-bc1b-a1b70455dc45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pbwgr" Jul 12 18:17:09.561441 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.560785 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/aaefba09-f06b-4e39-bc1b-a1b70455dc45-sys-fs\") pod \"aws-ebs-csi-driver-node-pbwgr\" (UID: \"aaefba09-f06b-4e39-bc1b-a1b70455dc45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pbwgr" Jul 12 18:17:09.561441 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.560805 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d69e388-ddc9-4dd8-b5dc-a81a59dc1698-metrics-certs\") pod \"network-metrics-daemon-r66xp\" (UID: \"7d69e388-ddc9-4dd8-b5dc-a81a59dc1698\") " pod="openshift-multus/network-metrics-daemon-r66xp" Jul 12 18:17:09.561441 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.560821 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cef566f5-8ee3-4192-bc98-fbcbbb9f29b8-host-slash\") pod \"iptables-alerter-7f47s\" (UID: \"cef566f5-8ee3-4192-bc98-fbcbbb9f29b8\") " pod="openshift-network-operator/iptables-alerter-7f47s" Jul 12 18:17:09.561441 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.560841 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a05fea52-1297-4bd7-82a2-d863030b675b-cnibin\") pod \"multus-t9jvk\" (UID: \"a05fea52-1297-4bd7-82a2-d863030b675b\") " pod="openshift-multus/multus-t9jvk" Jul 12 18:17:09.561441 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.560862 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ed115460-b220-476d-b9d5-65f95ec3d059-host-run-netns\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.561441 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.560878 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ed115460-b220-476d-b9d5-65f95ec3d059-var-lib-openvswitch\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.561441 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.560895 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z88qx\" (UniqueName: \"kubernetes.io/projected/ed115460-b220-476d-b9d5-65f95ec3d059-kube-api-access-z88qx\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.561441 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.560911 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbx9l\" (UniqueName: \"kubernetes.io/projected/7d69e388-ddc9-4dd8-b5dc-a81a59dc1698-kube-api-access-vbx9l\") pod \"network-metrics-daemon-r66xp\" (UID: \"7d69e388-ddc9-4dd8-b5dc-a81a59dc1698\") " pod="openshift-multus/network-metrics-daemon-r66xp" Jul 12 18:17:09.561441 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.560946 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nk5nd\" (UniqueName: \"kubernetes.io/projected/cef566f5-8ee3-4192-bc98-fbcbbb9f29b8-kube-api-access-nk5nd\") pod \"iptables-alerter-7f47s\" (UID: \"cef566f5-8ee3-4192-bc98-fbcbbb9f29b8\") " pod="openshift-network-operator/iptables-alerter-7f47s" Jul 12 18:17:09.561441 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.560974 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a05fea52-1297-4bd7-82a2-d863030b675b-host-var-lib-cni-multus\") pod \"multus-t9jvk\" (UID: \"a05fea52-1297-4bd7-82a2-d863030b675b\") " pod="openshift-multus/multus-t9jvk" Jul 12 18:17:09.562139 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.560999 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a05fea52-1297-4bd7-82a2-d863030b675b-etc-kubernetes\") pod \"multus-t9jvk\" (UID: \"a05fea52-1297-4bd7-82a2-d863030b675b\") " pod="openshift-multus/multus-t9jvk" Jul 12 18:17:09.562139 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.561036 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whnxw\" (UniqueName: \"kubernetes.io/projected/1b501a87-6ac0-4f0b-9072-aeecd64d5bc0-kube-api-access-whnxw\") pod \"node-ca-khqqs\" (UID: \"1b501a87-6ac0-4f0b-9072-aeecd64d5bc0\") " pod="openshift-image-registry/node-ca-khqqs" Jul 12 18:17:09.562139 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.561052 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmlnv\" (UniqueName: \"kubernetes.io/projected/1a19952f-4732-46da-98ee-05c476b44908-kube-api-access-fmlnv\") pod \"multus-additional-cni-plugins-29hd5\" (UID: \"1a19952f-4732-46da-98ee-05c476b44908\") " pod="openshift-multus/multus-additional-cni-plugins-29hd5" Jul 12 18:17:09.562139 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.561068 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ed115460-b220-476d-b9d5-65f95ec3d059-run-ovn\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.562139 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.561105 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/aaefba09-f06b-4e39-bc1b-a1b70455dc45-etc-selinux\") pod \"aws-ebs-csi-driver-node-pbwgr\" (UID: \"aaefba09-f06b-4e39-bc1b-a1b70455dc45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pbwgr" Jul 12 18:17:09.562139 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.561121 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a05fea52-1297-4bd7-82a2-d863030b675b-host-run-k8s-cni-cncf-io\") pod \"multus-t9jvk\" (UID: \"a05fea52-1297-4bd7-82a2-d863030b675b\") " pod="openshift-multus/multus-t9jvk" Jul 12 18:17:09.562139 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.561135 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a58d0c9c-b6c7-4962-90d1-34d10eb4e97c-etc-kubernetes\") pod \"tuned-j6pq9\" (UID: \"a58d0c9c-b6c7-4962-90d1-34d10eb4e97c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6pq9" Jul 12 18:17:09.562139 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.561158 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ed115460-b220-476d-b9d5-65f95ec3d059-host-slash\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.562139 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.561176 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ed115460-b220-476d-b9d5-65f95ec3d059-run-openvswitch\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.562139 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.561195 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/aaefba09-f06b-4e39-bc1b-a1b70455dc45-registration-dir\") pod \"aws-ebs-csi-driver-node-pbwgr\" (UID: \"aaefba09-f06b-4e39-bc1b-a1b70455dc45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pbwgr" Jul 12 18:17:09.562139 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.561211 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a05fea52-1297-4bd7-82a2-d863030b675b-multus-cni-dir\") pod \"multus-t9jvk\" (UID: \"a05fea52-1297-4bd7-82a2-d863030b675b\") " pod="openshift-multus/multus-t9jvk" Jul 12 18:17:09.562139 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.561247 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a58d0c9c-b6c7-4962-90d1-34d10eb4e97c-etc-sysctl-d\") pod \"tuned-j6pq9\" (UID: \"a58d0c9c-b6c7-4962-90d1-34d10eb4e97c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6pq9" Jul 12 18:17:09.562139 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.561264 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ed115460-b220-476d-b9d5-65f95ec3d059-log-socket\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.562139 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.561316 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cd8ab0de-04ec-4aa8-8d33-87aa4bbef597-hosts-file\") pod \"node-resolver-xwcbp\" (UID: \"cd8ab0de-04ec-4aa8-8d33-87aa4bbef597\") " pod="openshift-dns/node-resolver-xwcbp" Jul 12 18:17:09.562139 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.561337 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6nkz\" (UniqueName: \"kubernetes.io/projected/cd8ab0de-04ec-4aa8-8d33-87aa4bbef597-kube-api-access-d6nkz\") pod \"node-resolver-xwcbp\" (UID: \"cd8ab0de-04ec-4aa8-8d33-87aa4bbef597\") " pod="openshift-dns/node-resolver-xwcbp" Jul 12 18:17:09.567478 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.567461 2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 12 18:17:09.586961 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.586942 2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-nk7g5" Jul 12 18:17:09.596013 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.595988 2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-nk7g5" Jul 12 18:17:09.607575 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.607559 2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 18:17:09.655171 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.655137 2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 12 18:17:09.661826 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.661807 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9f8419c6-07be-4876-9762-d98cd772054b-original-pull-secret\") pod \"global-pull-secret-syncer-2598z\" (UID: \"9f8419c6-07be-4876-9762-d98cd772054b\") " pod="kube-system/global-pull-secret-syncer-2598z" Jul 12 18:17:09.661921 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.661833 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1a19952f-4732-46da-98ee-05c476b44908-cnibin\") pod \"multus-additional-cni-plugins-29hd5\" (UID: \"1a19952f-4732-46da-98ee-05c476b44908\") " pod="openshift-multus/multus-additional-cni-plugins-29hd5" Jul 12 18:17:09.661921 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.661854 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ed115460-b220-476d-b9d5-65f95ec3d059-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.661921 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.661872 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ed115460-b220-476d-b9d5-65f95ec3d059-ovnkube-config\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.662022 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.661915 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1a19952f-4732-46da-98ee-05c476b44908-cnibin\") pod \"multus-additional-cni-plugins-29hd5\" (UID: \"1a19952f-4732-46da-98ee-05c476b44908\") " pod="openshift-multus/multus-additional-cni-plugins-29hd5" Jul 12 18:17:09.662022 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.661933 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ed115460-b220-476d-b9d5-65f95ec3d059-env-overrides\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.662022 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.661960 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ed115460-b220-476d-b9d5-65f95ec3d059-host-cni-netd\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.662022 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.661946 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ed115460-b220-476d-b9d5-65f95ec3d059-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.662022 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:09.661992 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 18:17:09.662223 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.662025 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a05fea52-1297-4bd7-82a2-d863030b675b-multus-conf-dir\") pod \"multus-t9jvk\" (UID: \"a05fea52-1297-4bd7-82a2-d863030b675b\") " pod="openshift-multus/multus-t9jvk" Jul 12 18:17:09.662223 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.661979 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a05fea52-1297-4bd7-82a2-d863030b675b-multus-conf-dir\") pod \"multus-t9jvk\" (UID: \"a05fea52-1297-4bd7-82a2-d863030b675b\") " pod="openshift-multus/multus-t9jvk" Jul 12 18:17:09.662223 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.662055 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ed115460-b220-476d-b9d5-65f95ec3d059-host-cni-netd\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.662223 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:09.662065 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f8419c6-07be-4876-9762-d98cd772054b-original-pull-secret podName:9f8419c6-07be-4876-9762-d98cd772054b nodeName:}" failed. No retries permitted until 2026-07-12 18:17:10.162039142 +0000 UTC m=+3.288943946 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9f8419c6-07be-4876-9762-d98cd772054b-original-pull-secret") pod "global-pull-secret-syncer-2598z" (UID: "9f8419c6-07be-4876-9762-d98cd772054b") : object "kube-system"/"original-pull-secret" not registered Jul 12 18:17:09.662223 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.662112 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a58d0c9c-b6c7-4962-90d1-34d10eb4e97c-etc-sysconfig\") pod \"tuned-j6pq9\" (UID: \"a58d0c9c-b6c7-4962-90d1-34d10eb4e97c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6pq9" Jul 12 18:17:09.662223 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.662139 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1a19952f-4732-46da-98ee-05c476b44908-os-release\") pod \"multus-additional-cni-plugins-29hd5\" (UID: \"1a19952f-4732-46da-98ee-05c476b44908\") " pod="openshift-multus/multus-additional-cni-plugins-29hd5" Jul 12 18:17:09.662223 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.662169 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/aaefba09-f06b-4e39-bc1b-a1b70455dc45-device-dir\") pod \"aws-ebs-csi-driver-node-pbwgr\" (UID: \"aaefba09-f06b-4e39-bc1b-a1b70455dc45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pbwgr" Jul 12 18:17:09.662223 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.662196 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd8ab0de-04ec-4aa8-8d33-87aa4bbef597-tmp-dir\") pod \"node-resolver-xwcbp\" (UID: \"cd8ab0de-04ec-4aa8-8d33-87aa4bbef597\") " pod="openshift-dns/node-resolver-xwcbp" Jul 12 18:17:09.662223 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.662219 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a58d0c9c-b6c7-4962-90d1-34d10eb4e97c-etc-modprobe-d\") pod \"tuned-j6pq9\" (UID: \"a58d0c9c-b6c7-4962-90d1-34d10eb4e97c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6pq9" Jul 12 18:17:09.662223 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.662225 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a58d0c9c-b6c7-4962-90d1-34d10eb4e97c-etc-sysconfig\") pod \"tuned-j6pq9\" (UID: \"a58d0c9c-b6c7-4962-90d1-34d10eb4e97c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6pq9" Jul 12 18:17:09.662831 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.662245 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a58d0c9c-b6c7-4962-90d1-34d10eb4e97c-lib-modules\") pod \"tuned-j6pq9\" (UID: \"a58d0c9c-b6c7-4962-90d1-34d10eb4e97c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6pq9" Jul 12 18:17:09.662831 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.662244 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/aaefba09-f06b-4e39-bc1b-a1b70455dc45-device-dir\") pod \"aws-ebs-csi-driver-node-pbwgr\" (UID: \"aaefba09-f06b-4e39-bc1b-a1b70455dc45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pbwgr" Jul 12 18:17:09.662831 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.662269 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a58d0c9c-b6c7-4962-90d1-34d10eb4e97c-tmp\") pod \"tuned-j6pq9\" (UID: \"a58d0c9c-b6c7-4962-90d1-34d10eb4e97c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6pq9" Jul 12 18:17:09.662831 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.662294 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9f8419c6-07be-4876-9762-d98cd772054b-dbus\") pod \"global-pull-secret-syncer-2598z\" (UID: \"9f8419c6-07be-4876-9762-d98cd772054b\") " pod="kube-system/global-pull-secret-syncer-2598z" Jul 12 18:17:09.662831 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.662308 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1a19952f-4732-46da-98ee-05c476b44908-os-release\") pod \"multus-additional-cni-plugins-29hd5\" (UID: \"1a19952f-4732-46da-98ee-05c476b44908\") " pod="openshift-multus/multus-additional-cni-plugins-29hd5" Jul 12 18:17:09.662831 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.662317 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1b501a87-6ac0-4f0b-9072-aeecd64d5bc0-host\") pod \"node-ca-khqqs\" (UID: \"1b501a87-6ac0-4f0b-9072-aeecd64d5bc0\") " pod="openshift-image-registry/node-ca-khqqs" Jul 12 18:17:09.662831 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.662337 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a58d0c9c-b6c7-4962-90d1-34d10eb4e97c-etc-modprobe-d\") pod \"tuned-j6pq9\" (UID: \"a58d0c9c-b6c7-4962-90d1-34d10eb4e97c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6pq9" Jul 12 18:17:09.662831 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.662351 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1b501a87-6ac0-4f0b-9072-aeecd64d5bc0-host\") pod \"node-ca-khqqs\" (UID: \"1b501a87-6ac0-4f0b-9072-aeecd64d5bc0\") " pod="openshift-image-registry/node-ca-khqqs" Jul 12 18:17:09.662831 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.662377 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ed115460-b220-476d-b9d5-65f95ec3d059-host-kubelet\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.662831 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.662423 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1a19952f-4732-46da-98ee-05c476b44908-tuning-conf-dir\") pod \"multus-additional-cni-plugins-29hd5\" (UID: \"1a19952f-4732-46da-98ee-05c476b44908\") " pod="openshift-multus/multus-additional-cni-plugins-29hd5" Jul 12 18:17:09.662831 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.662458 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a58d0c9c-b6c7-4962-90d1-34d10eb4e97c-lib-modules\") pod \"tuned-j6pq9\" (UID: \"a58d0c9c-b6c7-4962-90d1-34d10eb4e97c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6pq9" Jul 12 18:17:09.662831 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.662468 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9f8419c6-07be-4876-9762-d98cd772054b-dbus\") pod \"global-pull-secret-syncer-2598z\" (UID: \"9f8419c6-07be-4876-9762-d98cd772054b\") " pod="kube-system/global-pull-secret-syncer-2598z" Jul 12 18:17:09.662831 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.662481 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a05fea52-1297-4bd7-82a2-d863030b675b-multus-daemon-config\") pod \"multus-t9jvk\" (UID: \"a05fea52-1297-4bd7-82a2-d863030b675b\") " pod="openshift-multus/multus-t9jvk" Jul 12 18:17:09.662831 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.662510 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a58d0c9c-b6c7-4962-90d1-34d10eb4e97c-run\") pod \"tuned-j6pq9\" (UID: \"a58d0c9c-b6c7-4962-90d1-34d10eb4e97c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6pq9" Jul 12 18:17:09.662831 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.662525 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ed115460-b220-476d-b9d5-65f95ec3d059-host-kubelet\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.662831 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.662552 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a58d0c9c-b6c7-4962-90d1-34d10eb4e97c-host\") pod \"tuned-j6pq9\" (UID: \"a58d0c9c-b6c7-4962-90d1-34d10eb4e97c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6pq9" Jul 12 18:17:09.662831 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.662576 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ed115460-b220-476d-b9d5-65f95ec3d059-env-overrides\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.662831 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.662613 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd8ab0de-04ec-4aa8-8d33-87aa4bbef597-tmp-dir\") pod \"node-resolver-xwcbp\" (UID: \"cd8ab0de-04ec-4aa8-8d33-87aa4bbef597\") " pod="openshift-dns/node-resolver-xwcbp" Jul 12 18:17:09.663762 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.662631 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ed115460-b220-476d-b9d5-65f95ec3d059-ovnkube-config\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.663762 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.662581 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1a19952f-4732-46da-98ee-05c476b44908-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-29hd5\" (UID: \"1a19952f-4732-46da-98ee-05c476b44908\") " pod="openshift-multus/multus-additional-cni-plugins-29hd5" Jul 12 18:17:09.663762 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.662625 2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 12 18:17:09.663762 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.662679 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ed115460-b220-476d-b9d5-65f95ec3d059-etc-openvswitch\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.663762 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.662735 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ed115460-b220-476d-b9d5-65f95ec3d059-node-log\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.663762 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.662722 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a58d0c9c-b6c7-4962-90d1-34d10eb4e97c-host\") pod \"tuned-j6pq9\" (UID: \"a58d0c9c-b6c7-4962-90d1-34d10eb4e97c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6pq9" Jul 12 18:17:09.663762 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.662767 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ed115460-b220-476d-b9d5-65f95ec3d059-host-run-ovn-kubernetes\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.663762 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.663042 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1a19952f-4732-46da-98ee-05c476b44908-tuning-conf-dir\") pod \"multus-additional-cni-plugins-29hd5\" (UID: \"1a19952f-4732-46da-98ee-05c476b44908\") " pod="openshift-multus/multus-additional-cni-plugins-29hd5" Jul 12 18:17:09.663762 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.662932 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ed115460-b220-476d-b9d5-65f95ec3d059-etc-openvswitch\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.663762 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.663492 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a05fea52-1297-4bd7-82a2-d863030b675b-multus-daemon-config\") pod \"multus-t9jvk\" (UID: \"a05fea52-1297-4bd7-82a2-d863030b675b\") " pod="openshift-multus/multus-t9jvk" Jul 12 18:17:09.663762 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.663654 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a58d0c9c-b6c7-4962-90d1-34d10eb4e97c-sys\") pod \"tuned-j6pq9\" (UID: \"a58d0c9c-b6c7-4962-90d1-34d10eb4e97c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6pq9" Jul 12 18:17:09.663762 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.663692 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1b501a87-6ac0-4f0b-9072-aeecd64d5bc0-serviceca\") pod \"node-ca-khqqs\" (UID: \"1b501a87-6ac0-4f0b-9072-aeecd64d5bc0\") " pod="openshift-image-registry/node-ca-khqqs" Jul 12 18:17:09.663762 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.663746 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ed115460-b220-476d-b9d5-65f95ec3d059-ovnkube-script-lib\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.664428 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.663847 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ed115460-b220-476d-b9d5-65f95ec3d059-node-log\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.664428 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.663935 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a58d0c9c-b6c7-4962-90d1-34d10eb4e97c-sys\") pod \"tuned-j6pq9\" (UID: \"a58d0c9c-b6c7-4962-90d1-34d10eb4e97c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6pq9" Jul 12 18:17:09.664428 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.663984 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e1c57461-73ff-431a-a529-60fa73bf37b5-agent-certs\") pod \"konnectivity-agent-hqhkm\" (UID: \"e1c57461-73ff-431a-a529-60fa73bf37b5\") " pod="kube-system/konnectivity-agent-hqhkm" Jul 12 18:17:09.664428 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.664014 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e1c57461-73ff-431a-a529-60fa73bf37b5-konnectivity-ca\") pod \"konnectivity-agent-hqhkm\" (UID: \"e1c57461-73ff-431a-a529-60fa73bf37b5\") " pod="kube-system/konnectivity-agent-hqhkm" Jul 12 18:17:09.664428 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.664042 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a05fea52-1297-4bd7-82a2-d863030b675b-system-cni-dir\") pod \"multus-t9jvk\" (UID: \"a05fea52-1297-4bd7-82a2-d863030b675b\") " pod="openshift-multus/multus-t9jvk" Jul 12 18:17:09.664428 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.664085 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a05fea52-1297-4bd7-82a2-d863030b675b-cni-binary-copy\") pod \"multus-t9jvk\" (UID: \"a05fea52-1297-4bd7-82a2-d863030b675b\") " pod="openshift-multus/multus-t9jvk" Jul 12 18:17:09.664428 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.664091 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1a19952f-4732-46da-98ee-05c476b44908-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-29hd5\" (UID: \"1a19952f-4732-46da-98ee-05c476b44908\") " pod="openshift-multus/multus-additional-cni-plugins-29hd5" Jul 12 18:17:09.664428 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.664115 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a05fea52-1297-4bd7-82a2-d863030b675b-host-var-lib-kubelet\") pod \"multus-t9jvk\" (UID: \"a05fea52-1297-4bd7-82a2-d863030b675b\") " pod="openshift-multus/multus-t9jvk" Jul 12 18:17:09.664428 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.664161 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94qmj\" (UniqueName: \"kubernetes.io/projected/a05fea52-1297-4bd7-82a2-d863030b675b-kube-api-access-94qmj\") pod \"multus-t9jvk\" (UID: \"a05fea52-1297-4bd7-82a2-d863030b675b\") " pod="openshift-multus/multus-t9jvk" Jul 12 18:17:09.664428 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.664185 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a58d0c9c-b6c7-4962-90d1-34d10eb4e97c-run\") pod \"tuned-j6pq9\" (UID: \"a58d0c9c-b6c7-4962-90d1-34d10eb4e97c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6pq9" Jul 12 18:17:09.664428 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.664239 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47zmn\" (UniqueName: \"kubernetes.io/projected/aaefba09-f06b-4e39-bc1b-a1b70455dc45-kube-api-access-47zmn\") pod \"aws-ebs-csi-driver-node-pbwgr\" (UID: \"aaefba09-f06b-4e39-bc1b-a1b70455dc45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pbwgr" Jul 12 18:17:09.664428 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.664252 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1b501a87-6ac0-4f0b-9072-aeecd64d5bc0-serviceca\") pod \"node-ca-khqqs\" (UID: \"1b501a87-6ac0-4f0b-9072-aeecd64d5bc0\") " pod="openshift-image-registry/node-ca-khqqs" Jul 12 18:17:09.664428 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.664278 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ed115460-b220-476d-b9d5-65f95ec3d059-ovn-node-metrics-cert\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.665131 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.664243 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ed115460-b220-476d-b9d5-65f95ec3d059-host-run-ovn-kubernetes\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.665131 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.664674 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a05fea52-1297-4bd7-82a2-d863030b675b-system-cni-dir\") pod \"multus-t9jvk\" (UID: \"a05fea52-1297-4bd7-82a2-d863030b675b\") " pod="openshift-multus/multus-t9jvk" Jul 12 18:17:09.665131 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.664670 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/aaefba09-f06b-4e39-bc1b-a1b70455dc45-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pbwgr\" (UID: \"aaefba09-f06b-4e39-bc1b-a1b70455dc45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pbwgr" Jul 12 18:17:09.665131 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.664760 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/aaefba09-f06b-4e39-bc1b-a1b70455dc45-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pbwgr\" (UID: \"aaefba09-f06b-4e39-bc1b-a1b70455dc45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pbwgr" Jul 12 18:17:09.665131 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.664808 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a05fea52-1297-4bd7-82a2-d863030b675b-multus-socket-dir-parent\") pod \"multus-t9jvk\" (UID: \"a05fea52-1297-4bd7-82a2-d863030b675b\") " pod="openshift-multus/multus-t9jvk" Jul 12 18:17:09.665131 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.664839 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a05fea52-1297-4bd7-82a2-d863030b675b-host-var-lib-kubelet\") pod \"multus-t9jvk\" (UID: \"a05fea52-1297-4bd7-82a2-d863030b675b\") " pod="openshift-multus/multus-t9jvk" Jul 12 18:17:09.665131 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.664893 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ed115460-b220-476d-b9d5-65f95ec3d059-ovnkube-script-lib\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.665131 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.664909 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a05fea52-1297-4bd7-82a2-d863030b675b-os-release\") pod \"multus-t9jvk\" (UID: \"a05fea52-1297-4bd7-82a2-d863030b675b\") " pod="openshift-multus/multus-t9jvk" Jul 12 18:17:09.665131 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.664951 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a05fea52-1297-4bd7-82a2-d863030b675b-multus-socket-dir-parent\") pod \"multus-t9jvk\" (UID: \"a05fea52-1297-4bd7-82a2-d863030b675b\") " pod="openshift-multus/multus-t9jvk" Jul 12 18:17:09.665131 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.665009 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a58d0c9c-b6c7-4962-90d1-34d10eb4e97c-etc-sysctl-conf\") pod \"tuned-j6pq9\" (UID: \"a58d0c9c-b6c7-4962-90d1-34d10eb4e97c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6pq9" Jul 12 18:17:09.665131 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.665048 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ed115460-b220-476d-b9d5-65f95ec3d059-systemd-units\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.665131 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.665083 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ed115460-b220-476d-b9d5-65f95ec3d059-host-cni-bin\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.665131 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.665102 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a05fea52-1297-4bd7-82a2-d863030b675b-os-release\") pod \"multus-t9jvk\" (UID: \"a05fea52-1297-4bd7-82a2-d863030b675b\") " pod="openshift-multus/multus-t9jvk" Jul 12 18:17:09.665131 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.665119 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/aaefba09-f06b-4e39-bc1b-a1b70455dc45-socket-dir\") pod \"aws-ebs-csi-driver-node-pbwgr\" (UID: \"aaefba09-f06b-4e39-bc1b-a1b70455dc45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pbwgr" Jul 12 18:17:09.665802 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.665155 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/aaefba09-f06b-4e39-bc1b-a1b70455dc45-sys-fs\") pod \"aws-ebs-csi-driver-node-pbwgr\" (UID: \"aaefba09-f06b-4e39-bc1b-a1b70455dc45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pbwgr" Jul 12 18:17:09.665802 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.665158 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ed115460-b220-476d-b9d5-65f95ec3d059-systemd-units\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.665802 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.665190 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d69e388-ddc9-4dd8-b5dc-a81a59dc1698-metrics-certs\") pod \"network-metrics-daemon-r66xp\" (UID: \"7d69e388-ddc9-4dd8-b5dc-a81a59dc1698\") " pod="openshift-multus/network-metrics-daemon-r66xp" Jul 12 18:17:09.665802 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.665225 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cef566f5-8ee3-4192-bc98-fbcbbb9f29b8-host-slash\") pod \"iptables-alerter-7f47s\" (UID: \"cef566f5-8ee3-4192-bc98-fbcbbb9f29b8\") " pod="openshift-network-operator/iptables-alerter-7f47s" Jul 12 18:17:09.665802 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.665259 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a05fea52-1297-4bd7-82a2-d863030b675b-cnibin\") pod \"multus-t9jvk\" (UID: \"a05fea52-1297-4bd7-82a2-d863030b675b\") " pod="openshift-multus/multus-t9jvk" Jul 12 18:17:09.665802 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.665269 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a58d0c9c-b6c7-4962-90d1-34d10eb4e97c-etc-sysctl-conf\") pod \"tuned-j6pq9\" (UID: \"a58d0c9c-b6c7-4962-90d1-34d10eb4e97c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6pq9" Jul 12 18:17:09.665802 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.665306 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ed115460-b220-476d-b9d5-65f95ec3d059-host-run-netns\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.665802 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.665334 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ed115460-b220-476d-b9d5-65f95ec3d059-var-lib-openvswitch\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.665802 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.665342 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/aaefba09-f06b-4e39-bc1b-a1b70455dc45-sys-fs\") pod \"aws-ebs-csi-driver-node-pbwgr\" (UID: \"aaefba09-f06b-4e39-bc1b-a1b70455dc45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pbwgr" Jul 12 18:17:09.665802 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.665370 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z88qx\" (UniqueName: \"kubernetes.io/projected/ed115460-b220-476d-b9d5-65f95ec3d059-kube-api-access-z88qx\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.665802 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.665391 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ed115460-b220-476d-b9d5-65f95ec3d059-host-cni-bin\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.665802 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.665402 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbx9l\" (UniqueName: \"kubernetes.io/projected/7d69e388-ddc9-4dd8-b5dc-a81a59dc1698-kube-api-access-vbx9l\") pod \"network-metrics-daemon-r66xp\" (UID: \"7d69e388-ddc9-4dd8-b5dc-a81a59dc1698\") " pod="openshift-multus/network-metrics-daemon-r66xp" Jul 12 18:17:09.665802 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.665434 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nk5nd\" (UniqueName: \"kubernetes.io/projected/cef566f5-8ee3-4192-bc98-fbcbbb9f29b8-kube-api-access-nk5nd\") pod \"iptables-alerter-7f47s\" (UID: \"cef566f5-8ee3-4192-bc98-fbcbbb9f29b8\") " pod="openshift-network-operator/iptables-alerter-7f47s" Jul 12 18:17:09.665802 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.665464 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a05fea52-1297-4bd7-82a2-d863030b675b-host-var-lib-cni-multus\") pod \"multus-t9jvk\" (UID: \"a05fea52-1297-4bd7-82a2-d863030b675b\") " pod="openshift-multus/multus-t9jvk" Jul 12 18:17:09.665802 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.665497 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a05fea52-1297-4bd7-82a2-d863030b675b-etc-kubernetes\") pod \"multus-t9jvk\" (UID: \"a05fea52-1297-4bd7-82a2-d863030b675b\") " pod="openshift-multus/multus-t9jvk" Jul 12 18:17:09.665802 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.665507 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/aaefba09-f06b-4e39-bc1b-a1b70455dc45-socket-dir\") pod \"aws-ebs-csi-driver-node-pbwgr\" (UID: \"aaefba09-f06b-4e39-bc1b-a1b70455dc45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pbwgr" Jul 12 18:17:09.665802 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.665534 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whnxw\" (UniqueName: \"kubernetes.io/projected/1b501a87-6ac0-4f0b-9072-aeecd64d5bc0-kube-api-access-whnxw\") pod \"node-ca-khqqs\" (UID: \"1b501a87-6ac0-4f0b-9072-aeecd64d5bc0\") " pod="openshift-image-registry/node-ca-khqqs" Jul 12 18:17:09.666573 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.665569 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ed115460-b220-476d-b9d5-65f95ec3d059-var-lib-openvswitch\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.666573 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:09.665862 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 18:17:09.666573 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.665915 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a05fea52-1297-4bd7-82a2-d863030b675b-host-var-lib-cni-multus\") pod \"multus-t9jvk\" (UID: \"a05fea52-1297-4bd7-82a2-d863030b675b\") " pod="openshift-multus/multus-t9jvk" Jul 12 18:17:09.666573 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:09.665933 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d69e388-ddc9-4dd8-b5dc-a81a59dc1698-metrics-certs podName:7d69e388-ddc9-4dd8-b5dc-a81a59dc1698 nodeName:}" failed. No retries permitted until 2026-07-12 18:17:10.165903003 +0000 UTC m=+3.292807787 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7d69e388-ddc9-4dd8-b5dc-a81a59dc1698-metrics-certs") pod "network-metrics-daemon-r66xp" (UID: "7d69e388-ddc9-4dd8-b5dc-a81a59dc1698") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 18:17:09.666573 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.666008 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a05fea52-1297-4bd7-82a2-d863030b675b-cnibin\") pod \"multus-t9jvk\" (UID: \"a05fea52-1297-4bd7-82a2-d863030b675b\") " pod="openshift-multus/multus-t9jvk" Jul 12 18:17:09.666573 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.666054 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cef566f5-8ee3-4192-bc98-fbcbbb9f29b8-host-slash\") pod \"iptables-alerter-7f47s\" (UID: \"cef566f5-8ee3-4192-bc98-fbcbbb9f29b8\") " pod="openshift-network-operator/iptables-alerter-7f47s" Jul 12 18:17:09.666573 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.666258 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ed115460-b220-476d-b9d5-65f95ec3d059-host-run-netns\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.666573 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.665571 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmlnv\" (UniqueName: \"kubernetes.io/projected/1a19952f-4732-46da-98ee-05c476b44908-kube-api-access-fmlnv\") pod \"multus-additional-cni-plugins-29hd5\" (UID: \"1a19952f-4732-46da-98ee-05c476b44908\") " pod="openshift-multus/multus-additional-cni-plugins-29hd5" Jul 12 18:17:09.666573 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.666317 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ed115460-b220-476d-b9d5-65f95ec3d059-run-ovn\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.666573 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.666357 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/aaefba09-f06b-4e39-bc1b-a1b70455dc45-etc-selinux\") pod \"aws-ebs-csi-driver-node-pbwgr\" (UID: \"aaefba09-f06b-4e39-bc1b-a1b70455dc45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pbwgr" Jul 12 18:17:09.666573 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.666393 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a05fea52-1297-4bd7-82a2-d863030b675b-host-run-k8s-cni-cncf-io\") pod \"multus-t9jvk\" (UID: \"a05fea52-1297-4bd7-82a2-d863030b675b\") " pod="openshift-multus/multus-t9jvk" Jul 12 18:17:09.666573 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.666423 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a58d0c9c-b6c7-4962-90d1-34d10eb4e97c-etc-kubernetes\") pod \"tuned-j6pq9\" (UID: \"a58d0c9c-b6c7-4962-90d1-34d10eb4e97c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6pq9" Jul 12 18:17:09.666573 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.666453 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ed115460-b220-476d-b9d5-65f95ec3d059-host-slash\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.666573 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.666486 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ed115460-b220-476d-b9d5-65f95ec3d059-run-openvswitch\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.666573 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.666517 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/aaefba09-f06b-4e39-bc1b-a1b70455dc45-registration-dir\") pod \"aws-ebs-csi-driver-node-pbwgr\" (UID: \"aaefba09-f06b-4e39-bc1b-a1b70455dc45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pbwgr" Jul 12 18:17:09.666573 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.666547 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a05fea52-1297-4bd7-82a2-d863030b675b-multus-cni-dir\") pod \"multus-t9jvk\" (UID: \"a05fea52-1297-4bd7-82a2-d863030b675b\") " pod="openshift-multus/multus-t9jvk" Jul 12 18:17:09.666573 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.666574 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a58d0c9c-b6c7-4962-90d1-34d10eb4e97c-etc-sysctl-d\") pod \"tuned-j6pq9\" (UID: \"a58d0c9c-b6c7-4962-90d1-34d10eb4e97c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6pq9" Jul 12 18:17:09.667448 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.666580 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a05fea52-1297-4bd7-82a2-d863030b675b-host-run-k8s-cni-cncf-io\") pod \"multus-t9jvk\" (UID: \"a05fea52-1297-4bd7-82a2-d863030b675b\") " pod="openshift-multus/multus-t9jvk" Jul 12 18:17:09.667448 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.666606 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ed115460-b220-476d-b9d5-65f95ec3d059-log-socket\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.667448 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.666642 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ed115460-b220-476d-b9d5-65f95ec3d059-run-ovn\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.667448 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.666660 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ed115460-b220-476d-b9d5-65f95ec3d059-log-socket\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.667448 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.666669 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a05fea52-1297-4bd7-82a2-d863030b675b-etc-kubernetes\") pod \"multus-t9jvk\" (UID: \"a05fea52-1297-4bd7-82a2-d863030b675b\") " pod="openshift-multus/multus-t9jvk" Jul 12 18:17:09.667448 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.666735 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ed115460-b220-476d-b9d5-65f95ec3d059-run-openvswitch\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.667448 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.666752 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/aaefba09-f06b-4e39-bc1b-a1b70455dc45-registration-dir\") pod \"aws-ebs-csi-driver-node-pbwgr\" (UID: \"aaefba09-f06b-4e39-bc1b-a1b70455dc45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pbwgr" Jul 12 18:17:09.667448 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.666791 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/aaefba09-f06b-4e39-bc1b-a1b70455dc45-etc-selinux\") pod \"aws-ebs-csi-driver-node-pbwgr\" (UID: \"aaefba09-f06b-4e39-bc1b-a1b70455dc45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pbwgr" Jul 12 18:17:09.667448 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.666808 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ed115460-b220-476d-b9d5-65f95ec3d059-host-slash\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.667448 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.666838 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cd8ab0de-04ec-4aa8-8d33-87aa4bbef597-hosts-file\") pod \"node-resolver-xwcbp\" (UID: \"cd8ab0de-04ec-4aa8-8d33-87aa4bbef597\") " pod="openshift-dns/node-resolver-xwcbp" Jul 12 18:17:09.667448 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.666852 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a58d0c9c-b6c7-4962-90d1-34d10eb4e97c-etc-sysctl-d\") pod \"tuned-j6pq9\" (UID: \"a58d0c9c-b6c7-4962-90d1-34d10eb4e97c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6pq9" Jul 12 18:17:09.667448 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.666874 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6nkz\" (UniqueName: \"kubernetes.io/projected/cd8ab0de-04ec-4aa8-8d33-87aa4bbef597-kube-api-access-d6nkz\") pod \"node-resolver-xwcbp\" (UID: \"cd8ab0de-04ec-4aa8-8d33-87aa4bbef597\") " pod="openshift-dns/node-resolver-xwcbp" Jul 12 18:17:09.667448 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.666879 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a05fea52-1297-4bd7-82a2-d863030b675b-multus-cni-dir\") pod \"multus-t9jvk\" (UID: \"a05fea52-1297-4bd7-82a2-d863030b675b\") " pod="openshift-multus/multus-t9jvk" Jul 12 18:17:09.667448 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.666904 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a58d0c9c-b6c7-4962-90d1-34d10eb4e97c-etc-kubernetes\") pod \"tuned-j6pq9\" (UID: \"a58d0c9c-b6c7-4962-90d1-34d10eb4e97c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6pq9" Jul 12 18:17:09.667448 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.666913 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpzmv\" (UniqueName: \"kubernetes.io/projected/4d1fc6eb-09af-4f96-8ec8-b193da43354c-kube-api-access-tpzmv\") pod \"network-check-target-lfnkm\" (UID: \"4d1fc6eb-09af-4f96-8ec8-b193da43354c\") " pod="openshift-network-diagnostics/network-check-target-lfnkm" Jul 12 18:17:09.667448 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.666947 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cd8ab0de-04ec-4aa8-8d33-87aa4bbef597-hosts-file\") pod \"node-resolver-xwcbp\" (UID: \"cd8ab0de-04ec-4aa8-8d33-87aa4bbef597\") " pod="openshift-dns/node-resolver-xwcbp" Jul 12 18:17:09.667448 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.667083 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cef566f5-8ee3-4192-bc98-fbcbbb9f29b8-iptables-alerter-script\") pod \"iptables-alerter-7f47s\" (UID: \"cef566f5-8ee3-4192-bc98-fbcbbb9f29b8\") " pod="openshift-network-operator/iptables-alerter-7f47s" Jul 12 18:17:09.667448 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.667115 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a58d0c9c-b6c7-4962-90d1-34d10eb4e97c-etc-systemd\") pod \"tuned-j6pq9\" (UID: \"a58d0c9c-b6c7-4962-90d1-34d10eb4e97c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6pq9" Jul 12 18:17:09.668333 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.667179 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a58d0c9c-b6c7-4962-90d1-34d10eb4e97c-etc-systemd\") pod \"tuned-j6pq9\" (UID: \"a58d0c9c-b6c7-4962-90d1-34d10eb4e97c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6pq9" Jul 12 18:17:09.668333 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.667221 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1a19952f-4732-46da-98ee-05c476b44908-system-cni-dir\") pod \"multus-additional-cni-plugins-29hd5\" (UID: \"1a19952f-4732-46da-98ee-05c476b44908\") " pod="openshift-multus/multus-additional-cni-plugins-29hd5" Jul 12 18:17:09.668333 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.667258 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1a19952f-4732-46da-98ee-05c476b44908-cni-binary-copy\") pod \"multus-additional-cni-plugins-29hd5\" (UID: \"1a19952f-4732-46da-98ee-05c476b44908\") " pod="openshift-multus/multus-additional-cni-plugins-29hd5" Jul 12 18:17:09.668333 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.667261 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a05fea52-1297-4bd7-82a2-d863030b675b-cni-binary-copy\") pod \"multus-t9jvk\" (UID: \"a05fea52-1297-4bd7-82a2-d863030b675b\") " pod="openshift-multus/multus-t9jvk" Jul 12 18:17:09.668333 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.667296 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ed115460-b220-476d-b9d5-65f95ec3d059-run-systemd\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.668333 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.667354 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1a19952f-4732-46da-98ee-05c476b44908-system-cni-dir\") pod \"multus-additional-cni-plugins-29hd5\" (UID: \"1a19952f-4732-46da-98ee-05c476b44908\") " pod="openshift-multus/multus-additional-cni-plugins-29hd5" Jul 12 18:17:09.668333 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.667357 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ed115460-b220-476d-b9d5-65f95ec3d059-run-systemd\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.668333 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.667415 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1a19952f-4732-46da-98ee-05c476b44908-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-29hd5\" (UID: \"1a19952f-4732-46da-98ee-05c476b44908\") " pod="openshift-multus/multus-additional-cni-plugins-29hd5" Jul 12 18:17:09.668333 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.667453 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a05fea52-1297-4bd7-82a2-d863030b675b-host-run-netns\") pod \"multus-t9jvk\" (UID: \"a05fea52-1297-4bd7-82a2-d863030b675b\") " pod="openshift-multus/multus-t9jvk" Jul 12 18:17:09.668333 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.667475 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a58d0c9c-b6c7-4962-90d1-34d10eb4e97c-tmp\") pod \"tuned-j6pq9\" (UID: \"a58d0c9c-b6c7-4962-90d1-34d10eb4e97c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6pq9" Jul 12 18:17:09.668333 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.667485 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a05fea52-1297-4bd7-82a2-d863030b675b-host-var-lib-cni-bin\") pod \"multus-t9jvk\" (UID: \"a05fea52-1297-4bd7-82a2-d863030b675b\") " pod="openshift-multus/multus-t9jvk" Jul 12 18:17:09.668333 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.667531 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a05fea52-1297-4bd7-82a2-d863030b675b-host-run-netns\") pod \"multus-t9jvk\" (UID: \"a05fea52-1297-4bd7-82a2-d863030b675b\") " pod="openshift-multus/multus-t9jvk" Jul 12 18:17:09.668333 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.667537 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a05fea52-1297-4bd7-82a2-d863030b675b-host-run-multus-certs\") pod \"multus-t9jvk\" (UID: \"a05fea52-1297-4bd7-82a2-d863030b675b\") " pod="openshift-multus/multus-t9jvk" Jul 12 18:17:09.668333 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.667574 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a58d0c9c-b6c7-4962-90d1-34d10eb4e97c-etc-tuned\") pod \"tuned-j6pq9\" (UID: \"a58d0c9c-b6c7-4962-90d1-34d10eb4e97c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6pq9" Jul 12 18:17:09.668333 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.667609 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rc9w4\" (UniqueName: \"kubernetes.io/projected/a58d0c9c-b6c7-4962-90d1-34d10eb4e97c-kube-api-access-rc9w4\") pod \"tuned-j6pq9\" (UID: \"a58d0c9c-b6c7-4962-90d1-34d10eb4e97c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6pq9" Jul 12 18:17:09.668333 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.667638 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a05fea52-1297-4bd7-82a2-d863030b675b-hostroot\") pod \"multus-t9jvk\" (UID: \"a05fea52-1297-4bd7-82a2-d863030b675b\") " pod="openshift-multus/multus-t9jvk" Jul 12 18:17:09.668333 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.667641 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a05fea52-1297-4bd7-82a2-d863030b675b-host-var-lib-cni-bin\") pod \"multus-t9jvk\" (UID: \"a05fea52-1297-4bd7-82a2-d863030b675b\") " pod="openshift-multus/multus-t9jvk" Jul 12 18:17:09.668333 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.667694 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a58d0c9c-b6c7-4962-90d1-34d10eb4e97c-var-lib-kubelet\") pod \"tuned-j6pq9\" (UID: \"a58d0c9c-b6c7-4962-90d1-34d10eb4e97c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6pq9" Jul 12 18:17:09.669035 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.667730 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9f8419c6-07be-4876-9762-d98cd772054b-kubelet-config\") pod \"global-pull-secret-syncer-2598z\" (UID: \"9f8419c6-07be-4876-9762-d98cd772054b\") " pod="kube-system/global-pull-secret-syncer-2598z" Jul 12 18:17:09.669035 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.667838 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9f8419c6-07be-4876-9762-d98cd772054b-kubelet-config\") pod \"global-pull-secret-syncer-2598z\" (UID: \"9f8419c6-07be-4876-9762-d98cd772054b\") " pod="kube-system/global-pull-secret-syncer-2598z" Jul 12 18:17:09.669035 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.667609 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a05fea52-1297-4bd7-82a2-d863030b675b-host-run-multus-certs\") pod \"multus-t9jvk\" (UID: \"a05fea52-1297-4bd7-82a2-d863030b675b\") " pod="openshift-multus/multus-t9jvk" Jul 12 18:17:09.669035 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.667891 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1a19952f-4732-46da-98ee-05c476b44908-cni-binary-copy\") pod \"multus-additional-cni-plugins-29hd5\" (UID: \"1a19952f-4732-46da-98ee-05c476b44908\") " pod="openshift-multus/multus-additional-cni-plugins-29hd5" Jul 12 18:17:09.669035 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.667914 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a05fea52-1297-4bd7-82a2-d863030b675b-hostroot\") pod \"multus-t9jvk\" (UID: \"a05fea52-1297-4bd7-82a2-d863030b675b\") " pod="openshift-multus/multus-t9jvk" Jul 12 18:17:09.669035 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.668013 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a58d0c9c-b6c7-4962-90d1-34d10eb4e97c-var-lib-kubelet\") pod \"tuned-j6pq9\" (UID: \"a58d0c9c-b6c7-4962-90d1-34d10eb4e97c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6pq9" Jul 12 18:17:09.669035 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.668363 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cef566f5-8ee3-4192-bc98-fbcbbb9f29b8-iptables-alerter-script\") pod \"iptables-alerter-7f47s\" (UID: \"cef566f5-8ee3-4192-bc98-fbcbbb9f29b8\") " pod="openshift-network-operator/iptables-alerter-7f47s" Jul 12 18:17:09.669035 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.668417 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ed115460-b220-476d-b9d5-65f95ec3d059-ovn-node-metrics-cert\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.669393 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.669255 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1a19952f-4732-46da-98ee-05c476b44908-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-29hd5\" (UID: \"1a19952f-4732-46da-98ee-05c476b44908\") " pod="openshift-multus/multus-additional-cni-plugins-29hd5" Jul 12 18:17:09.669808 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.669789 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a58d0c9c-b6c7-4962-90d1-34d10eb4e97c-etc-tuned\") pod \"tuned-j6pq9\" (UID: \"a58d0c9c-b6c7-4962-90d1-34d10eb4e97c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6pq9" Jul 12 18:17:09.672827 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.672800 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94qmj\" (UniqueName: \"kubernetes.io/projected/a05fea52-1297-4bd7-82a2-d863030b675b-kube-api-access-94qmj\") pod \"multus-t9jvk\" (UID: \"a05fea52-1297-4bd7-82a2-d863030b675b\") " pod="openshift-multus/multus-t9jvk" Jul 12 18:17:09.672932 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.672846 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47zmn\" (UniqueName: \"kubernetes.io/projected/aaefba09-f06b-4e39-bc1b-a1b70455dc45-kube-api-access-47zmn\") pod \"aws-ebs-csi-driver-node-pbwgr\" (UID: \"aaefba09-f06b-4e39-bc1b-a1b70455dc45\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pbwgr" Jul 12 18:17:09.674082 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.674056 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whnxw\" (UniqueName: \"kubernetes.io/projected/1b501a87-6ac0-4f0b-9072-aeecd64d5bc0-kube-api-access-whnxw\") pod \"node-ca-khqqs\" (UID: \"1b501a87-6ac0-4f0b-9072-aeecd64d5bc0\") " pod="openshift-image-registry/node-ca-khqqs" Jul 12 18:17:09.675111 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.675085 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbx9l\" (UniqueName: \"kubernetes.io/projected/7d69e388-ddc9-4dd8-b5dc-a81a59dc1698-kube-api-access-vbx9l\") pod \"network-metrics-daemon-r66xp\" (UID: \"7d69e388-ddc9-4dd8-b5dc-a81a59dc1698\") " pod="openshift-multus/network-metrics-daemon-r66xp" Jul 12 18:17:09.675200 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.675089 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z88qx\" (UniqueName: \"kubernetes.io/projected/ed115460-b220-476d-b9d5-65f95ec3d059-kube-api-access-z88qx\") pod \"ovnkube-node-spzqd\" (UID: \"ed115460-b220-476d-b9d5-65f95ec3d059\") " pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.675200 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.675178 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nk5nd\" (UniqueName: \"kubernetes.io/projected/cef566f5-8ee3-4192-bc98-fbcbbb9f29b8-kube-api-access-nk5nd\") pod \"iptables-alerter-7f47s\" (UID: \"cef566f5-8ee3-4192-bc98-fbcbbb9f29b8\") " pod="openshift-network-operator/iptables-alerter-7f47s" Jul 12 18:17:09.675638 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.675618 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmlnv\" (UniqueName: \"kubernetes.io/projected/1a19952f-4732-46da-98ee-05c476b44908-kube-api-access-fmlnv\") pod \"multus-additional-cni-plugins-29hd5\" (UID: \"1a19952f-4732-46da-98ee-05c476b44908\") " pod="openshift-multus/multus-additional-cni-plugins-29hd5" Jul 12 18:17:09.678127 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:09.678110 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 18:17:09.678203 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:09.678130 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 18:17:09.678203 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:09.678141 2577 projected.go:196] Error preparing data for projected volume kube-api-access-tpzmv for pod openshift-network-diagnostics/network-check-target-lfnkm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 18:17:09.678203 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:09.678198 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4d1fc6eb-09af-4f96-8ec8-b193da43354c-kube-api-access-tpzmv podName:4d1fc6eb-09af-4f96-8ec8-b193da43354c nodeName:}" failed. No retries permitted until 2026-07-12 18:17:10.178179904 +0000 UTC m=+3.305084689 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-tpzmv" (UniqueName: "kubernetes.io/projected/4d1fc6eb-09af-4f96-8ec8-b193da43354c-kube-api-access-tpzmv") pod "network-check-target-lfnkm" (UID: "4d1fc6eb-09af-4f96-8ec8-b193da43354c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 18:17:09.679182 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.679161 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6nkz\" (UniqueName: \"kubernetes.io/projected/cd8ab0de-04ec-4aa8-8d33-87aa4bbef597-kube-api-access-d6nkz\") pod \"node-resolver-xwcbp\" (UID: \"cd8ab0de-04ec-4aa8-8d33-87aa4bbef597\") " pod="openshift-dns/node-resolver-xwcbp" Jul 12 18:17:09.680457 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.680440 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rc9w4\" (UniqueName: \"kubernetes.io/projected/a58d0c9c-b6c7-4962-90d1-34d10eb4e97c-kube-api-access-rc9w4\") pod \"tuned-j6pq9\" (UID: \"a58d0c9c-b6c7-4962-90d1-34d10eb4e97c\") " pod="openshift-cluster-node-tuning-operator/tuned-j6pq9" Jul 12 18:17:09.688293 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.688275 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xwcbp" Jul 12 18:17:09.698065 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.698027 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7f47s" Jul 12 18:17:09.768870 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.768781 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e1c57461-73ff-431a-a529-60fa73bf37b5-agent-certs\") pod \"konnectivity-agent-hqhkm\" (UID: \"e1c57461-73ff-431a-a529-60fa73bf37b5\") " pod="kube-system/konnectivity-agent-hqhkm" Jul 12 18:17:09.768870 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.768814 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e1c57461-73ff-431a-a529-60fa73bf37b5-konnectivity-ca\") pod \"konnectivity-agent-hqhkm\" (UID: \"e1c57461-73ff-431a-a529-60fa73bf37b5\") " pod="kube-system/konnectivity-agent-hqhkm" Jul 12 18:17:09.769409 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.769388 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e1c57461-73ff-431a-a529-60fa73bf37b5-konnectivity-ca\") pod \"konnectivity-agent-hqhkm\" (UID: \"e1c57461-73ff-431a-a529-60fa73bf37b5\") " pod="kube-system/konnectivity-agent-hqhkm" Jul 12 18:17:09.771230 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.771202 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e1c57461-73ff-431a-a529-60fa73bf37b5-agent-certs\") pod \"konnectivity-agent-hqhkm\" (UID: \"e1c57461-73ff-431a-a529-60fa73bf37b5\") " pod="kube-system/konnectivity-agent-hqhkm" Jul 12 18:17:09.867754 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.867719 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-khqqs" Jul 12 18:17:09.872892 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.872868 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-29hd5" Jul 12 18:17:09.902745 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.902718 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-t9jvk" Jul 12 18:17:09.921822 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.921791 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:09.938560 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.938533 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pbwgr" Jul 12 18:17:09.955345 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:09.955323 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-j6pq9" Jul 12 18:17:10.003432 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:10.003400 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hqhkm" Jul 12 18:17:10.040622 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:10.040566 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98d2a84da404683679bff0f03b9d595c.slice/crio-c87051ec067b70905084640229ca25535ac1fcb86be68098524b9cd7acf31f4c WatchSource:0}: Error finding container c87051ec067b70905084640229ca25535ac1fcb86be68098524b9cd7acf31f4c: Status 404 returned error can't find the container with id c87051ec067b70905084640229ca25535ac1fcb86be68098524b9cd7acf31f4c Jul 12 18:17:10.042217 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:10.042190 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d5059b2ae6f7bac7125eda1da58ccd7.slice/crio-2b75a26baee830f552731ad1bf334f3baf493a4cfa2fa73122097fd1d096d773 WatchSource:0}: Error finding container 2b75a26baee830f552731ad1bf334f3baf493a4cfa2fa73122097fd1d096d773: Status 404 returned error can't find the container with id 2b75a26baee830f552731ad1bf334f3baf493a4cfa2fa73122097fd1d096d773 Jul 12 18:17:10.047405 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:10.047389 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 18:17:10.072123 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:10.072074 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal" event={"ID":"7d5059b2ae6f7bac7125eda1da58ccd7","Type":"ContainerStarted","Data":"2b75a26baee830f552731ad1bf334f3baf493a4cfa2fa73122097fd1d096d773"} Jul 12 18:17:10.073206 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:10.073185 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-133.ec2.internal" event={"ID":"98d2a84da404683679bff0f03b9d595c","Type":"ContainerStarted","Data":"c87051ec067b70905084640229ca25535ac1fcb86be68098524b9cd7acf31f4c"} Jul 12 18:17:10.098618 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:10.098597 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd8ab0de_04ec_4aa8_8d33_87aa4bbef597.slice/crio-74b7ea89df2efe88cd7c8d848a7af76b67c69d7ed48deb2dc9350d241ddb775d WatchSource:0}: Error finding container 74b7ea89df2efe88cd7c8d848a7af76b67c69d7ed48deb2dc9350d241ddb775d: Status 404 returned error can't find the container with id 74b7ea89df2efe88cd7c8d848a7af76b67c69d7ed48deb2dc9350d241ddb775d Jul 12 18:17:10.119735 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:10.119708 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcef566f5_8ee3_4192_bc98_fbcbbb9f29b8.slice/crio-04a7017d427a16cf7f1d95dd1308eed2b3358a2701ac3f457200ea8ac23fe2a5 WatchSource:0}: Error finding container 04a7017d427a16cf7f1d95dd1308eed2b3358a2701ac3f457200ea8ac23fe2a5: Status 404 returned error can't find the container with id 04a7017d427a16cf7f1d95dd1308eed2b3358a2701ac3f457200ea8ac23fe2a5 Jul 12 18:17:10.171895 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:10.171856 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d69e388-ddc9-4dd8-b5dc-a81a59dc1698-metrics-certs\") pod \"network-metrics-daemon-r66xp\" (UID: \"7d69e388-ddc9-4dd8-b5dc-a81a59dc1698\") " pod="openshift-multus/network-metrics-daemon-r66xp" Jul 12 18:17:10.172044 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:10.171993 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9f8419c6-07be-4876-9762-d98cd772054b-original-pull-secret\") pod \"global-pull-secret-syncer-2598z\" (UID: \"9f8419c6-07be-4876-9762-d98cd772054b\") " pod="kube-system/global-pull-secret-syncer-2598z" Jul 12 18:17:10.172044 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:10.171990 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 18:17:10.172044 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:10.172037 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 18:17:10.172173 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:10.172065 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d69e388-ddc9-4dd8-b5dc-a81a59dc1698-metrics-certs podName:7d69e388-ddc9-4dd8-b5dc-a81a59dc1698 nodeName:}" failed. No retries permitted until 2026-07-12 18:17:11.172045864 +0000 UTC m=+4.298950649 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7d69e388-ddc9-4dd8-b5dc-a81a59dc1698-metrics-certs") pod "network-metrics-daemon-r66xp" (UID: "7d69e388-ddc9-4dd8-b5dc-a81a59dc1698") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 18:17:10.172173 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:10.172089 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f8419c6-07be-4876-9762-d98cd772054b-original-pull-secret podName:9f8419c6-07be-4876-9762-d98cd772054b nodeName:}" failed. No retries permitted until 2026-07-12 18:17:11.172075136 +0000 UTC m=+4.298979920 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9f8419c6-07be-4876-9762-d98cd772054b-original-pull-secret") pod "global-pull-secret-syncer-2598z" (UID: "9f8419c6-07be-4876-9762-d98cd772054b") : object "kube-system"/"original-pull-secret" not registered Jul 12 18:17:10.199535 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:10.199510 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a19952f_4732_46da_98ee_05c476b44908.slice/crio-a9aec8dcd40ec91496c31acc09c164b32eb4c4bbc9341aa45e699b43b30fc064 WatchSource:0}: Error finding container a9aec8dcd40ec91496c31acc09c164b32eb4c4bbc9341aa45e699b43b30fc064: Status 404 returned error can't find the container with id a9aec8dcd40ec91496c31acc09c164b32eb4c4bbc9341aa45e699b43b30fc064 Jul 12 18:17:10.216837 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:10.216811 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b501a87_6ac0_4f0b_9072_aeecd64d5bc0.slice/crio-1b0b0d4ceb146ec8a8771f08e7853026405eafae266960d16f14d2fd2f3d726f WatchSource:0}: Error finding container 1b0b0d4ceb146ec8a8771f08e7853026405eafae266960d16f14d2fd2f3d726f: Status 404 returned error can't find the container with id 1b0b0d4ceb146ec8a8771f08e7853026405eafae266960d16f14d2fd2f3d726f Jul 12 18:17:10.242047 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:10.242023 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded115460_b220_476d_b9d5_65f95ec3d059.slice/crio-0b8adf8ca1a95db3f6a57bf522eeef9d2763e9f67d240a8670f83601d0d8bb63 WatchSource:0}: Error finding container 0b8adf8ca1a95db3f6a57bf522eeef9d2763e9f67d240a8670f83601d0d8bb63: Status 404 returned error can't find the container with id 0b8adf8ca1a95db3f6a57bf522eeef9d2763e9f67d240a8670f83601d0d8bb63 Jul 12 18:17:10.263312 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:10.263285 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda05fea52_1297_4bd7_82a2_d863030b675b.slice/crio-e144ab158bd749b54d5aff6b3bf1abcba7965738bdc7bee828e2fd2a0b6cfb8e WatchSource:0}: Error finding container e144ab158bd749b54d5aff6b3bf1abcba7965738bdc7bee828e2fd2a0b6cfb8e: Status 404 returned error can't find the container with id e144ab158bd749b54d5aff6b3bf1abcba7965738bdc7bee828e2fd2a0b6cfb8e Jul 12 18:17:10.272958 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:10.272933 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpzmv\" (UniqueName: \"kubernetes.io/projected/4d1fc6eb-09af-4f96-8ec8-b193da43354c-kube-api-access-tpzmv\") pod \"network-check-target-lfnkm\" (UID: \"4d1fc6eb-09af-4f96-8ec8-b193da43354c\") " pod="openshift-network-diagnostics/network-check-target-lfnkm" Jul 12 18:17:10.273081 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:10.273068 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 18:17:10.273128 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:10.273085 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 18:17:10.273128 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:10.273095 2577 projected.go:196] Error preparing data for projected volume kube-api-access-tpzmv for pod openshift-network-diagnostics/network-check-target-lfnkm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 18:17:10.273208 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:10.273139 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4d1fc6eb-09af-4f96-8ec8-b193da43354c-kube-api-access-tpzmv podName:4d1fc6eb-09af-4f96-8ec8-b193da43354c nodeName:}" failed. No retries permitted until 2026-07-12 18:17:11.273125959 +0000 UTC m=+4.400030743 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-tpzmv" (UniqueName: "kubernetes.io/projected/4d1fc6eb-09af-4f96-8ec8-b193da43354c-kube-api-access-tpzmv") pod "network-check-target-lfnkm" (UID: "4d1fc6eb-09af-4f96-8ec8-b193da43354c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 18:17:10.337995 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:10.337971 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaaefba09_f06b_4e39_bc1b_a1b70455dc45.slice/crio-a11ebfe32d95b8f4677276f3e2d8ed784f3a4e01f8d35291d62cef129b69598e WatchSource:0}: Error finding container a11ebfe32d95b8f4677276f3e2d8ed784f3a4e01f8d35291d62cef129b69598e: Status 404 returned error can't find the container with id a11ebfe32d95b8f4677276f3e2d8ed784f3a4e01f8d35291d62cef129b69598e Jul 12 18:17:10.417899 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:10.417866 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda58d0c9c_b6c7_4962_90d1_34d10eb4e97c.slice/crio-f45adf06a9dff3382ae8a76041925e26eef5d038b79fb519c81e60904e930193 WatchSource:0}: Error finding container f45adf06a9dff3382ae8a76041925e26eef5d038b79fb519c81e60904e930193: Status 404 returned error can't find the container with id f45adf06a9dff3382ae8a76041925e26eef5d038b79fb519c81e60904e930193 Jul 12 18:17:10.446338 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:10.446274 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1c57461_73ff_431a_a529_60fa73bf37b5.slice/crio-8594546543cc6870bfbfb11c41e6012840f3c738eee577346bbdfccd319a850e WatchSource:0}: Error finding container 8594546543cc6870bfbfb11c41e6012840f3c738eee577346bbdfccd319a850e: Status 404 returned error can't find the container with id 8594546543cc6870bfbfb11c41e6012840f3c738eee577346bbdfccd319a850e Jul 12 18:17:10.474342 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:10.474248 2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 18:17:10.598315 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:10.598118 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-11 18:12:09 +0000 UTC" deadline="2028-03-12 10:39:17.588331161 +0000 UTC" Jul 12 18:17:10.598315 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:10.598170 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14608h22m6.990165482s" Jul 12 18:17:10.831429 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:10.831387 2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 18:17:11.071599 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:11.070791 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2598z" Jul 12 18:17:11.071599 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:11.070918 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2598z" podUID="9f8419c6-07be-4876-9762-d98cd772054b" Jul 12 18:17:11.071599 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:11.071323 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lfnkm" Jul 12 18:17:11.071599 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:11.071413 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lfnkm" podUID="4d1fc6eb-09af-4f96-8ec8-b193da43354c" Jul 12 18:17:11.071599 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:11.071472 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r66xp" Jul 12 18:17:11.071599 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:11.071558 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r66xp" podUID="7d69e388-ddc9-4dd8-b5dc-a81a59dc1698" Jul 12 18:17:11.101893 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:11.101775 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hqhkm" event={"ID":"e1c57461-73ff-431a-a529-60fa73bf37b5","Type":"ContainerStarted","Data":"8594546543cc6870bfbfb11c41e6012840f3c738eee577346bbdfccd319a850e"} Jul 12 18:17:11.136145 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:11.136007 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-j6pq9" event={"ID":"a58d0c9c-b6c7-4962-90d1-34d10eb4e97c","Type":"ContainerStarted","Data":"f45adf06a9dff3382ae8a76041925e26eef5d038b79fb519c81e60904e930193"} Jul 12 18:17:11.164024 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:11.163971 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pbwgr" event={"ID":"aaefba09-f06b-4e39-bc1b-a1b70455dc45","Type":"ContainerStarted","Data":"a11ebfe32d95b8f4677276f3e2d8ed784f3a4e01f8d35291d62cef129b69598e"} Jul 12 18:17:11.180493 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:11.180449 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d69e388-ddc9-4dd8-b5dc-a81a59dc1698-metrics-certs\") pod \"network-metrics-daemon-r66xp\" (UID: \"7d69e388-ddc9-4dd8-b5dc-a81a59dc1698\") " pod="openshift-multus/network-metrics-daemon-r66xp" Jul 12 18:17:11.180685 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:11.180539 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9f8419c6-07be-4876-9762-d98cd772054b-original-pull-secret\") pod \"global-pull-secret-syncer-2598z\" (UID: \"9f8419c6-07be-4876-9762-d98cd772054b\") " pod="kube-system/global-pull-secret-syncer-2598z" Jul 12 18:17:11.180808 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:11.180728 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 18:17:11.180808 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:11.180798 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f8419c6-07be-4876-9762-d98cd772054b-original-pull-secret podName:9f8419c6-07be-4876-9762-d98cd772054b nodeName:}" failed. No retries permitted until 2026-07-12 18:17:13.180776675 +0000 UTC m=+6.307681464 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9f8419c6-07be-4876-9762-d98cd772054b-original-pull-secret") pod "global-pull-secret-syncer-2598z" (UID: "9f8419c6-07be-4876-9762-d98cd772054b") : object "kube-system"/"original-pull-secret" not registered Jul 12 18:17:11.181289 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:11.181267 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 18:17:11.181409 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:11.181366 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d69e388-ddc9-4dd8-b5dc-a81a59dc1698-metrics-certs podName:7d69e388-ddc9-4dd8-b5dc-a81a59dc1698 nodeName:}" failed. No retries permitted until 2026-07-12 18:17:13.18134781 +0000 UTC m=+6.308252598 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7d69e388-ddc9-4dd8-b5dc-a81a59dc1698-metrics-certs") pod "network-metrics-daemon-r66xp" (UID: "7d69e388-ddc9-4dd8-b5dc-a81a59dc1698") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 18:17:11.192925 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:11.192852 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-khqqs" event={"ID":"1b501a87-6ac0-4f0b-9072-aeecd64d5bc0","Type":"ContainerStarted","Data":"1b0b0d4ceb146ec8a8771f08e7853026405eafae266960d16f14d2fd2f3d726f"} Jul 12 18:17:11.216409 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:11.216321 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-29hd5" event={"ID":"1a19952f-4732-46da-98ee-05c476b44908","Type":"ContainerStarted","Data":"a9aec8dcd40ec91496c31acc09c164b32eb4c4bbc9341aa45e699b43b30fc064"} Jul 12 18:17:11.222622 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:11.222585 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xwcbp" event={"ID":"cd8ab0de-04ec-4aa8-8d33-87aa4bbef597","Type":"ContainerStarted","Data":"74b7ea89df2efe88cd7c8d848a7af76b67c69d7ed48deb2dc9350d241ddb775d"} Jul 12 18:17:11.226596 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:11.226562 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-t9jvk" event={"ID":"a05fea52-1297-4bd7-82a2-d863030b675b","Type":"ContainerStarted","Data":"e144ab158bd749b54d5aff6b3bf1abcba7965738bdc7bee828e2fd2a0b6cfb8e"} Jul 12 18:17:11.237638 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:11.237602 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" event={"ID":"ed115460-b220-476d-b9d5-65f95ec3d059","Type":"ContainerStarted","Data":"0b8adf8ca1a95db3f6a57bf522eeef9d2763e9f67d240a8670f83601d0d8bb63"} Jul 12 18:17:11.247563 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:11.247525 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7f47s" event={"ID":"cef566f5-8ee3-4192-bc98-fbcbbb9f29b8","Type":"ContainerStarted","Data":"04a7017d427a16cf7f1d95dd1308eed2b3358a2701ac3f457200ea8ac23fe2a5"} Jul 12 18:17:11.281113 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:11.281075 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpzmv\" (UniqueName: \"kubernetes.io/projected/4d1fc6eb-09af-4f96-8ec8-b193da43354c-kube-api-access-tpzmv\") pod \"network-check-target-lfnkm\" (UID: \"4d1fc6eb-09af-4f96-8ec8-b193da43354c\") " pod="openshift-network-diagnostics/network-check-target-lfnkm" Jul 12 18:17:11.281302 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:11.281284 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 18:17:11.281373 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:11.281308 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 18:17:11.281373 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:11.281323 2577 projected.go:196] Error preparing data for projected volume kube-api-access-tpzmv for pod openshift-network-diagnostics/network-check-target-lfnkm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 18:17:11.281480 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:11.281385 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4d1fc6eb-09af-4f96-8ec8-b193da43354c-kube-api-access-tpzmv podName:4d1fc6eb-09af-4f96-8ec8-b193da43354c nodeName:}" failed. No retries permitted until 2026-07-12 18:17:13.281364432 +0000 UTC m=+6.408269220 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-tpzmv" (UniqueName: "kubernetes.io/projected/4d1fc6eb-09af-4f96-8ec8-b193da43354c-kube-api-access-tpzmv") pod "network-check-target-lfnkm" (UID: "4d1fc6eb-09af-4f96-8ec8-b193da43354c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 18:17:11.599230 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:11.599080 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-11 18:12:09 +0000 UTC" deadline="2028-01-17 18:27:12.615216406 +0000 UTC" Jul 12 18:17:11.599230 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:11.599147 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13296h10m1.016074046s" Jul 12 18:17:12.411118 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:12.411084 2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 18:17:13.072585 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:13.071411 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r66xp" Jul 12 18:17:13.072585 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:13.071636 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r66xp" podUID="7d69e388-ddc9-4dd8-b5dc-a81a59dc1698" Jul 12 18:17:13.072585 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:13.072055 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lfnkm" Jul 12 18:17:13.072585 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:13.072178 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lfnkm" podUID="4d1fc6eb-09af-4f96-8ec8-b193da43354c" Jul 12 18:17:13.072585 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:13.072266 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2598z" Jul 12 18:17:13.072585 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:13.072362 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2598z" podUID="9f8419c6-07be-4876-9762-d98cd772054b" Jul 12 18:17:13.201980 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:13.201060 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9f8419c6-07be-4876-9762-d98cd772054b-original-pull-secret\") pod \"global-pull-secret-syncer-2598z\" (UID: \"9f8419c6-07be-4876-9762-d98cd772054b\") " pod="kube-system/global-pull-secret-syncer-2598z" Jul 12 18:17:13.201980 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:13.201148 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d69e388-ddc9-4dd8-b5dc-a81a59dc1698-metrics-certs\") pod \"network-metrics-daemon-r66xp\" (UID: \"7d69e388-ddc9-4dd8-b5dc-a81a59dc1698\") " pod="openshift-multus/network-metrics-daemon-r66xp" Jul 12 18:17:13.201980 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:13.201315 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 18:17:13.201980 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:13.201383 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d69e388-ddc9-4dd8-b5dc-a81a59dc1698-metrics-certs podName:7d69e388-ddc9-4dd8-b5dc-a81a59dc1698 nodeName:}" failed. No retries permitted until 2026-07-12 18:17:17.201361577 +0000 UTC m=+10.328266363 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7d69e388-ddc9-4dd8-b5dc-a81a59dc1698-metrics-certs") pod "network-metrics-daemon-r66xp" (UID: "7d69e388-ddc9-4dd8-b5dc-a81a59dc1698") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 18:17:13.201980 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:13.201813 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 18:17:13.201980 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:13.201901 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f8419c6-07be-4876-9762-d98cd772054b-original-pull-secret podName:9f8419c6-07be-4876-9762-d98cd772054b nodeName:}" failed. No retries permitted until 2026-07-12 18:17:17.201882217 +0000 UTC m=+10.328787020 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9f8419c6-07be-4876-9762-d98cd772054b-original-pull-secret") pod "global-pull-secret-syncer-2598z" (UID: "9f8419c6-07be-4876-9762-d98cd772054b") : object "kube-system"/"original-pull-secret" not registered Jul 12 18:17:13.302251 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:13.301493 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpzmv\" (UniqueName: \"kubernetes.io/projected/4d1fc6eb-09af-4f96-8ec8-b193da43354c-kube-api-access-tpzmv\") pod \"network-check-target-lfnkm\" (UID: \"4d1fc6eb-09af-4f96-8ec8-b193da43354c\") " pod="openshift-network-diagnostics/network-check-target-lfnkm" Jul 12 18:17:13.302251 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:13.301767 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 18:17:13.302251 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:13.301791 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 18:17:13.302251 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:13.301807 2577 projected.go:196] Error preparing data for projected volume kube-api-access-tpzmv for pod openshift-network-diagnostics/network-check-target-lfnkm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 18:17:13.302251 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:13.301869 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4d1fc6eb-09af-4f96-8ec8-b193da43354c-kube-api-access-tpzmv podName:4d1fc6eb-09af-4f96-8ec8-b193da43354c nodeName:}" failed. No retries permitted until 2026-07-12 18:17:17.301849388 +0000 UTC m=+10.428754176 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-tpzmv" (UniqueName: "kubernetes.io/projected/4d1fc6eb-09af-4f96-8ec8-b193da43354c-kube-api-access-tpzmv") pod "network-check-target-lfnkm" (UID: "4d1fc6eb-09af-4f96-8ec8-b193da43354c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 18:17:15.069932 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:15.069900 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r66xp" Jul 12 18:17:15.070371 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:15.070052 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r66xp" podUID="7d69e388-ddc9-4dd8-b5dc-a81a59dc1698" Jul 12 18:17:15.070371 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:15.070173 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lfnkm" Jul 12 18:17:15.070371 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:15.070265 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lfnkm" podUID="4d1fc6eb-09af-4f96-8ec8-b193da43354c" Jul 12 18:17:15.070371 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:15.070309 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2598z" Jul 12 18:17:15.070577 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:15.070441 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2598z" podUID="9f8419c6-07be-4876-9762-d98cd772054b" Jul 12 18:17:17.070659 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:17.070595 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lfnkm" Jul 12 18:17:17.071130 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:17.070595 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r66xp" Jul 12 18:17:17.071130 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:17.070740 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lfnkm" podUID="4d1fc6eb-09af-4f96-8ec8-b193da43354c" Jul 12 18:17:17.071130 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:17.070745 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2598z" Jul 12 18:17:17.071130 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:17.070860 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r66xp" podUID="7d69e388-ddc9-4dd8-b5dc-a81a59dc1698" Jul 12 18:17:17.071130 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:17.070958 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2598z" podUID="9f8419c6-07be-4876-9762-d98cd772054b" Jul 12 18:17:17.237958 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:17.237897 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d69e388-ddc9-4dd8-b5dc-a81a59dc1698-metrics-certs\") pod \"network-metrics-daemon-r66xp\" (UID: \"7d69e388-ddc9-4dd8-b5dc-a81a59dc1698\") " pod="openshift-multus/network-metrics-daemon-r66xp" Jul 12 18:17:17.238152 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:17.237979 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9f8419c6-07be-4876-9762-d98cd772054b-original-pull-secret\") pod \"global-pull-secret-syncer-2598z\" (UID: \"9f8419c6-07be-4876-9762-d98cd772054b\") " pod="kube-system/global-pull-secret-syncer-2598z" Jul 12 18:17:17.238152 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:17.238098 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 18:17:17.238269 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:17.238157 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f8419c6-07be-4876-9762-d98cd772054b-original-pull-secret podName:9f8419c6-07be-4876-9762-d98cd772054b nodeName:}" failed. No retries permitted until 2026-07-12 18:17:25.238139084 +0000 UTC m=+18.365043869 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9f8419c6-07be-4876-9762-d98cd772054b-original-pull-secret") pod "global-pull-secret-syncer-2598z" (UID: "9f8419c6-07be-4876-9762-d98cd772054b") : object "kube-system"/"original-pull-secret" not registered Jul 12 18:17:17.238511 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:17.238432 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 18:17:17.238511 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:17.238494 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d69e388-ddc9-4dd8-b5dc-a81a59dc1698-metrics-certs podName:7d69e388-ddc9-4dd8-b5dc-a81a59dc1698 nodeName:}" failed. No retries permitted until 2026-07-12 18:17:25.238477201 +0000 UTC m=+18.365381998 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7d69e388-ddc9-4dd8-b5dc-a81a59dc1698-metrics-certs") pod "network-metrics-daemon-r66xp" (UID: "7d69e388-ddc9-4dd8-b5dc-a81a59dc1698") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 18:17:17.338676 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:17.338578 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpzmv\" (UniqueName: \"kubernetes.io/projected/4d1fc6eb-09af-4f96-8ec8-b193da43354c-kube-api-access-tpzmv\") pod \"network-check-target-lfnkm\" (UID: \"4d1fc6eb-09af-4f96-8ec8-b193da43354c\") " pod="openshift-network-diagnostics/network-check-target-lfnkm" Jul 12 18:17:17.338833 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:17.338770 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 18:17:17.338833 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:17.338799 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 18:17:17.338833 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:17.338811 2577 projected.go:196] Error preparing data for projected volume kube-api-access-tpzmv for pod openshift-network-diagnostics/network-check-target-lfnkm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 18:17:17.339002 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:17.338864 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4d1fc6eb-09af-4f96-8ec8-b193da43354c-kube-api-access-tpzmv podName:4d1fc6eb-09af-4f96-8ec8-b193da43354c nodeName:}" failed. No retries permitted until 2026-07-12 18:17:25.338847418 +0000 UTC m=+18.465752211 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-tpzmv" (UniqueName: "kubernetes.io/projected/4d1fc6eb-09af-4f96-8ec8-b193da43354c-kube-api-access-tpzmv") pod "network-check-target-lfnkm" (UID: "4d1fc6eb-09af-4f96-8ec8-b193da43354c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 18:17:19.071358 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:19.071326 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r66xp" Jul 12 18:17:19.071843 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:19.071449 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r66xp" podUID="7d69e388-ddc9-4dd8-b5dc-a81a59dc1698" Jul 12 18:17:19.071843 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:19.071503 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lfnkm" Jul 12 18:17:19.071843 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:19.071567 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lfnkm" podUID="4d1fc6eb-09af-4f96-8ec8-b193da43354c" Jul 12 18:17:19.072006 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:19.071924 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2598z" Jul 12 18:17:19.072060 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:19.072012 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2598z" podUID="9f8419c6-07be-4876-9762-d98cd772054b" Jul 12 18:17:21.070343 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:21.070305 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r66xp" Jul 12 18:17:21.070796 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:21.070308 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lfnkm" Jul 12 18:17:21.070796 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:21.070458 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r66xp" podUID="7d69e388-ddc9-4dd8-b5dc-a81a59dc1698" Jul 12 18:17:21.070796 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:21.070470 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2598z" Jul 12 18:17:21.070796 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:21.070572 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lfnkm" podUID="4d1fc6eb-09af-4f96-8ec8-b193da43354c" Jul 12 18:17:21.070796 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:21.070680 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2598z" podUID="9f8419c6-07be-4876-9762-d98cd772054b" Jul 12 18:17:23.070151 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:23.070111 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r66xp" Jul 12 18:17:23.070618 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:23.070117 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lfnkm" Jul 12 18:17:23.070618 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:23.070260 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r66xp" podUID="7d69e388-ddc9-4dd8-b5dc-a81a59dc1698" Jul 12 18:17:23.070618 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:23.070127 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2598z" Jul 12 18:17:23.070618 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:23.070283 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lfnkm" podUID="4d1fc6eb-09af-4f96-8ec8-b193da43354c" Jul 12 18:17:23.070618 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:23.070399 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2598z" podUID="9f8419c6-07be-4876-9762-d98cd772054b" Jul 12 18:17:25.069838 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:25.069760 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r66xp" Jul 12 18:17:25.069838 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:25.069820 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2598z" Jul 12 18:17:25.070360 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:25.069760 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lfnkm" Jul 12 18:17:25.070360 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:25.069907 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r66xp" podUID="7d69e388-ddc9-4dd8-b5dc-a81a59dc1698" Jul 12 18:17:25.070360 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:25.069976 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lfnkm" podUID="4d1fc6eb-09af-4f96-8ec8-b193da43354c" Jul 12 18:17:25.070360 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:25.070063 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2598z" podUID="9f8419c6-07be-4876-9762-d98cd772054b" Jul 12 18:17:25.301828 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:25.301792 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d69e388-ddc9-4dd8-b5dc-a81a59dc1698-metrics-certs\") pod \"network-metrics-daemon-r66xp\" (UID: \"7d69e388-ddc9-4dd8-b5dc-a81a59dc1698\") " pod="openshift-multus/network-metrics-daemon-r66xp" Jul 12 18:17:25.302004 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:25.301861 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9f8419c6-07be-4876-9762-d98cd772054b-original-pull-secret\") pod \"global-pull-secret-syncer-2598z\" (UID: \"9f8419c6-07be-4876-9762-d98cd772054b\") " pod="kube-system/global-pull-secret-syncer-2598z" Jul 12 18:17:25.302004 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:25.301958 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 18:17:25.302122 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:25.302024 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d69e388-ddc9-4dd8-b5dc-a81a59dc1698-metrics-certs podName:7d69e388-ddc9-4dd8-b5dc-a81a59dc1698 nodeName:}" failed. No retries permitted until 2026-07-12 18:17:41.302008359 +0000 UTC m=+34.428913147 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7d69e388-ddc9-4dd8-b5dc-a81a59dc1698-metrics-certs") pod "network-metrics-daemon-r66xp" (UID: "7d69e388-ddc9-4dd8-b5dc-a81a59dc1698") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 18:17:25.302122 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:25.301966 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 18:17:25.302122 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:25.302114 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f8419c6-07be-4876-9762-d98cd772054b-original-pull-secret podName:9f8419c6-07be-4876-9762-d98cd772054b nodeName:}" failed. No retries permitted until 2026-07-12 18:17:41.302096353 +0000 UTC m=+34.429001149 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9f8419c6-07be-4876-9762-d98cd772054b-original-pull-secret") pod "global-pull-secret-syncer-2598z" (UID: "9f8419c6-07be-4876-9762-d98cd772054b") : object "kube-system"/"original-pull-secret" not registered Jul 12 18:17:25.402780 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:25.402697 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpzmv\" (UniqueName: \"kubernetes.io/projected/4d1fc6eb-09af-4f96-8ec8-b193da43354c-kube-api-access-tpzmv\") pod \"network-check-target-lfnkm\" (UID: \"4d1fc6eb-09af-4f96-8ec8-b193da43354c\") " pod="openshift-network-diagnostics/network-check-target-lfnkm" Jul 12 18:17:25.402920 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:25.402855 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 18:17:25.402920 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:25.402875 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 18:17:25.402920 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:25.402886 2577 projected.go:196] Error preparing data for projected volume kube-api-access-tpzmv for pod openshift-network-diagnostics/network-check-target-lfnkm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 18:17:25.403072 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:25.402932 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4d1fc6eb-09af-4f96-8ec8-b193da43354c-kube-api-access-tpzmv podName:4d1fc6eb-09af-4f96-8ec8-b193da43354c nodeName:}" failed. No retries permitted until 2026-07-12 18:17:41.402919252 +0000 UTC m=+34.529824035 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-tpzmv" (UniqueName: "kubernetes.io/projected/4d1fc6eb-09af-4f96-8ec8-b193da43354c-kube-api-access-tpzmv") pod "network-check-target-lfnkm" (UID: "4d1fc6eb-09af-4f96-8ec8-b193da43354c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 18:17:27.069834 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:27.069794 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r66xp" Jul 12 18:17:27.069834 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:27.069818 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2598z" Jul 12 18:17:27.070341 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:27.069795 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lfnkm" Jul 12 18:17:27.070341 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:27.069929 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r66xp" podUID="7d69e388-ddc9-4dd8-b5dc-a81a59dc1698" Jul 12 18:17:27.070341 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:27.070002 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lfnkm" podUID="4d1fc6eb-09af-4f96-8ec8-b193da43354c" Jul 12 18:17:27.070341 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:27.070077 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2598z" podUID="9f8419c6-07be-4876-9762-d98cd772054b" Jul 12 18:17:29.071350 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:29.071118 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r66xp" Jul 12 18:17:29.071804 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:29.071197 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lfnkm" Jul 12 18:17:29.071804 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:29.071487 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r66xp" podUID="7d69e388-ddc9-4dd8-b5dc-a81a59dc1698" Jul 12 18:17:29.071804 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:29.071226 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2598z" Jul 12 18:17:29.071804 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:29.071590 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lfnkm" podUID="4d1fc6eb-09af-4f96-8ec8-b193da43354c" Jul 12 18:17:29.071804 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:29.071700 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2598z" podUID="9f8419c6-07be-4876-9762-d98cd772054b" Jul 12 18:17:31.070374 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:31.070303 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lfnkm" Jul 12 18:17:31.070915 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:31.070303 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r66xp" Jul 12 18:17:31.070915 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:31.070399 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lfnkm" podUID="4d1fc6eb-09af-4f96-8ec8-b193da43354c" Jul 12 18:17:31.070915 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:31.070425 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2598z" Jul 12 18:17:31.070915 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:31.070531 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r66xp" podUID="7d69e388-ddc9-4dd8-b5dc-a81a59dc1698" Jul 12 18:17:31.070915 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:31.070618 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2598z" podUID="9f8419c6-07be-4876-9762-d98cd772054b" Jul 12 18:17:31.296890 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:31.296855 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-j6pq9" event={"ID":"a58d0c9c-b6c7-4962-90d1-34d10eb4e97c","Type":"ContainerStarted","Data":"66ce87224844040a3f2eefa6f8c4c83c5cbb0e268ab837f770f3ff87d12334f0"} Jul 12 18:17:31.298422 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:31.298394 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-t9jvk" event={"ID":"a05fea52-1297-4bd7-82a2-d863030b675b","Type":"ContainerStarted","Data":"16896959096df4b348abe05a09b47a528895ee983782f009ce3e9b75f652957f"} Jul 12 18:17:31.304707 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:31.304683 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-133.ec2.internal" event={"ID":"98d2a84da404683679bff0f03b9d595c","Type":"ContainerStarted","Data":"190d572e745f58cfc9162a79b87b3f377e966d7b791aa6dcd2e8ad42f1ec49a4"} Jul 12 18:17:31.315049 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:31.314990 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-j6pq9" podStartSLOduration=2.2740322539999998 podStartE2EDuration="22.314975984s" totalImagesPullingTime="20.04094373s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 18:17:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:17:31.314723492 +0000 UTC m=+24.441628299" watchObservedRunningTime="2026-07-12 18:17:31.314975984 +0000 UTC m=+24.441880789" Jul 12 18:17:31.329769 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:31.329719 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-t9jvk" podStartSLOduration=1.919297717 podStartE2EDuration="22.329698814s" totalImagesPullingTime="20.410401097s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 18:17:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:17:31.329572034 +0000 UTC m=+24.456476840" watchObservedRunningTime="2026-07-12 18:17:31.329698814 +0000 UTC m=+24.456603623" Jul 12 18:17:31.342707 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:31.342665 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-133.ec2.internal" podStartSLOduration=22.342651515 podStartE2EDuration="22.342651515s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 18:17:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:17:31.34226141 +0000 UTC m=+24.469166216" watchObservedRunningTime="2026-07-12 18:17:31.342651515 +0000 UTC m=+24.469556320" Jul 12 18:17:32.309439 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:32.309100 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pbwgr" event={"ID":"aaefba09-f06b-4e39-bc1b-a1b70455dc45","Type":"ContainerStarted","Data":"b57754f7c5a63eeae3b4f8d786eb8229a5dcb2708cce7136e2a06304f0b7f4de"} Jul 12 18:17:32.311014 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:32.310746 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-khqqs" event={"ID":"1b501a87-6ac0-4f0b-9072-aeecd64d5bc0","Type":"ContainerStarted","Data":"100f32be83c1e8fb1d6e05af208cc0049971602eb241685594992ca9a9f951b8"} Jul 12 18:17:32.312427 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:32.312403 2577 generic.go:356] "Generic (PLEG): container finished" podID="1a19952f-4732-46da-98ee-05c476b44908" containerID="e7c47c51e68e248f53ef352105420eee0a0cfed6f18c377ea78421e7af59474e" exitCode=0 Jul 12 18:17:32.312520 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:32.312454 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-29hd5" event={"ID":"1a19952f-4732-46da-98ee-05c476b44908","Type":"ContainerDied","Data":"e7c47c51e68e248f53ef352105420eee0a0cfed6f18c377ea78421e7af59474e"} Jul 12 18:17:32.314147 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:32.314127 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xwcbp" event={"ID":"cd8ab0de-04ec-4aa8-8d33-87aa4bbef597","Type":"ContainerStarted","Data":"ad479ba33d93a51d9e62417bb8de3e335797c10fa1721f921e2ddfaeaf1c4eeb"} Jul 12 18:17:32.315966 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:32.315943 2577 generic.go:356] "Generic (PLEG): container finished" podID="7d5059b2ae6f7bac7125eda1da58ccd7" containerID="4ea1cc495e57c7ac6a6c91fd6b8b1acde523965dba0fd2565b139e2f6cf2d57f" exitCode=0 Jul 12 18:17:32.316075 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:32.316028 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal" event={"ID":"7d5059b2ae6f7bac7125eda1da58ccd7","Type":"ContainerDied","Data":"4ea1cc495e57c7ac6a6c91fd6b8b1acde523965dba0fd2565b139e2f6cf2d57f"} Jul 12 18:17:32.319823 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:32.319801 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-spzqd_ed115460-b220-476d-b9d5-65f95ec3d059/ovn-acl-logging/0.log" Jul 12 18:17:32.320199 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:32.320181 2577 generic.go:356] "Generic (PLEG): container finished" podID="ed115460-b220-476d-b9d5-65f95ec3d059" containerID="1a0de48e904e2b9755bf8ef27faa8b76fda646d38b45ace90dd43f3468ebf926" exitCode=1 Jul 12 18:17:32.320273 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:32.320245 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" event={"ID":"ed115460-b220-476d-b9d5-65f95ec3d059","Type":"ContainerStarted","Data":"8eb422fdf92359c8d37de2c9c7021c21c69d5f721963fc9d21bc6093744475a5"} Jul 12 18:17:32.320329 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:32.320318 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" event={"ID":"ed115460-b220-476d-b9d5-65f95ec3d059","Type":"ContainerStarted","Data":"528045fd471a0bb0a5daa803d3c1b96822d646afd4b77a37ddb6a5fbe8fa5dd4"} Jul 12 18:17:32.320388 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:32.320368 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" event={"ID":"ed115460-b220-476d-b9d5-65f95ec3d059","Type":"ContainerStarted","Data":"899e85d7d89ffff6559cec29dd548d444a401efffc1221b4ecd169f508e4d921"} Jul 12 18:17:32.320432 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:32.320414 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" event={"ID":"ed115460-b220-476d-b9d5-65f95ec3d059","Type":"ContainerStarted","Data":"960984ceab1bdfd8dfd5722a03e49c3aabe4386a0c8689689fce1b0a468a5f24"} Jul 12 18:17:32.320472 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:32.320460 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" event={"ID":"ed115460-b220-476d-b9d5-65f95ec3d059","Type":"ContainerDied","Data":"1a0de48e904e2b9755bf8ef27faa8b76fda646d38b45ace90dd43f3468ebf926"} Jul 12 18:17:32.320528 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:32.320512 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" event={"ID":"ed115460-b220-476d-b9d5-65f95ec3d059","Type":"ContainerStarted","Data":"16aace3ea7ce4e664705fdb0865366dd2aa1466e8ac2313bf0a6e54c70c4cd4f"} Jul 12 18:17:32.321765 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:32.321735 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7f47s" event={"ID":"cef566f5-8ee3-4192-bc98-fbcbbb9f29b8","Type":"ContainerStarted","Data":"469a6084a3f063ea5f3dc8d8b70d28d268139f7981264fd170a9274c8f4a5ed2"} Jul 12 18:17:32.323279 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:32.323201 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hqhkm" event={"ID":"e1c57461-73ff-431a-a529-60fa73bf37b5","Type":"ContainerStarted","Data":"5481c62962a40ff3a6912739638ada04efb03bf89eb6c518db5d29b2e0688f27"} Jul 12 18:17:32.323577 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:32.323535 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-khqqs" podStartSLOduration=7.663137675 podStartE2EDuration="23.323521106s" totalImagesPullingTime="15.660383431s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 18:17:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:17:32.323488201 +0000 UTC m=+25.450393006" watchObservedRunningTime="2026-07-12 18:17:32.323521106 +0000 UTC m=+25.450425911" Jul 12 18:17:32.336359 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:32.336316 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-7f47s" podStartSLOduration=3.000134344 podStartE2EDuration="23.336302371s" totalImagesPullingTime="20.336168027s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 18:17:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:17:32.336036073 +0000 UTC m=+25.462940880" watchObservedRunningTime="2026-07-12 18:17:32.336302371 +0000 UTC m=+25.463207177" Jul 12 18:17:32.349610 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:32.349570 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-hqhkm" podStartSLOduration=3.340812378 podStartE2EDuration="23.349559372s" totalImagesPullingTime="20.008746994s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:17:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:17:32.348847436 +0000 UTC m=+25.475752242" watchObservedRunningTime="2026-07-12 18:17:32.349559372 +0000 UTC m=+25.476464177" Jul 12 18:17:32.423223 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:32.423178 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-xwcbp" podStartSLOduration=3.064917124 podStartE2EDuration="23.423162784s" totalImagesPullingTime="20.35824566s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 18:17:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:17:32.423161927 +0000 UTC m=+25.550066731" watchObservedRunningTime="2026-07-12 18:17:32.423162784 +0000 UTC m=+25.550067589" Jul 12 18:17:32.809864 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:32.809838 2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 12 18:17:32.993298 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:32.993155 2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-12T18:17:32.809861953Z","UUID":"4bfb528c-aafe-4d17-90f3-5649d6e68eb3","Handler":null,"Name":"","Endpoint":""} Jul 12 18:17:32.994582 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:32.994565 2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 12 18:17:32.994680 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:32.994596 2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 12 18:17:33.070207 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:33.070175 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r66xp" Jul 12 18:17:33.070392 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:33.070184 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lfnkm" Jul 12 18:17:33.070392 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:33.070345 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2598z" Jul 12 18:17:33.070490 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:33.070308 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r66xp" podUID="7d69e388-ddc9-4dd8-b5dc-a81a59dc1698" Jul 12 18:17:33.070490 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:33.070420 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lfnkm" podUID="4d1fc6eb-09af-4f96-8ec8-b193da43354c" Jul 12 18:17:33.070584 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:33.070496 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2598z" podUID="9f8419c6-07be-4876-9762-d98cd772054b" Jul 12 18:17:33.327596 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:33.327334 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal" event={"ID":"7d5059b2ae6f7bac7125eda1da58ccd7","Type":"ContainerStarted","Data":"fab2ecb95d220dec26b04b555698f0841b4a8cc250f9489814d02850ddf8979b"} Jul 12 18:17:33.329667 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:33.329621 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pbwgr" event={"ID":"aaefba09-f06b-4e39-bc1b-a1b70455dc45","Type":"ContainerStarted","Data":"ae660cd1ac45069349bf95df6caa41257e4da1c69850d395f2436819662b6de2"} Jul 12 18:17:33.342470 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:33.342422 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal" podStartSLOduration=24.342406236 podStartE2EDuration="24.342406236s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 18:17:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:17:33.341306694 +0000 UTC m=+26.468211504" watchObservedRunningTime="2026-07-12 18:17:33.342406236 +0000 UTC m=+26.469311045" Jul 12 18:17:34.139362 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:34.139147 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-hqhkm" Jul 12 18:17:34.139901 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:34.139878 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-hqhkm" Jul 12 18:17:34.334186 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:34.333996 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-spzqd_ed115460-b220-476d-b9d5-65f95ec3d059/ovn-acl-logging/0.log" Jul 12 18:17:34.334542 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:34.334497 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" event={"ID":"ed115460-b220-476d-b9d5-65f95ec3d059","Type":"ContainerStarted","Data":"c1e0c7845f07c0df60d6aacd33f414df559dcd3a85984b8656739d15a28c2733"} Jul 12 18:17:34.336443 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:34.336418 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pbwgr" event={"ID":"aaefba09-f06b-4e39-bc1b-a1b70455dc45","Type":"ContainerStarted","Data":"d702378e5f226d9b574114c7942535a697f59541e8dde8ebc8e7c164d0f9f0c8"} Jul 12 18:17:34.355314 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:34.355270 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pbwgr" podStartSLOduration=2.961498981 podStartE2EDuration="25.355256718s" totalImagesPullingTime="22.393757737s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 18:17:09 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:17:34.354862523 +0000 UTC m=+27.481767331" watchObservedRunningTime="2026-07-12 18:17:34.355256718 +0000 UTC m=+27.482161523" Jul 12 18:17:35.070176 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:35.070139 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lfnkm" Jul 12 18:17:35.070362 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:35.070139 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2598z" Jul 12 18:17:35.070362 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:35.070252 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lfnkm" podUID="4d1fc6eb-09af-4f96-8ec8-b193da43354c" Jul 12 18:17:35.070362 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:35.070157 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r66xp" Jul 12 18:17:35.070532 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:35.070355 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2598z" podUID="9f8419c6-07be-4876-9762-d98cd772054b" Jul 12 18:17:35.070532 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:35.070443 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r66xp" podUID="7d69e388-ddc9-4dd8-b5dc-a81a59dc1698" Jul 12 18:17:35.338652 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:35.338562 2577 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 12 18:17:37.070339 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:37.070154 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2598z" Jul 12 18:17:37.071134 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:37.070218 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lfnkm" Jul 12 18:17:37.071134 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:37.070442 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2598z" podUID="9f8419c6-07be-4876-9762-d98cd772054b" Jul 12 18:17:37.071134 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:37.070154 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r66xp" Jul 12 18:17:37.071134 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:37.070518 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lfnkm" podUID="4d1fc6eb-09af-4f96-8ec8-b193da43354c" Jul 12 18:17:37.071134 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:37.070590 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r66xp" podUID="7d69e388-ddc9-4dd8-b5dc-a81a59dc1698" Jul 12 18:17:37.344012 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:37.343982 2577 generic.go:356] "Generic (PLEG): container finished" podID="1a19952f-4732-46da-98ee-05c476b44908" containerID="ee21e71305019210b3ca7d29c029adab954189ec594a9e9d14a96e6f501a05a6" exitCode=0 Jul 12 18:17:37.344193 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:37.344060 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-29hd5" event={"ID":"1a19952f-4732-46da-98ee-05c476b44908","Type":"ContainerDied","Data":"ee21e71305019210b3ca7d29c029adab954189ec594a9e9d14a96e6f501a05a6"} Jul 12 18:17:37.347496 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:37.347478 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-spzqd_ed115460-b220-476d-b9d5-65f95ec3d059/ovn-acl-logging/0.log" Jul 12 18:17:37.347847 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:37.347822 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" event={"ID":"ed115460-b220-476d-b9d5-65f95ec3d059","Type":"ContainerStarted","Data":"3c777b8561408d768327683b62e8256b194e16dbad015572b1de890e1de92273"} Jul 12 18:17:37.348057 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:37.348031 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:37.348124 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:37.348111 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:37.348181 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:37.348166 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:37.348247 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:37.348231 2577 scope.go:117] "RemoveContainer" containerID="1a0de48e904e2b9755bf8ef27faa8b76fda646d38b45ace90dd43f3468ebf926" Jul 12 18:17:37.363459 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:37.363438 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:37.363589 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:37.363575 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:17:37.982283 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:37.981971 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-hqhkm" Jul 12 18:17:37.982283 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:37.982102 2577 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 12 18:17:37.992572 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:37.991998 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-hqhkm" Jul 12 18:17:38.354222 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:38.354147 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-spzqd_ed115460-b220-476d-b9d5-65f95ec3d059/ovn-acl-logging/0.log" Jul 12 18:17:38.354680 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:38.354554 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" event={"ID":"ed115460-b220-476d-b9d5-65f95ec3d059","Type":"ContainerStarted","Data":"c417483fdc9f4517cba73207ea7a093cf4fd31bab185fad9ddbaa907fc0bb457"} Jul 12 18:17:38.386082 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:38.386025 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" podStartSLOduration=8.452696199 podStartE2EDuration="29.38600626s" totalImagesPullingTime="20.933310061s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 18:17:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:17:38.385577682 +0000 UTC m=+31.512482495" watchObservedRunningTime="2026-07-12 18:17:38.38600626 +0000 UTC m=+31.512911065" Jul 12 18:17:38.639536 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:38.639500 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lfnkm"] Jul 12 18:17:38.639691 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:38.639676 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lfnkm" Jul 12 18:17:38.639793 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:38.639772 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lfnkm" podUID="4d1fc6eb-09af-4f96-8ec8-b193da43354c" Jul 12 18:17:38.640100 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:38.640080 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-r66xp"] Jul 12 18:17:38.640184 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:38.640172 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r66xp" Jul 12 18:17:38.640270 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:38.640253 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r66xp" podUID="7d69e388-ddc9-4dd8-b5dc-a81a59dc1698" Jul 12 18:17:38.649818 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:38.649791 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2598z"] Jul 12 18:17:38.649910 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:38.649867 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2598z" Jul 12 18:17:38.649948 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:38.649928 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2598z" podUID="9f8419c6-07be-4876-9762-d98cd772054b" Jul 12 18:17:39.358335 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:39.358300 2577 generic.go:356] "Generic (PLEG): container finished" podID="1a19952f-4732-46da-98ee-05c476b44908" containerID="a4b04f284278f474b470ea61943fc6b5de013e03423e595ee09ab49f41ad07ce" exitCode=0 Jul 12 18:17:39.358769 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:39.358362 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-29hd5" event={"ID":"1a19952f-4732-46da-98ee-05c476b44908","Type":"ContainerDied","Data":"a4b04f284278f474b470ea61943fc6b5de013e03423e595ee09ab49f41ad07ce"} Jul 12 18:17:40.069565 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:40.069520 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2598z" Jul 12 18:17:40.069565 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:40.069551 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lfnkm" Jul 12 18:17:40.069565 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:40.069565 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r66xp" Jul 12 18:17:40.069834 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:40.069660 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lfnkm" podUID="4d1fc6eb-09af-4f96-8ec8-b193da43354c" Jul 12 18:17:40.069834 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:40.069738 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r66xp" podUID="7d69e388-ddc9-4dd8-b5dc-a81a59dc1698" Jul 12 18:17:40.069834 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:40.069803 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2598z" podUID="9f8419c6-07be-4876-9762-d98cd772054b" Jul 12 18:17:41.326318 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:41.326285 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d69e388-ddc9-4dd8-b5dc-a81a59dc1698-metrics-certs\") pod \"network-metrics-daemon-r66xp\" (UID: \"7d69e388-ddc9-4dd8-b5dc-a81a59dc1698\") " pod="openshift-multus/network-metrics-daemon-r66xp" Jul 12 18:17:41.326905 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:41.326339 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9f8419c6-07be-4876-9762-d98cd772054b-original-pull-secret\") pod \"global-pull-secret-syncer-2598z\" (UID: \"9f8419c6-07be-4876-9762-d98cd772054b\") " pod="kube-system/global-pull-secret-syncer-2598z" Jul 12 18:17:41.326905 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:41.326435 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 18:17:41.326905 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:41.326442 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 18:17:41.326905 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:41.326485 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f8419c6-07be-4876-9762-d98cd772054b-original-pull-secret podName:9f8419c6-07be-4876-9762-d98cd772054b nodeName:}" failed. No retries permitted until 2026-07-12 18:18:13.326472479 +0000 UTC m=+66.453377262 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9f8419c6-07be-4876-9762-d98cd772054b-original-pull-secret") pod "global-pull-secret-syncer-2598z" (UID: "9f8419c6-07be-4876-9762-d98cd772054b") : object "kube-system"/"original-pull-secret" not registered Jul 12 18:17:41.326905 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:41.326500 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d69e388-ddc9-4dd8-b5dc-a81a59dc1698-metrics-certs podName:7d69e388-ddc9-4dd8-b5dc-a81a59dc1698 nodeName:}" failed. No retries permitted until 2026-07-12 18:18:13.326494002 +0000 UTC m=+66.453398785 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7d69e388-ddc9-4dd8-b5dc-a81a59dc1698-metrics-certs") pod "network-metrics-daemon-r66xp" (UID: "7d69e388-ddc9-4dd8-b5dc-a81a59dc1698") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 18:17:41.365671 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:41.365639 2577 generic.go:356] "Generic (PLEG): container finished" podID="1a19952f-4732-46da-98ee-05c476b44908" containerID="c25d552ac5d4f62a9009f302db1c642b5d3f73cf2fe8e61ffe25c6dce6c3d388" exitCode=0 Jul 12 18:17:41.365789 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:41.365696 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-29hd5" event={"ID":"1a19952f-4732-46da-98ee-05c476b44908","Type":"ContainerDied","Data":"c25d552ac5d4f62a9009f302db1c642b5d3f73cf2fe8e61ffe25c6dce6c3d388"} Jul 12 18:17:41.427425 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:41.427394 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpzmv\" (UniqueName: \"kubernetes.io/projected/4d1fc6eb-09af-4f96-8ec8-b193da43354c-kube-api-access-tpzmv\") pod \"network-check-target-lfnkm\" (UID: \"4d1fc6eb-09af-4f96-8ec8-b193da43354c\") " pod="openshift-network-diagnostics/network-check-target-lfnkm" Jul 12 18:17:41.427603 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:41.427513 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 18:17:41.427603 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:41.427531 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 18:17:41.427603 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:41.427540 2577 projected.go:196] Error preparing data for projected volume kube-api-access-tpzmv for pod openshift-network-diagnostics/network-check-target-lfnkm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 18:17:41.427603 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:41.427583 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4d1fc6eb-09af-4f96-8ec8-b193da43354c-kube-api-access-tpzmv podName:4d1fc6eb-09af-4f96-8ec8-b193da43354c nodeName:}" failed. No retries permitted until 2026-07-12 18:18:13.427571637 +0000 UTC m=+66.554476419 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-tpzmv" (UniqueName: "kubernetes.io/projected/4d1fc6eb-09af-4f96-8ec8-b193da43354c-kube-api-access-tpzmv") pod "network-check-target-lfnkm" (UID: "4d1fc6eb-09af-4f96-8ec8-b193da43354c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 18:17:42.070169 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:42.070140 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2598z" Jul 12 18:17:42.070366 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:42.070141 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r66xp" Jul 12 18:17:42.070366 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:42.070258 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2598z" podUID="9f8419c6-07be-4876-9762-d98cd772054b" Jul 12 18:17:42.070366 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:42.070353 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r66xp" podUID="7d69e388-ddc9-4dd8-b5dc-a81a59dc1698" Jul 12 18:17:42.070582 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:42.070141 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lfnkm" Jul 12 18:17:42.070582 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:42.070440 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lfnkm" podUID="4d1fc6eb-09af-4f96-8ec8-b193da43354c" Jul 12 18:17:44.070527 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.070492 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2598z" Jul 12 18:17:44.071122 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.070608 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r66xp" Jul 12 18:17:44.071122 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:44.070617 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2598z" podUID="9f8419c6-07be-4876-9762-d98cd772054b" Jul 12 18:17:44.071122 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.070643 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lfnkm" Jul 12 18:17:44.071122 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:44.070753 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r66xp" podUID="7d69e388-ddc9-4dd8-b5dc-a81a59dc1698" Jul 12 18:17:44.071122 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:44.070839 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lfnkm" podUID="4d1fc6eb-09af-4f96-8ec8-b193da43354c" Jul 12 18:17:44.704266 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.704236 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeReady" Jul 12 18:17:44.704422 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.704394 2577 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 12 18:17:44.736418 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.736209 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7bc8fbd9f8-mc6xx"] Jul 12 18:17:44.741096 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.741067 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7bc8fbd9f8-mc6xx" Jul 12 18:17:44.743448 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.743410 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 12 18:17:44.743995 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.743969 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 12 18:17:44.744119 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.744037 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 12 18:17:44.744438 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.744416 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-p9vq2\"" Jul 12 18:17:44.746836 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.746805 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-ld5vx"] Jul 12 18:17:44.750243 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.750122 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-zf4zr"] Jul 12 18:17:44.750312 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.750263 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ld5vx" Jul 12 18:17:44.751122 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.751098 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 12 18:17:44.753252 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.753231 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-h7xmt\"" Jul 12 18:17:44.753439 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.753420 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7bc8fbd9f8-mc6xx"] Jul 12 18:17:44.753551 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.753506 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 12 18:17:44.753551 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.753523 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-zf4zr" Jul 12 18:17:44.753807 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.753772 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 12 18:17:44.755955 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.755940 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 12 18:17:44.756156 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.756134 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 12 18:17:44.756296 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.755943 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-hwzkn\"" Jul 12 18:17:44.756435 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.756395 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 12 18:17:44.759444 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.759382 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ld5vx"] Jul 12 18:17:44.762778 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.762756 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-zf4zr"] Jul 12 18:17:44.857075 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.857043 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-registry-certificates\") pod \"image-registry-7bc8fbd9f8-mc6xx\" (UID: \"d5113b21-a815-40a5-8b61-0c3b0e8c58ca\") " pod="openshift-image-registry/image-registry-7bc8fbd9f8-mc6xx" Jul 12 18:17:44.857266 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.857093 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e812eecd-24a1-443b-aee0-0372c0324c56-tmp-dir\") pod \"dns-default-ld5vx\" (UID: \"e812eecd-24a1-443b-aee0-0372c0324c56\") " pod="openshift-dns/dns-default-ld5vx" Jul 12 18:17:44.857266 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.857115 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trgdn\" (UniqueName: \"kubernetes.io/projected/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-kube-api-access-trgdn\") pod \"image-registry-7bc8fbd9f8-mc6xx\" (UID: \"d5113b21-a815-40a5-8b61-0c3b0e8c58ca\") " pod="openshift-image-registry/image-registry-7bc8fbd9f8-mc6xx" Jul 12 18:17:44.857266 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.857141 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e812eecd-24a1-443b-aee0-0372c0324c56-config-volume\") pod \"dns-default-ld5vx\" (UID: \"e812eecd-24a1-443b-aee0-0372c0324c56\") " pod="openshift-dns/dns-default-ld5vx" Jul 12 18:17:44.857266 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.857163 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e812eecd-24a1-443b-aee0-0372c0324c56-metrics-tls\") pod \"dns-default-ld5vx\" (UID: \"e812eecd-24a1-443b-aee0-0372c0324c56\") " pod="openshift-dns/dns-default-ld5vx" Jul 12 18:17:44.857266 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.857212 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d7643e94-b3c1-428f-842e-4079174ea430-cert\") pod \"ingress-canary-zf4zr\" (UID: \"d7643e94-b3c1-428f-842e-4079174ea430\") " pod="openshift-ingress-canary/ingress-canary-zf4zr" Jul 12 18:17:44.857499 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.857282 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7g8xg\" (UniqueName: \"kubernetes.io/projected/e812eecd-24a1-443b-aee0-0372c0324c56-kube-api-access-7g8xg\") pod \"dns-default-ld5vx\" (UID: \"e812eecd-24a1-443b-aee0-0372c0324c56\") " pod="openshift-dns/dns-default-ld5vx" Jul 12 18:17:44.857499 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.857303 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xd6kj\" (UniqueName: \"kubernetes.io/projected/d7643e94-b3c1-428f-842e-4079174ea430-kube-api-access-xd6kj\") pod \"ingress-canary-zf4zr\" (UID: \"d7643e94-b3c1-428f-842e-4079174ea430\") " pod="openshift-ingress-canary/ingress-canary-zf4zr" Jul 12 18:17:44.857499 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.857334 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-bound-sa-token\") pod \"image-registry-7bc8fbd9f8-mc6xx\" (UID: \"d5113b21-a815-40a5-8b61-0c3b0e8c58ca\") " pod="openshift-image-registry/image-registry-7bc8fbd9f8-mc6xx" Jul 12 18:17:44.857499 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.857396 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-registry-tls\") pod \"image-registry-7bc8fbd9f8-mc6xx\" (UID: \"d5113b21-a815-40a5-8b61-0c3b0e8c58ca\") " pod="openshift-image-registry/image-registry-7bc8fbd9f8-mc6xx" Jul 12 18:17:44.857499 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.857433 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-installation-pull-secrets\") pod \"image-registry-7bc8fbd9f8-mc6xx\" (UID: \"d5113b21-a815-40a5-8b61-0c3b0e8c58ca\") " pod="openshift-image-registry/image-registry-7bc8fbd9f8-mc6xx" Jul 12 18:17:44.857499 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.857475 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-image-registry-private-configuration\") pod \"image-registry-7bc8fbd9f8-mc6xx\" (UID: \"d5113b21-a815-40a5-8b61-0c3b0e8c58ca\") " pod="openshift-image-registry/image-registry-7bc8fbd9f8-mc6xx" Jul 12 18:17:44.857761 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.857538 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-ca-trust-extracted\") pod \"image-registry-7bc8fbd9f8-mc6xx\" (UID: \"d5113b21-a815-40a5-8b61-0c3b0e8c58ca\") " pod="openshift-image-registry/image-registry-7bc8fbd9f8-mc6xx" Jul 12 18:17:44.857761 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.857558 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-trusted-ca\") pod \"image-registry-7bc8fbd9f8-mc6xx\" (UID: \"d5113b21-a815-40a5-8b61-0c3b0e8c58ca\") " pod="openshift-image-registry/image-registry-7bc8fbd9f8-mc6xx" Jul 12 18:17:44.958315 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.958274 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-image-registry-private-configuration\") pod \"image-registry-7bc8fbd9f8-mc6xx\" (UID: \"d5113b21-a815-40a5-8b61-0c3b0e8c58ca\") " pod="openshift-image-registry/image-registry-7bc8fbd9f8-mc6xx" Jul 12 18:17:44.958480 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.958344 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-ca-trust-extracted\") pod \"image-registry-7bc8fbd9f8-mc6xx\" (UID: \"d5113b21-a815-40a5-8b61-0c3b0e8c58ca\") " pod="openshift-image-registry/image-registry-7bc8fbd9f8-mc6xx" Jul 12 18:17:44.958480 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.958373 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-trusted-ca\") pod \"image-registry-7bc8fbd9f8-mc6xx\" (UID: \"d5113b21-a815-40a5-8b61-0c3b0e8c58ca\") " pod="openshift-image-registry/image-registry-7bc8fbd9f8-mc6xx" Jul 12 18:17:44.958480 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.958400 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-registry-certificates\") pod \"image-registry-7bc8fbd9f8-mc6xx\" (UID: \"d5113b21-a815-40a5-8b61-0c3b0e8c58ca\") " pod="openshift-image-registry/image-registry-7bc8fbd9f8-mc6xx" Jul 12 18:17:44.958480 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.958440 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e812eecd-24a1-443b-aee0-0372c0324c56-tmp-dir\") pod \"dns-default-ld5vx\" (UID: \"e812eecd-24a1-443b-aee0-0372c0324c56\") " pod="openshift-dns/dns-default-ld5vx" Jul 12 18:17:44.958480 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.958463 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trgdn\" (UniqueName: \"kubernetes.io/projected/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-kube-api-access-trgdn\") pod \"image-registry-7bc8fbd9f8-mc6xx\" (UID: \"d5113b21-a815-40a5-8b61-0c3b0e8c58ca\") " pod="openshift-image-registry/image-registry-7bc8fbd9f8-mc6xx" Jul 12 18:17:44.958760 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.958488 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e812eecd-24a1-443b-aee0-0372c0324c56-config-volume\") pod \"dns-default-ld5vx\" (UID: \"e812eecd-24a1-443b-aee0-0372c0324c56\") " pod="openshift-dns/dns-default-ld5vx" Jul 12 18:17:44.958857 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.958830 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-ca-trust-extracted\") pod \"image-registry-7bc8fbd9f8-mc6xx\" (UID: \"d5113b21-a815-40a5-8b61-0c3b0e8c58ca\") " pod="openshift-image-registry/image-registry-7bc8fbd9f8-mc6xx" Jul 12 18:17:44.958986 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.958857 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e812eecd-24a1-443b-aee0-0372c0324c56-tmp-dir\") pod \"dns-default-ld5vx\" (UID: \"e812eecd-24a1-443b-aee0-0372c0324c56\") " pod="openshift-dns/dns-default-ld5vx" Jul 12 18:17:44.958986 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.958972 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e812eecd-24a1-443b-aee0-0372c0324c56-metrics-tls\") pod \"dns-default-ld5vx\" (UID: \"e812eecd-24a1-443b-aee0-0372c0324c56\") " pod="openshift-dns/dns-default-ld5vx" Jul 12 18:17:44.959098 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.959005 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d7643e94-b3c1-428f-842e-4079174ea430-cert\") pod \"ingress-canary-zf4zr\" (UID: \"d7643e94-b3c1-428f-842e-4079174ea430\") " pod="openshift-ingress-canary/ingress-canary-zf4zr" Jul 12 18:17:44.959098 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.959057 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7g8xg\" (UniqueName: \"kubernetes.io/projected/e812eecd-24a1-443b-aee0-0372c0324c56-kube-api-access-7g8xg\") pod \"dns-default-ld5vx\" (UID: \"e812eecd-24a1-443b-aee0-0372c0324c56\") " pod="openshift-dns/dns-default-ld5vx" Jul 12 18:17:44.959098 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.959083 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xd6kj\" (UniqueName: \"kubernetes.io/projected/d7643e94-b3c1-428f-842e-4079174ea430-kube-api-access-xd6kj\") pod \"ingress-canary-zf4zr\" (UID: \"d7643e94-b3c1-428f-842e-4079174ea430\") " pod="openshift-ingress-canary/ingress-canary-zf4zr" Jul 12 18:17:44.959239 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.959113 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-bound-sa-token\") pod \"image-registry-7bc8fbd9f8-mc6xx\" (UID: \"d5113b21-a815-40a5-8b61-0c3b0e8c58ca\") " pod="openshift-image-registry/image-registry-7bc8fbd9f8-mc6xx" Jul 12 18:17:44.959239 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.959115 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e812eecd-24a1-443b-aee0-0372c0324c56-config-volume\") pod \"dns-default-ld5vx\" (UID: \"e812eecd-24a1-443b-aee0-0372c0324c56\") " pod="openshift-dns/dns-default-ld5vx" Jul 12 18:17:44.959239 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.959146 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-registry-tls\") pod \"image-registry-7bc8fbd9f8-mc6xx\" (UID: \"d5113b21-a815-40a5-8b61-0c3b0e8c58ca\") " pod="openshift-image-registry/image-registry-7bc8fbd9f8-mc6xx" Jul 12 18:17:44.959239 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.959150 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-registry-certificates\") pod \"image-registry-7bc8fbd9f8-mc6xx\" (UID: \"d5113b21-a815-40a5-8b61-0c3b0e8c58ca\") " pod="openshift-image-registry/image-registry-7bc8fbd9f8-mc6xx" Jul 12 18:17:44.959239 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.959171 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-installation-pull-secrets\") pod \"image-registry-7bc8fbd9f8-mc6xx\" (UID: \"d5113b21-a815-40a5-8b61-0c3b0e8c58ca\") " pod="openshift-image-registry/image-registry-7bc8fbd9f8-mc6xx" Jul 12 18:17:44.959239 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:44.959186 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 18:17:44.959536 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:44.959247 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e812eecd-24a1-443b-aee0-0372c0324c56-metrics-tls podName:e812eecd-24a1-443b-aee0-0372c0324c56 nodeName:}" failed. No retries permitted until 2026-07-12 18:17:45.459224184 +0000 UTC m=+38.586128967 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e812eecd-24a1-443b-aee0-0372c0324c56-metrics-tls") pod "dns-default-ld5vx" (UID: "e812eecd-24a1-443b-aee0-0372c0324c56") : secret "dns-default-metrics-tls" not found Jul 12 18:17:44.959536 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.959450 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-trusted-ca\") pod \"image-registry-7bc8fbd9f8-mc6xx\" (UID: \"d5113b21-a815-40a5-8b61-0c3b0e8c58ca\") " pod="openshift-image-registry/image-registry-7bc8fbd9f8-mc6xx" Jul 12 18:17:44.959536 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:44.959488 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 12 18:17:44.959536 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:44.959505 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7bc8fbd9f8-mc6xx: secret "image-registry-tls" not found Jul 12 18:17:44.959762 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:44.959565 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-registry-tls podName:d5113b21-a815-40a5-8b61-0c3b0e8c58ca nodeName:}" failed. No retries permitted until 2026-07-12 18:17:45.459547778 +0000 UTC m=+38.586452576 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-registry-tls") pod "image-registry-7bc8fbd9f8-mc6xx" (UID: "d5113b21-a815-40a5-8b61-0c3b0e8c58ca") : secret "image-registry-tls" not found Jul 12 18:17:44.963122 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.963103 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d7643e94-b3c1-428f-842e-4079174ea430-cert\") pod \"ingress-canary-zf4zr\" (UID: \"d7643e94-b3c1-428f-842e-4079174ea430\") " pod="openshift-ingress-canary/ingress-canary-zf4zr" Jul 12 18:17:44.963530 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.963494 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-image-registry-private-configuration\") pod \"image-registry-7bc8fbd9f8-mc6xx\" (UID: \"d5113b21-a815-40a5-8b61-0c3b0e8c58ca\") " pod="openshift-image-registry/image-registry-7bc8fbd9f8-mc6xx" Jul 12 18:17:44.963618 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.963493 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-installation-pull-secrets\") pod \"image-registry-7bc8fbd9f8-mc6xx\" (UID: \"d5113b21-a815-40a5-8b61-0c3b0e8c58ca\") " pod="openshift-image-registry/image-registry-7bc8fbd9f8-mc6xx" Jul 12 18:17:44.968350 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.968318 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xd6kj\" (UniqueName: \"kubernetes.io/projected/d7643e94-b3c1-428f-842e-4079174ea430-kube-api-access-xd6kj\") pod \"ingress-canary-zf4zr\" (UID: \"d7643e94-b3c1-428f-842e-4079174ea430\") " pod="openshift-ingress-canary/ingress-canary-zf4zr" Jul 12 18:17:44.968466 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.968412 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7g8xg\" (UniqueName: \"kubernetes.io/projected/e812eecd-24a1-443b-aee0-0372c0324c56-kube-api-access-7g8xg\") pod \"dns-default-ld5vx\" (UID: \"e812eecd-24a1-443b-aee0-0372c0324c56\") " pod="openshift-dns/dns-default-ld5vx" Jul 12 18:17:44.968525 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.968471 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-bound-sa-token\") pod \"image-registry-7bc8fbd9f8-mc6xx\" (UID: \"d5113b21-a815-40a5-8b61-0c3b0e8c58ca\") " pod="openshift-image-registry/image-registry-7bc8fbd9f8-mc6xx" Jul 12 18:17:44.968786 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:44.968742 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trgdn\" (UniqueName: \"kubernetes.io/projected/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-kube-api-access-trgdn\") pod \"image-registry-7bc8fbd9f8-mc6xx\" (UID: \"d5113b21-a815-40a5-8b61-0c3b0e8c58ca\") " pod="openshift-image-registry/image-registry-7bc8fbd9f8-mc6xx" Jul 12 18:17:45.071594 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:45.071500 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-zf4zr" Jul 12 18:17:45.464961 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:45.464922 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-registry-tls\") pod \"image-registry-7bc8fbd9f8-mc6xx\" (UID: \"d5113b21-a815-40a5-8b61-0c3b0e8c58ca\") " pod="openshift-image-registry/image-registry-7bc8fbd9f8-mc6xx" Jul 12 18:17:45.465121 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:45.465028 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e812eecd-24a1-443b-aee0-0372c0324c56-metrics-tls\") pod \"dns-default-ld5vx\" (UID: \"e812eecd-24a1-443b-aee0-0372c0324c56\") " pod="openshift-dns/dns-default-ld5vx" Jul 12 18:17:45.465121 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:45.465047 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 12 18:17:45.465121 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:45.465070 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7bc8fbd9f8-mc6xx: secret "image-registry-tls" not found Jul 12 18:17:45.465272 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:45.465168 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 18:17:45.465272 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:45.465156 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-registry-tls podName:d5113b21-a815-40a5-8b61-0c3b0e8c58ca nodeName:}" failed. No retries permitted until 2026-07-12 18:17:46.465135511 +0000 UTC m=+39.592040311 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-registry-tls") pod "image-registry-7bc8fbd9f8-mc6xx" (UID: "d5113b21-a815-40a5-8b61-0c3b0e8c58ca") : secret "image-registry-tls" not found Jul 12 18:17:45.465272 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:45.465229 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e812eecd-24a1-443b-aee0-0372c0324c56-metrics-tls podName:e812eecd-24a1-443b-aee0-0372c0324c56 nodeName:}" failed. No retries permitted until 2026-07-12 18:17:46.465209939 +0000 UTC m=+39.592114725 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e812eecd-24a1-443b-aee0-0372c0324c56-metrics-tls") pod "dns-default-ld5vx" (UID: "e812eecd-24a1-443b-aee0-0372c0324c56") : secret "dns-default-metrics-tls" not found Jul 12 18:17:45.709354 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:45.709190 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-zf4zr"] Jul 12 18:17:45.719538 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:45.719512 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7643e94_b3c1_428f_842e_4079174ea430.slice/crio-326fb82b05fcee8b3ecdcde1f11ddd54b3dd3fd400bdb4f249c3d84328d11c32 WatchSource:0}: Error finding container 326fb82b05fcee8b3ecdcde1f11ddd54b3dd3fd400bdb4f249c3d84328d11c32: Status 404 returned error can't find the container with id 326fb82b05fcee8b3ecdcde1f11ddd54b3dd3fd400bdb4f249c3d84328d11c32 Jul 12 18:17:46.070149 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:46.070041 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2598z" Jul 12 18:17:46.070149 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:46.070087 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lfnkm" Jul 12 18:17:46.070343 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:46.070078 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r66xp" Jul 12 18:17:46.073892 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:46.073872 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 12 18:17:46.074254 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:46.073892 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hb5mg\"" Jul 12 18:17:46.074254 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:46.073894 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 12 18:17:46.074254 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:46.074128 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 12 18:17:46.074254 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:46.074152 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 12 18:17:46.074254 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:46.074166 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-p9cbp\"" Jul 12 18:17:46.381637 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:46.381546 2577 generic.go:356] "Generic (PLEG): container finished" podID="1a19952f-4732-46da-98ee-05c476b44908" containerID="87e3a3958ceeafc78fa892da3e7451f572cc33a9532a6cb83cf6222230656bae" exitCode=0 Jul 12 18:17:46.381791 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:46.381644 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-29hd5" event={"ID":"1a19952f-4732-46da-98ee-05c476b44908","Type":"ContainerDied","Data":"87e3a3958ceeafc78fa892da3e7451f572cc33a9532a6cb83cf6222230656bae"} Jul 12 18:17:46.382853 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:46.382831 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-zf4zr" event={"ID":"d7643e94-b3c1-428f-842e-4079174ea430","Type":"ContainerStarted","Data":"326fb82b05fcee8b3ecdcde1f11ddd54b3dd3fd400bdb4f249c3d84328d11c32"} Jul 12 18:17:46.473903 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:46.473874 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e812eecd-24a1-443b-aee0-0372c0324c56-metrics-tls\") pod \"dns-default-ld5vx\" (UID: \"e812eecd-24a1-443b-aee0-0372c0324c56\") " pod="openshift-dns/dns-default-ld5vx" Jul 12 18:17:46.474011 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:46.473956 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-registry-tls\") pod \"image-registry-7bc8fbd9f8-mc6xx\" (UID: \"d5113b21-a815-40a5-8b61-0c3b0e8c58ca\") " pod="openshift-image-registry/image-registry-7bc8fbd9f8-mc6xx" Jul 12 18:17:46.474070 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:46.474047 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 18:17:46.474133 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:46.474108 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e812eecd-24a1-443b-aee0-0372c0324c56-metrics-tls podName:e812eecd-24a1-443b-aee0-0372c0324c56 nodeName:}" failed. No retries permitted until 2026-07-12 18:17:48.47408722 +0000 UTC m=+41.600992003 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e812eecd-24a1-443b-aee0-0372c0324c56-metrics-tls") pod "dns-default-ld5vx" (UID: "e812eecd-24a1-443b-aee0-0372c0324c56") : secret "dns-default-metrics-tls" not found Jul 12 18:17:46.474287 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:46.474265 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 12 18:17:46.474287 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:46.474289 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7bc8fbd9f8-mc6xx: secret "image-registry-tls" not found Jul 12 18:17:46.474437 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:46.474337 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-registry-tls podName:d5113b21-a815-40a5-8b61-0c3b0e8c58ca nodeName:}" failed. No retries permitted until 2026-07-12 18:17:48.474324559 +0000 UTC m=+41.601229342 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-registry-tls") pod "image-registry-7bc8fbd9f8-mc6xx" (UID: "d5113b21-a815-40a5-8b61-0c3b0e8c58ca") : secret "image-registry-tls" not found Jul 12 18:17:47.388505 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:47.388467 2577 generic.go:356] "Generic (PLEG): container finished" podID="1a19952f-4732-46da-98ee-05c476b44908" containerID="ef2c6c404a3a2279840c30826847b56a54b0f54852fefb62160879d39c761154" exitCode=0 Jul 12 18:17:47.388971 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:47.388534 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-29hd5" event={"ID":"1a19952f-4732-46da-98ee-05c476b44908","Type":"ContainerDied","Data":"ef2c6c404a3a2279840c30826847b56a54b0f54852fefb62160879d39c761154"} Jul 12 18:17:48.394213 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:48.393978 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-29hd5" event={"ID":"1a19952f-4732-46da-98ee-05c476b44908","Type":"ContainerStarted","Data":"ba96eb097fb1547f58f989cfb6b263b83b091766c6e1b8288b88f3e2999427b2"} Jul 12 18:17:48.395433 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:48.395411 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-zf4zr" event={"ID":"d7643e94-b3c1-428f-842e-4079174ea430","Type":"ContainerStarted","Data":"ac73d5d2ee878bf3e7be3631b69a7fad6f24b76e7104b9155715a425fd102dfd"} Jul 12 18:17:48.415501 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:48.415448 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-29hd5" podStartSLOduration=8.264405749 podStartE2EDuration="39.415430236s" totalImagesPullingTime="31.151024487s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 18:17:09 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:17:48.413651271 +0000 UTC m=+41.540556076" watchObservedRunningTime="2026-07-12 18:17:48.415430236 +0000 UTC m=+41.542335042" Jul 12 18:17:48.427067 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:48.427025 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-zf4zr" podStartSLOduration=2.1205298790000002 podStartE2EDuration="4.42700891s" totalImagesPullingTime="2.306479031s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:17:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:17:48.426843521 +0000 UTC m=+41.553748350" watchObservedRunningTime="2026-07-12 18:17:48.42700891 +0000 UTC m=+41.553913715" Jul 12 18:17:48.488260 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:48.488228 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e812eecd-24a1-443b-aee0-0372c0324c56-metrics-tls\") pod \"dns-default-ld5vx\" (UID: \"e812eecd-24a1-443b-aee0-0372c0324c56\") " pod="openshift-dns/dns-default-ld5vx" Jul 12 18:17:48.488408 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:48.488278 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-registry-tls\") pod \"image-registry-7bc8fbd9f8-mc6xx\" (UID: \"d5113b21-a815-40a5-8b61-0c3b0e8c58ca\") " pod="openshift-image-registry/image-registry-7bc8fbd9f8-mc6xx" Jul 12 18:17:48.488408 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:48.488386 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 18:17:48.488408 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:48.488403 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 12 18:17:48.488507 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:48.488416 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7bc8fbd9f8-mc6xx: secret "image-registry-tls" not found Jul 12 18:17:48.488507 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:48.488448 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e812eecd-24a1-443b-aee0-0372c0324c56-metrics-tls podName:e812eecd-24a1-443b-aee0-0372c0324c56 nodeName:}" failed. No retries permitted until 2026-07-12 18:17:52.488432449 +0000 UTC m=+45.615337231 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e812eecd-24a1-443b-aee0-0372c0324c56-metrics-tls") pod "dns-default-ld5vx" (UID: "e812eecd-24a1-443b-aee0-0372c0324c56") : secret "dns-default-metrics-tls" not found Jul 12 18:17:48.488507 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:48.488462 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-registry-tls podName:d5113b21-a815-40a5-8b61-0c3b0e8c58ca nodeName:}" failed. No retries permitted until 2026-07-12 18:17:52.488456659 +0000 UTC m=+45.615361441 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-registry-tls") pod "image-registry-7bc8fbd9f8-mc6xx" (UID: "d5113b21-a815-40a5-8b61-0c3b0e8c58ca") : secret "image-registry-tls" not found Jul 12 18:17:51.879635 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:51.879601 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-hvczh"] Jul 12 18:17:51.909075 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:51.909040 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-hvczh"] Jul 12 18:17:51.909196 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:51.909141 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-hvczh" Jul 12 18:17:51.912175 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:51.912142 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 12 18:17:51.912175 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:51.912142 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 12 18:17:51.912334 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:51.912254 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-w5zqd\"" Jul 12 18:17:51.913537 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:51.913509 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rh57b\" (UniqueName: \"kubernetes.io/projected/aa7d16a7-5382-47dd-9455-07282d523685-kube-api-access-rh57b\") pod \"migrator-7f59b54f6c-hvczh\" (UID: \"aa7d16a7-5382-47dd-9455-07282d523685\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-hvczh" Jul 12 18:17:52.014513 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:52.014482 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rh57b\" (UniqueName: \"kubernetes.io/projected/aa7d16a7-5382-47dd-9455-07282d523685-kube-api-access-rh57b\") pod \"migrator-7f59b54f6c-hvczh\" (UID: \"aa7d16a7-5382-47dd-9455-07282d523685\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-hvczh" Jul 12 18:17:52.023920 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:52.023892 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rh57b\" (UniqueName: \"kubernetes.io/projected/aa7d16a7-5382-47dd-9455-07282d523685-kube-api-access-rh57b\") pod \"migrator-7f59b54f6c-hvczh\" (UID: \"aa7d16a7-5382-47dd-9455-07282d523685\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-hvczh" Jul 12 18:17:52.217971 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:52.217942 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-hvczh" Jul 12 18:17:52.358259 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:52.358102 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-hvczh"] Jul 12 18:17:52.361858 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:52.361831 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa7d16a7_5382_47dd_9455_07282d523685.slice/crio-d575b96b2d9a31e7f2eed60e4d53846be3ae8db44aa5176dfadf7d1520a9e800 WatchSource:0}: Error finding container d575b96b2d9a31e7f2eed60e4d53846be3ae8db44aa5176dfadf7d1520a9e800: Status 404 returned error can't find the container with id d575b96b2d9a31e7f2eed60e4d53846be3ae8db44aa5176dfadf7d1520a9e800 Jul 12 18:17:52.406104 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:52.406065 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-hvczh" event={"ID":"aa7d16a7-5382-47dd-9455-07282d523685","Type":"ContainerStarted","Data":"d575b96b2d9a31e7f2eed60e4d53846be3ae8db44aa5176dfadf7d1520a9e800"} Jul 12 18:17:52.518508 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:52.518414 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e812eecd-24a1-443b-aee0-0372c0324c56-metrics-tls\") pod \"dns-default-ld5vx\" (UID: \"e812eecd-24a1-443b-aee0-0372c0324c56\") " pod="openshift-dns/dns-default-ld5vx" Jul 12 18:17:52.518508 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:52.518469 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-registry-tls\") pod \"image-registry-7bc8fbd9f8-mc6xx\" (UID: \"d5113b21-a815-40a5-8b61-0c3b0e8c58ca\") " pod="openshift-image-registry/image-registry-7bc8fbd9f8-mc6xx" Jul 12 18:17:52.518768 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:52.518583 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 12 18:17:52.518768 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:52.518582 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 18:17:52.518768 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:52.518594 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7bc8fbd9f8-mc6xx: secret "image-registry-tls" not found Jul 12 18:17:52.518768 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:52.518688 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e812eecd-24a1-443b-aee0-0372c0324c56-metrics-tls podName:e812eecd-24a1-443b-aee0-0372c0324c56 nodeName:}" failed. No retries permitted until 2026-07-12 18:18:00.518668024 +0000 UTC m=+53.645572806 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e812eecd-24a1-443b-aee0-0372c0324c56-metrics-tls") pod "dns-default-ld5vx" (UID: "e812eecd-24a1-443b-aee0-0372c0324c56") : secret "dns-default-metrics-tls" not found Jul 12 18:17:52.518768 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:52.518714 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-registry-tls podName:d5113b21-a815-40a5-8b61-0c3b0e8c58ca nodeName:}" failed. No retries permitted until 2026-07-12 18:18:00.518702806 +0000 UTC m=+53.645607589 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-registry-tls") pod "image-registry-7bc8fbd9f8-mc6xx" (UID: "d5113b21-a815-40a5-8b61-0c3b0e8c58ca") : secret "image-registry-tls" not found Jul 12 18:17:53.129293 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:53.129260 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-6s2lc"] Jul 12 18:17:53.149205 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:53.149172 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6s2lc"] Jul 12 18:17:53.149340 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:53.149301 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6s2lc" Jul 12 18:17:53.151911 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:53.151887 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9cwfc\"" Jul 12 18:17:53.152028 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:53.151886 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 12 18:17:53.152743 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:53.152726 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 12 18:17:53.152743 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:53.152739 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 12 18:17:53.152828 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:53.152726 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 12 18:17:53.224326 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:53.224291 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1cd288a5-920c-4b68-ad97-27d449622108-data-volume\") pod \"insights-runtime-extractor-6s2lc\" (UID: \"1cd288a5-920c-4b68-ad97-27d449622108\") " pod="openshift-insights/insights-runtime-extractor-6s2lc" Jul 12 18:17:53.224326 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:53.224332 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1cd288a5-920c-4b68-ad97-27d449622108-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6s2lc\" (UID: \"1cd288a5-920c-4b68-ad97-27d449622108\") " pod="openshift-insights/insights-runtime-extractor-6s2lc" Jul 12 18:17:53.224525 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:53.224354 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1cd288a5-920c-4b68-ad97-27d449622108-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6s2lc\" (UID: \"1cd288a5-920c-4b68-ad97-27d449622108\") " pod="openshift-insights/insights-runtime-extractor-6s2lc" Jul 12 18:17:53.224525 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:53.224434 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1cd288a5-920c-4b68-ad97-27d449622108-crio-socket\") pod \"insights-runtime-extractor-6s2lc\" (UID: \"1cd288a5-920c-4b68-ad97-27d449622108\") " pod="openshift-insights/insights-runtime-extractor-6s2lc" Jul 12 18:17:53.224525 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:53.224475 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4jpl\" (UniqueName: \"kubernetes.io/projected/1cd288a5-920c-4b68-ad97-27d449622108-kube-api-access-d4jpl\") pod \"insights-runtime-extractor-6s2lc\" (UID: \"1cd288a5-920c-4b68-ad97-27d449622108\") " pod="openshift-insights/insights-runtime-extractor-6s2lc" Jul 12 18:17:53.325229 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:53.325187 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1cd288a5-920c-4b68-ad97-27d449622108-data-volume\") pod \"insights-runtime-extractor-6s2lc\" (UID: \"1cd288a5-920c-4b68-ad97-27d449622108\") " pod="openshift-insights/insights-runtime-extractor-6s2lc" Jul 12 18:17:53.325229 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:53.325228 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1cd288a5-920c-4b68-ad97-27d449622108-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6s2lc\" (UID: \"1cd288a5-920c-4b68-ad97-27d449622108\") " pod="openshift-insights/insights-runtime-extractor-6s2lc" Jul 12 18:17:53.325473 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:53.325259 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1cd288a5-920c-4b68-ad97-27d449622108-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6s2lc\" (UID: \"1cd288a5-920c-4b68-ad97-27d449622108\") " pod="openshift-insights/insights-runtime-extractor-6s2lc" Jul 12 18:17:53.325473 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:53.325296 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1cd288a5-920c-4b68-ad97-27d449622108-crio-socket\") pod \"insights-runtime-extractor-6s2lc\" (UID: \"1cd288a5-920c-4b68-ad97-27d449622108\") " pod="openshift-insights/insights-runtime-extractor-6s2lc" Jul 12 18:17:53.325473 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:53.325323 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4jpl\" (UniqueName: \"kubernetes.io/projected/1cd288a5-920c-4b68-ad97-27d449622108-kube-api-access-d4jpl\") pod \"insights-runtime-extractor-6s2lc\" (UID: \"1cd288a5-920c-4b68-ad97-27d449622108\") " pod="openshift-insights/insights-runtime-extractor-6s2lc" Jul 12 18:17:53.325473 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:53.325336 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 12 18:17:53.325473 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:53.325413 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1cd288a5-920c-4b68-ad97-27d449622108-kube-rbac-proxy-cm podName:1cd288a5-920c-4b68-ad97-27d449622108 nodeName:}" failed. No retries permitted until 2026-07-12 18:17:53.825393629 +0000 UTC m=+46.952298426 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1cd288a5-920c-4b68-ad97-27d449622108-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6s2lc" (UID: "1cd288a5-920c-4b68-ad97-27d449622108") : configmap "kube-rbac-proxy" not found Jul 12 18:17:53.325473 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:53.325438 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 12 18:17:53.325792 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:53.325489 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cd288a5-920c-4b68-ad97-27d449622108-insights-runtime-extractor-tls podName:1cd288a5-920c-4b68-ad97-27d449622108 nodeName:}" failed. No retries permitted until 2026-07-12 18:17:53.825474354 +0000 UTC m=+46.952379139 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1cd288a5-920c-4b68-ad97-27d449622108-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6s2lc" (UID: "1cd288a5-920c-4b68-ad97-27d449622108") : secret "insights-runtime-extractor-tls" not found Jul 12 18:17:53.325792 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:53.325536 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1cd288a5-920c-4b68-ad97-27d449622108-data-volume\") pod \"insights-runtime-extractor-6s2lc\" (UID: \"1cd288a5-920c-4b68-ad97-27d449622108\") " pod="openshift-insights/insights-runtime-extractor-6s2lc" Jul 12 18:17:53.325792 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:53.325649 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1cd288a5-920c-4b68-ad97-27d449622108-crio-socket\") pod \"insights-runtime-extractor-6s2lc\" (UID: \"1cd288a5-920c-4b68-ad97-27d449622108\") " pod="openshift-insights/insights-runtime-extractor-6s2lc" Jul 12 18:17:53.333527 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:53.333496 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4jpl\" (UniqueName: \"kubernetes.io/projected/1cd288a5-920c-4b68-ad97-27d449622108-kube-api-access-d4jpl\") pod \"insights-runtime-extractor-6s2lc\" (UID: \"1cd288a5-920c-4b68-ad97-27d449622108\") " pod="openshift-insights/insights-runtime-extractor-6s2lc" Jul 12 18:17:53.828175 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:53.828141 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1cd288a5-920c-4b68-ad97-27d449622108-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6s2lc\" (UID: \"1cd288a5-920c-4b68-ad97-27d449622108\") " pod="openshift-insights/insights-runtime-extractor-6s2lc" Jul 12 18:17:53.828175 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:53.828180 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1cd288a5-920c-4b68-ad97-27d449622108-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6s2lc\" (UID: \"1cd288a5-920c-4b68-ad97-27d449622108\") " pod="openshift-insights/insights-runtime-extractor-6s2lc" Jul 12 18:17:53.828399 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:53.828270 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 12 18:17:53.828399 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:53.828271 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 12 18:17:53.828399 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:53.828321 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cd288a5-920c-4b68-ad97-27d449622108-insights-runtime-extractor-tls podName:1cd288a5-920c-4b68-ad97-27d449622108 nodeName:}" failed. No retries permitted until 2026-07-12 18:17:54.828308287 +0000 UTC m=+47.955213070 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1cd288a5-920c-4b68-ad97-27d449622108-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6s2lc" (UID: "1cd288a5-920c-4b68-ad97-27d449622108") : secret "insights-runtime-extractor-tls" not found Jul 12 18:17:53.828399 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:53.828334 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1cd288a5-920c-4b68-ad97-27d449622108-kube-rbac-proxy-cm podName:1cd288a5-920c-4b68-ad97-27d449622108 nodeName:}" failed. No retries permitted until 2026-07-12 18:17:54.828328071 +0000 UTC m=+47.955232853 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1cd288a5-920c-4b68-ad97-27d449622108-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6s2lc" (UID: "1cd288a5-920c-4b68-ad97-27d449622108") : configmap "kube-rbac-proxy" not found Jul 12 18:17:54.412855 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:54.412772 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-hvczh" event={"ID":"aa7d16a7-5382-47dd-9455-07282d523685","Type":"ContainerStarted","Data":"e114ea54b0f0b7199671b8bb58e928d435c30b4257777165fe514a8ce9de6508"} Jul 12 18:17:54.412855 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:54.412825 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-hvczh" event={"ID":"aa7d16a7-5382-47dd-9455-07282d523685","Type":"ContainerStarted","Data":"5e7cd6bb287775134e212a79329b6255d5ef73297cbbb1e50a43b9b98b04da36"} Jul 12 18:17:54.429466 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:54.429416 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-hvczh" podStartSLOduration=1.881156915 podStartE2EDuration="3.429400604s" totalImagesPullingTime="1.548243689s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:17:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:17:54.429151533 +0000 UTC m=+47.556056343" watchObservedRunningTime="2026-07-12 18:17:54.429400604 +0000 UTC m=+47.556305408" Jul 12 18:17:54.837219 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:54.837180 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1cd288a5-920c-4b68-ad97-27d449622108-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6s2lc\" (UID: \"1cd288a5-920c-4b68-ad97-27d449622108\") " pod="openshift-insights/insights-runtime-extractor-6s2lc" Jul 12 18:17:54.837382 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:54.837225 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1cd288a5-920c-4b68-ad97-27d449622108-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6s2lc\" (UID: \"1cd288a5-920c-4b68-ad97-27d449622108\") " pod="openshift-insights/insights-runtime-extractor-6s2lc" Jul 12 18:17:54.837382 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:54.837311 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 12 18:17:54.837382 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:54.837369 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1cd288a5-920c-4b68-ad97-27d449622108-kube-rbac-proxy-cm podName:1cd288a5-920c-4b68-ad97-27d449622108 nodeName:}" failed. No retries permitted until 2026-07-12 18:17:56.837355548 +0000 UTC m=+49.964260334 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1cd288a5-920c-4b68-ad97-27d449622108-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6s2lc" (UID: "1cd288a5-920c-4b68-ad97-27d449622108") : configmap "kube-rbac-proxy" not found Jul 12 18:17:54.837488 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:54.837395 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 12 18:17:54.837488 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:54.837453 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cd288a5-920c-4b68-ad97-27d449622108-insights-runtime-extractor-tls podName:1cd288a5-920c-4b68-ad97-27d449622108 nodeName:}" failed. No retries permitted until 2026-07-12 18:17:56.837439952 +0000 UTC m=+49.964344735 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1cd288a5-920c-4b68-ad97-27d449622108-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6s2lc" (UID: "1cd288a5-920c-4b68-ad97-27d449622108") : secret "insights-runtime-extractor-tls" not found Jul 12 18:17:55.718067 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:55.718039 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xwcbp_cd8ab0de-04ec-4aa8-8d33-87aa4bbef597/dns-node-resolver/0.log" Jul 12 18:17:55.884274 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:55.884239 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-mltlx"] Jul 12 18:17:55.919069 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:55.919043 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-mltlx"] Jul 12 18:17:55.919216 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:55.919136 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-mltlx" Jul 12 18:17:55.921869 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:55.921853 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 12 18:17:55.922366 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:55.922351 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 12 18:17:55.922419 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:55.922355 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 12 18:17:55.922975 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:55.922963 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 12 18:17:55.923235 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:55.923221 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-xpvj9\"" Jul 12 18:17:55.945918 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:55.945895 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d59f9a8f-dbf1-40a4-92ed-795c3602abc0-signing-cabundle\") pod \"service-ca-79f68b9f4f-mltlx\" (UID: \"d59f9a8f-dbf1-40a4-92ed-795c3602abc0\") " pod="openshift-service-ca/service-ca-79f68b9f4f-mltlx" Jul 12 18:17:55.946045 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:55.945924 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zg5g4\" (UniqueName: \"kubernetes.io/projected/d59f9a8f-dbf1-40a4-92ed-795c3602abc0-kube-api-access-zg5g4\") pod \"service-ca-79f68b9f4f-mltlx\" (UID: \"d59f9a8f-dbf1-40a4-92ed-795c3602abc0\") " pod="openshift-service-ca/service-ca-79f68b9f4f-mltlx" Jul 12 18:17:55.946045 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:55.945988 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d59f9a8f-dbf1-40a4-92ed-795c3602abc0-signing-key\") pod \"service-ca-79f68b9f4f-mltlx\" (UID: \"d59f9a8f-dbf1-40a4-92ed-795c3602abc0\") " pod="openshift-service-ca/service-ca-79f68b9f4f-mltlx" Jul 12 18:17:56.046532 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:56.046451 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d59f9a8f-dbf1-40a4-92ed-795c3602abc0-signing-cabundle\") pod \"service-ca-79f68b9f4f-mltlx\" (UID: \"d59f9a8f-dbf1-40a4-92ed-795c3602abc0\") " pod="openshift-service-ca/service-ca-79f68b9f4f-mltlx" Jul 12 18:17:56.046532 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:56.046497 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zg5g4\" (UniqueName: \"kubernetes.io/projected/d59f9a8f-dbf1-40a4-92ed-795c3602abc0-kube-api-access-zg5g4\") pod \"service-ca-79f68b9f4f-mltlx\" (UID: \"d59f9a8f-dbf1-40a4-92ed-795c3602abc0\") " pod="openshift-service-ca/service-ca-79f68b9f4f-mltlx" Jul 12 18:17:56.046724 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:56.046586 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d59f9a8f-dbf1-40a4-92ed-795c3602abc0-signing-key\") pod \"service-ca-79f68b9f4f-mltlx\" (UID: \"d59f9a8f-dbf1-40a4-92ed-795c3602abc0\") " pod="openshift-service-ca/service-ca-79f68b9f4f-mltlx" Jul 12 18:17:56.047141 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:56.047116 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d59f9a8f-dbf1-40a4-92ed-795c3602abc0-signing-cabundle\") pod \"service-ca-79f68b9f4f-mltlx\" (UID: \"d59f9a8f-dbf1-40a4-92ed-795c3602abc0\") " pod="openshift-service-ca/service-ca-79f68b9f4f-mltlx" Jul 12 18:17:56.048890 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:56.048863 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d59f9a8f-dbf1-40a4-92ed-795c3602abc0-signing-key\") pod \"service-ca-79f68b9f4f-mltlx\" (UID: \"d59f9a8f-dbf1-40a4-92ed-795c3602abc0\") " pod="openshift-service-ca/service-ca-79f68b9f4f-mltlx" Jul 12 18:17:56.054450 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:56.054432 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zg5g4\" (UniqueName: \"kubernetes.io/projected/d59f9a8f-dbf1-40a4-92ed-795c3602abc0-kube-api-access-zg5g4\") pod \"service-ca-79f68b9f4f-mltlx\" (UID: \"d59f9a8f-dbf1-40a4-92ed-795c3602abc0\") " pod="openshift-service-ca/service-ca-79f68b9f4f-mltlx" Jul 12 18:17:56.228170 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:56.228144 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-mltlx" Jul 12 18:17:56.320865 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:56.320721 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-khqqs_1b501a87-6ac0-4f0b-9072-aeecd64d5bc0/node-ca/0.log" Jul 12 18:17:56.381046 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:56.381012 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-mltlx"] Jul 12 18:17:56.384808 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:17:56.384783 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd59f9a8f_dbf1_40a4_92ed_795c3602abc0.slice/crio-6b7aa3642ecf3f124b9b9fbdd737c72c17d9d0b178c4c49663d92f11f962f097 WatchSource:0}: Error finding container 6b7aa3642ecf3f124b9b9fbdd737c72c17d9d0b178c4c49663d92f11f962f097: Status 404 returned error can't find the container with id 6b7aa3642ecf3f124b9b9fbdd737c72c17d9d0b178c4c49663d92f11f962f097 Jul 12 18:17:56.419111 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:56.419075 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-mltlx" event={"ID":"d59f9a8f-dbf1-40a4-92ed-795c3602abc0","Type":"ContainerStarted","Data":"6b7aa3642ecf3f124b9b9fbdd737c72c17d9d0b178c4c49663d92f11f962f097"} Jul 12 18:17:56.853206 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:56.853170 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1cd288a5-920c-4b68-ad97-27d449622108-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6s2lc\" (UID: \"1cd288a5-920c-4b68-ad97-27d449622108\") " pod="openshift-insights/insights-runtime-extractor-6s2lc" Jul 12 18:17:56.853681 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:56.853212 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1cd288a5-920c-4b68-ad97-27d449622108-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6s2lc\" (UID: \"1cd288a5-920c-4b68-ad97-27d449622108\") " pod="openshift-insights/insights-runtime-extractor-6s2lc" Jul 12 18:17:56.853681 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:56.853322 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 12 18:17:56.853681 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:56.853372 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 12 18:17:56.853681 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:56.853406 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1cd288a5-920c-4b68-ad97-27d449622108-kube-rbac-proxy-cm podName:1cd288a5-920c-4b68-ad97-27d449622108 nodeName:}" failed. No retries permitted until 2026-07-12 18:18:00.853382346 +0000 UTC m=+53.980287151 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1cd288a5-920c-4b68-ad97-27d449622108-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6s2lc" (UID: "1cd288a5-920c-4b68-ad97-27d449622108") : configmap "kube-rbac-proxy" not found Jul 12 18:17:56.853681 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:17:56.853427 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cd288a5-920c-4b68-ad97-27d449622108-insights-runtime-extractor-tls podName:1cd288a5-920c-4b68-ad97-27d449622108 nodeName:}" failed. No retries permitted until 2026-07-12 18:18:00.853417238 +0000 UTC m=+53.980322024 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1cd288a5-920c-4b68-ad97-27d449622108-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6s2lc" (UID: "1cd288a5-920c-4b68-ad97-27d449622108") : secret "insights-runtime-extractor-tls" not found Jul 12 18:17:57.318769 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:57.318723 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-zf4zr_d7643e94-b3c1-428f-842e-4079174ea430/serve-healthcheck-canary/0.log" Jul 12 18:17:57.519559 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:57.519533 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-hvczh_aa7d16a7-5382-47dd-9455-07282d523685/migrator/0.log" Jul 12 18:17:57.718251 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:57.718219 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-hvczh_aa7d16a7-5382-47dd-9455-07282d523685/graceful-termination/0.log" Jul 12 18:17:59.428163 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:59.428123 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-mltlx" event={"ID":"d59f9a8f-dbf1-40a4-92ed-795c3602abc0","Type":"ContainerStarted","Data":"079ed73b2090f1b32ce1e68884f9b5018e6be692ffdd48623aaf6afd2fc84cb7"} Jul 12 18:17:59.444358 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:17:59.444306 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-mltlx" podStartSLOduration=2.276029088 podStartE2EDuration="4.444289979s" totalImagesPullingTime="2.168260891s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:17:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:17:59.443374064 +0000 UTC m=+52.570278873" watchObservedRunningTime="2026-07-12 18:17:59.444289979 +0000 UTC m=+52.571194761" Jul 12 18:18:00.583491 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:00.583449 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e812eecd-24a1-443b-aee0-0372c0324c56-metrics-tls\") pod \"dns-default-ld5vx\" (UID: \"e812eecd-24a1-443b-aee0-0372c0324c56\") " pod="openshift-dns/dns-default-ld5vx" Jul 12 18:18:00.584023 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:00.583526 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-registry-tls\") pod \"image-registry-7bc8fbd9f8-mc6xx\" (UID: \"d5113b21-a815-40a5-8b61-0c3b0e8c58ca\") " pod="openshift-image-registry/image-registry-7bc8fbd9f8-mc6xx" Jul 12 18:18:00.584023 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:18:00.583596 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 18:18:00.584023 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:18:00.583688 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e812eecd-24a1-443b-aee0-0372c0324c56-metrics-tls podName:e812eecd-24a1-443b-aee0-0372c0324c56 nodeName:}" failed. No retries permitted until 2026-07-12 18:18:16.583667413 +0000 UTC m=+69.710572218 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e812eecd-24a1-443b-aee0-0372c0324c56-metrics-tls") pod "dns-default-ld5vx" (UID: "e812eecd-24a1-443b-aee0-0372c0324c56") : secret "dns-default-metrics-tls" not found Jul 12 18:18:00.584023 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:18:00.583696 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 12 18:18:00.584023 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:18:00.583712 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7bc8fbd9f8-mc6xx: secret "image-registry-tls" not found Jul 12 18:18:00.584023 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:18:00.583770 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-registry-tls podName:d5113b21-a815-40a5-8b61-0c3b0e8c58ca nodeName:}" failed. No retries permitted until 2026-07-12 18:18:16.583756117 +0000 UTC m=+69.710660900 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-registry-tls") pod "image-registry-7bc8fbd9f8-mc6xx" (UID: "d5113b21-a815-40a5-8b61-0c3b0e8c58ca") : secret "image-registry-tls" not found Jul 12 18:18:00.885755 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:00.885666 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1cd288a5-920c-4b68-ad97-27d449622108-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6s2lc\" (UID: \"1cd288a5-920c-4b68-ad97-27d449622108\") " pod="openshift-insights/insights-runtime-extractor-6s2lc" Jul 12 18:18:00.885755 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:00.885712 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1cd288a5-920c-4b68-ad97-27d449622108-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6s2lc\" (UID: \"1cd288a5-920c-4b68-ad97-27d449622108\") " pod="openshift-insights/insights-runtime-extractor-6s2lc" Jul 12 18:18:00.885933 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:18:00.885797 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 12 18:18:00.885933 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:18:00.885863 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1cd288a5-920c-4b68-ad97-27d449622108-kube-rbac-proxy-cm podName:1cd288a5-920c-4b68-ad97-27d449622108 nodeName:}" failed. No retries permitted until 2026-07-12 18:18:08.885846101 +0000 UTC m=+62.012750887 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1cd288a5-920c-4b68-ad97-27d449622108-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6s2lc" (UID: "1cd288a5-920c-4b68-ad97-27d449622108") : configmap "kube-rbac-proxy" not found Jul 12 18:18:00.885933 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:18:00.885895 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 12 18:18:00.886032 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:18:00.885952 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cd288a5-920c-4b68-ad97-27d449622108-insights-runtime-extractor-tls podName:1cd288a5-920c-4b68-ad97-27d449622108 nodeName:}" failed. No retries permitted until 2026-07-12 18:18:08.88593885 +0000 UTC m=+62.012843637 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1cd288a5-920c-4b68-ad97-27d449622108-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6s2lc" (UID: "1cd288a5-920c-4b68-ad97-27d449622108") : secret "insights-runtime-extractor-tls" not found Jul 12 18:18:08.953746 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:08.953554 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1cd288a5-920c-4b68-ad97-27d449622108-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6s2lc\" (UID: \"1cd288a5-920c-4b68-ad97-27d449622108\") " pod="openshift-insights/insights-runtime-extractor-6s2lc" Jul 12 18:18:08.953746 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:08.953736 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1cd288a5-920c-4b68-ad97-27d449622108-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6s2lc\" (UID: \"1cd288a5-920c-4b68-ad97-27d449622108\") " pod="openshift-insights/insights-runtime-extractor-6s2lc" Jul 12 18:18:08.954406 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:18:08.954041 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 12 18:18:08.954406 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:18:08.954123 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cd288a5-920c-4b68-ad97-27d449622108-insights-runtime-extractor-tls podName:1cd288a5-920c-4b68-ad97-27d449622108 nodeName:}" failed. No retries permitted until 2026-07-12 18:18:24.954103215 +0000 UTC m=+78.081008012 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1cd288a5-920c-4b68-ad97-27d449622108-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6s2lc" (UID: "1cd288a5-920c-4b68-ad97-27d449622108") : secret "insights-runtime-extractor-tls" not found Jul 12 18:18:08.954406 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:08.954229 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1cd288a5-920c-4b68-ad97-27d449622108-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6s2lc\" (UID: \"1cd288a5-920c-4b68-ad97-27d449622108\") " pod="openshift-insights/insights-runtime-extractor-6s2lc" Jul 12 18:18:09.369063 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:09.368991 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-spzqd" Jul 12 18:18:13.383185 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:13.383142 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d69e388-ddc9-4dd8-b5dc-a81a59dc1698-metrics-certs\") pod \"network-metrics-daemon-r66xp\" (UID: \"7d69e388-ddc9-4dd8-b5dc-a81a59dc1698\") " pod="openshift-multus/network-metrics-daemon-r66xp" Jul 12 18:18:13.383612 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:13.383234 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9f8419c6-07be-4876-9762-d98cd772054b-original-pull-secret\") pod \"global-pull-secret-syncer-2598z\" (UID: \"9f8419c6-07be-4876-9762-d98cd772054b\") " pod="kube-system/global-pull-secret-syncer-2598z" Jul 12 18:18:13.385817 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:13.385792 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 12 18:18:13.385893 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:13.385792 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 12 18:18:13.396087 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:13.396057 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d69e388-ddc9-4dd8-b5dc-a81a59dc1698-metrics-certs\") pod \"network-metrics-daemon-r66xp\" (UID: \"7d69e388-ddc9-4dd8-b5dc-a81a59dc1698\") " pod="openshift-multus/network-metrics-daemon-r66xp" Jul 12 18:18:13.396195 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:13.396055 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9f8419c6-07be-4876-9762-d98cd772054b-original-pull-secret\") pod \"global-pull-secret-syncer-2598z\" (UID: \"9f8419c6-07be-4876-9762-d98cd772054b\") " pod="kube-system/global-pull-secret-syncer-2598z" Jul 12 18:18:13.483931 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:13.483895 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpzmv\" (UniqueName: \"kubernetes.io/projected/4d1fc6eb-09af-4f96-8ec8-b193da43354c-kube-api-access-tpzmv\") pod \"network-check-target-lfnkm\" (UID: \"4d1fc6eb-09af-4f96-8ec8-b193da43354c\") " pod="openshift-network-diagnostics/network-check-target-lfnkm" Jul 12 18:18:13.486518 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:13.486500 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 12 18:18:13.496513 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:13.496495 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 12 18:18:13.507788 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:13.507753 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpzmv\" (UniqueName: \"kubernetes.io/projected/4d1fc6eb-09af-4f96-8ec8-b193da43354c-kube-api-access-tpzmv\") pod \"network-check-target-lfnkm\" (UID: \"4d1fc6eb-09af-4f96-8ec8-b193da43354c\") " pod="openshift-network-diagnostics/network-check-target-lfnkm" Jul 12 18:18:13.679203 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:13.679116 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2598z" Jul 12 18:18:13.687545 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:13.687522 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-p9cbp\"" Jul 12 18:18:13.691402 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:13.691383 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hb5mg\"" Jul 12 18:18:13.696452 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:13.696430 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r66xp" Jul 12 18:18:13.699452 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:13.699419 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lfnkm" Jul 12 18:18:13.825875 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:13.825842 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2598z"] Jul 12 18:18:13.828999 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:18:13.828973 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f8419c6_07be_4876_9762_d98cd772054b.slice/crio-6d037744be7d590fff3dfc22024636727ddce8cc2ba968c83bd6b5a350aacbc2 WatchSource:0}: Error finding container 6d037744be7d590fff3dfc22024636727ddce8cc2ba968c83bd6b5a350aacbc2: Status 404 returned error can't find the container with id 6d037744be7d590fff3dfc22024636727ddce8cc2ba968c83bd6b5a350aacbc2 Jul 12 18:18:14.051177 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:14.051113 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lfnkm"] Jul 12 18:18:14.052189 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:14.052165 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-r66xp"] Jul 12 18:18:14.055120 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:18:14.055097 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d1fc6eb_09af_4f96_8ec8_b193da43354c.slice/crio-eef5425dd9294c82dcc8fd5f3837ff01ac2770c30df9077b374d4079963b6ee3 WatchSource:0}: Error finding container eef5425dd9294c82dcc8fd5f3837ff01ac2770c30df9077b374d4079963b6ee3: Status 404 returned error can't find the container with id eef5425dd9294c82dcc8fd5f3837ff01ac2770c30df9077b374d4079963b6ee3 Jul 12 18:18:14.055664 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:18:14.055649 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d69e388_ddc9_4dd8_b5dc_a81a59dc1698.slice/crio-27770164fac471b65c5b38f33a3170b6bd57dad243e6639fbea7aba9ad90505b WatchSource:0}: Error finding container 27770164fac471b65c5b38f33a3170b6bd57dad243e6639fbea7aba9ad90505b: Status 404 returned error can't find the container with id 27770164fac471b65c5b38f33a3170b6bd57dad243e6639fbea7aba9ad90505b Jul 12 18:18:14.469063 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:14.469007 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-r66xp" event={"ID":"7d69e388-ddc9-4dd8-b5dc-a81a59dc1698","Type":"ContainerStarted","Data":"27770164fac471b65c5b38f33a3170b6bd57dad243e6639fbea7aba9ad90505b"} Jul 12 18:18:14.470456 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:14.470424 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2598z" event={"ID":"9f8419c6-07be-4876-9762-d98cd772054b","Type":"ContainerStarted","Data":"6d037744be7d590fff3dfc22024636727ddce8cc2ba968c83bd6b5a350aacbc2"} Jul 12 18:18:14.471821 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:14.471793 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lfnkm" event={"ID":"4d1fc6eb-09af-4f96-8ec8-b193da43354c","Type":"ContainerStarted","Data":"eef5425dd9294c82dcc8fd5f3837ff01ac2770c30df9077b374d4079963b6ee3"} Jul 12 18:18:15.478711 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:15.478656 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-r66xp" event={"ID":"7d69e388-ddc9-4dd8-b5dc-a81a59dc1698","Type":"ContainerStarted","Data":"21786b46b0665fbc7bf462f68918df2f849bc970ebd6e05eebee58e559f93f24"} Jul 12 18:18:15.478711 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:15.478711 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-r66xp" event={"ID":"7d69e388-ddc9-4dd8-b5dc-a81a59dc1698","Type":"ContainerStarted","Data":"996f26f23c507f60ff1218571b329e3cba92f0d29c3794f7fab3db991723eff4"} Jul 12 18:18:15.496375 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:15.496309 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-r66xp" podStartSLOduration=65.548088709 podStartE2EDuration="1m6.496287775s" totalImagesPullingTime="948.199066ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:17:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:18:15.495015294 +0000 UTC m=+68.621920099" watchObservedRunningTime="2026-07-12 18:18:15.496287775 +0000 UTC m=+68.623192624" Jul 12 18:18:16.611108 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:16.611072 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e812eecd-24a1-443b-aee0-0372c0324c56-metrics-tls\") pod \"dns-default-ld5vx\" (UID: \"e812eecd-24a1-443b-aee0-0372c0324c56\") " pod="openshift-dns/dns-default-ld5vx" Jul 12 18:18:16.611546 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:16.611153 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-registry-tls\") pod \"image-registry-7bc8fbd9f8-mc6xx\" (UID: \"d5113b21-a815-40a5-8b61-0c3b0e8c58ca\") " pod="openshift-image-registry/image-registry-7bc8fbd9f8-mc6xx" Jul 12 18:18:16.613658 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:16.613612 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e812eecd-24a1-443b-aee0-0372c0324c56-metrics-tls\") pod \"dns-default-ld5vx\" (UID: \"e812eecd-24a1-443b-aee0-0372c0324c56\") " pod="openshift-dns/dns-default-ld5vx" Jul 12 18:18:16.613791 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:16.613723 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-registry-tls\") pod \"image-registry-7bc8fbd9f8-mc6xx\" (UID: \"d5113b21-a815-40a5-8b61-0c3b0e8c58ca\") " pod="openshift-image-registry/image-registry-7bc8fbd9f8-mc6xx" Jul 12 18:18:16.856493 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:16.856463 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-p9vq2\"" Jul 12 18:18:16.864596 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:16.864517 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7bc8fbd9f8-mc6xx" Jul 12 18:18:16.866803 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:16.866781 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-h7xmt\"" Jul 12 18:18:16.874719 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:16.874691 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ld5vx" Jul 12 18:18:18.160045 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:18.160001 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7bc8fbd9f8-mc6xx"] Jul 12 18:18:19.133483 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:19.133452 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ld5vx"] Jul 12 18:18:19.137353 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:18:19.137309 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode812eecd_24a1_443b_aee0_0372c0324c56.slice/crio-a49ace6d33428db12529e7d9c6df551408f61de5cc41782d76ae265b594f1c3d WatchSource:0}: Error finding container a49ace6d33428db12529e7d9c6df551408f61de5cc41782d76ae265b594f1c3d: Status 404 returned error can't find the container with id a49ace6d33428db12529e7d9c6df551408f61de5cc41782d76ae265b594f1c3d Jul 12 18:18:19.160220 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:19.160197 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7bc8fbd9f8-mc6xx"] Jul 12 18:18:19.162769 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:18:19.162747 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5113b21_a815_40a5_8b61_0c3b0e8c58ca.slice/crio-bab75a3ca2bf00ca3bd9bb67a9b9ec844bc8723ff970cde9ecb8fe56ec245985 WatchSource:0}: Error finding container bab75a3ca2bf00ca3bd9bb67a9b9ec844bc8723ff970cde9ecb8fe56ec245985: Status 404 returned error can't find the container with id bab75a3ca2bf00ca3bd9bb67a9b9ec844bc8723ff970cde9ecb8fe56ec245985 Jul 12 18:18:19.494040 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:19.493958 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lfnkm" event={"ID":"4d1fc6eb-09af-4f96-8ec8-b193da43354c","Type":"ContainerStarted","Data":"269a8fe0a2ee7ee9178e728a4bd8a4ee7452cedaa6991e88202f617c0aeabfe4"} Jul 12 18:18:19.494203 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:19.494077 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-lfnkm" Jul 12 18:18:19.495396 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:19.495356 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ld5vx" event={"ID":"e812eecd-24a1-443b-aee0-0372c0324c56","Type":"ContainerStarted","Data":"a49ace6d33428db12529e7d9c6df551408f61de5cc41782d76ae265b594f1c3d"} Jul 12 18:18:19.497359 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:19.497255 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7bc8fbd9f8-mc6xx" event={"ID":"d5113b21-a815-40a5-8b61-0c3b0e8c58ca","Type":"ContainerStarted","Data":"e74014e3c73427ea56855d5c95ab81c8336b333edd462927ac300a9f9a8d16cb"} Jul 12 18:18:19.497359 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:19.497303 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7bc8fbd9f8-mc6xx" event={"ID":"d5113b21-a815-40a5-8b61-0c3b0e8c58ca","Type":"ContainerStarted","Data":"bab75a3ca2bf00ca3bd9bb67a9b9ec844bc8723ff970cde9ecb8fe56ec245985"} Jul 12 18:18:19.497359 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:19.497329 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7bc8fbd9f8-mc6xx" Jul 12 18:18:19.498992 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:19.498957 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2598z" event={"ID":"9f8419c6-07be-4876-9762-d98cd772054b","Type":"ContainerStarted","Data":"949532f5d7cf9f8c4e4ab14b15a9d5876d8fcfcd3f44a1d8c5c68d195adb1932"} Jul 12 18:18:19.515456 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:19.515381 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-lfnkm" podStartSLOduration=65.574804548 podStartE2EDuration="1m10.515365134s" totalImagesPullingTime="4.940560586s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:17:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:18:19.513845521 +0000 UTC m=+72.640750327" watchObservedRunningTime="2026-07-12 18:18:19.515365134 +0000 UTC m=+72.642269939" Jul 12 18:18:19.529295 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:19.529238 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-2598z" podStartSLOduration=65.345386927 podStartE2EDuration="1m10.529218191s" totalImagesPullingTime="5.183831264s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 18:17:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:18:19.527409331 +0000 UTC m=+72.654314137" watchObservedRunningTime="2026-07-12 18:18:19.529218191 +0000 UTC m=+72.656122996" Jul 12 18:18:19.546261 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:19.546204 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7bc8fbd9f8-mc6xx" podStartSLOduration=72.546188274 podStartE2EDuration="1m12.546188274s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:17:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:18:19.545614316 +0000 UTC m=+72.672519119" watchObservedRunningTime="2026-07-12 18:18:19.546188274 +0000 UTC m=+72.673093078" Jul 12 18:18:21.510306 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:21.510270 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ld5vx" event={"ID":"e812eecd-24a1-443b-aee0-0372c0324c56","Type":"ContainerStarted","Data":"a3975dbb692369a1e2f13a1ddbe7fe01cf2c98283746f5b892ce15514c4b30f1"} Jul 12 18:18:21.510760 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:21.510327 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ld5vx" event={"ID":"e812eecd-24a1-443b-aee0-0372c0324c56","Type":"ContainerStarted","Data":"fcb9441b75d038721ff9ea383f681846c51f00fb5e11285fcd749a4806ee0400"} Jul 12 18:18:21.510760 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:21.510366 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-ld5vx" Jul 12 18:18:21.527188 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:21.527133 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-ld5vx" podStartSLOduration=36.051058696 podStartE2EDuration="37.527113006s" totalImagesPullingTime="1.47605431s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:17:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:18:21.526016867 +0000 UTC m=+74.652921684" watchObservedRunningTime="2026-07-12 18:18:21.527113006 +0000 UTC m=+74.654017812" Jul 12 18:18:24.972153 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:24.972117 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1cd288a5-920c-4b68-ad97-27d449622108-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6s2lc\" (UID: \"1cd288a5-920c-4b68-ad97-27d449622108\") " pod="openshift-insights/insights-runtime-extractor-6s2lc" Jul 12 18:18:24.974340 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:24.974322 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1cd288a5-920c-4b68-ad97-27d449622108-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6s2lc\" (UID: \"1cd288a5-920c-4b68-ad97-27d449622108\") " pod="openshift-insights/insights-runtime-extractor-6s2lc" Jul 12 18:18:25.261438 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:25.261358 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9cwfc\"" Jul 12 18:18:25.269172 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:25.269151 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6s2lc" Jul 12 18:18:25.393364 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:25.393330 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6s2lc"] Jul 12 18:18:25.396142 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:18:25.396111 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cd288a5_920c_4b68_ad97_27d449622108.slice/crio-f0901dbd0420c89adcd8ef3ca71eff91bcc2d63bb7a281bc0fef18fe068ef86c WatchSource:0}: Error finding container f0901dbd0420c89adcd8ef3ca71eff91bcc2d63bb7a281bc0fef18fe068ef86c: Status 404 returned error can't find the container with id f0901dbd0420c89adcd8ef3ca71eff91bcc2d63bb7a281bc0fef18fe068ef86c Jul 12 18:18:25.525441 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:25.525364 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6s2lc" event={"ID":"1cd288a5-920c-4b68-ad97-27d449622108","Type":"ContainerStarted","Data":"f0901dbd0420c89adcd8ef3ca71eff91bcc2d63bb7a281bc0fef18fe068ef86c"} Jul 12 18:18:29.542175 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:29.542127 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6s2lc" event={"ID":"1cd288a5-920c-4b68-ad97-27d449622108","Type":"ContainerStarted","Data":"613994121c3b697476179e7fcecdd895fc2944457dfe9b1a0080ddffc389d1bc"} Jul 12 18:18:30.178932 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.178895 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-22qtq"] Jul 12 18:18:30.183492 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.183467 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-22qtq" Jul 12 18:18:30.187693 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.187671 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 12 18:18:30.188400 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.187892 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-ss2pq\"" Jul 12 18:18:30.188400 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.188107 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 12 18:18:30.188400 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.188264 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 12 18:18:30.190673 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.189260 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 12 18:18:30.190673 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.189445 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 12 18:18:30.190673 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.189652 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 12 18:18:30.206886 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.206861 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-wvfwn"] Jul 12 18:18:30.212426 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.212411 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-wvfwn" Jul 12 18:18:30.215351 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.215332 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 12 18:18:30.215922 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.215879 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 12 18:18:30.216067 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.216037 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-kgmw5\"" Jul 12 18:18:30.216566 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.216553 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 12 18:18:30.220012 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.219835 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-wvfwn"] Jul 12 18:18:30.309714 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.309605 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ea7d5a92-5fad-43db-b210-956555ed01e8-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-wvfwn\" (UID: \"ea7d5a92-5fad-43db-b210-956555ed01e8\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wvfwn" Jul 12 18:18:30.309714 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.309713 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ea7d5a92-5fad-43db-b210-956555ed01e8-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-wvfwn\" (UID: \"ea7d5a92-5fad-43db-b210-956555ed01e8\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wvfwn" Jul 12 18:18:30.309966 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.309744 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhbhf\" (UniqueName: \"kubernetes.io/projected/04f3aab6-b724-4e1c-95b4-1a85691a5cca-kube-api-access-dhbhf\") pod \"node-exporter-22qtq\" (UID: \"04f3aab6-b724-4e1c-95b4-1a85691a5cca\") " pod="openshift-monitoring/node-exporter-22qtq" Jul 12 18:18:30.309966 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.309778 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/04f3aab6-b724-4e1c-95b4-1a85691a5cca-metrics-client-ca\") pod \"node-exporter-22qtq\" (UID: \"04f3aab6-b724-4e1c-95b4-1a85691a5cca\") " pod="openshift-monitoring/node-exporter-22qtq" Jul 12 18:18:30.309966 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.309807 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ea7d5a92-5fad-43db-b210-956555ed01e8-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-wvfwn\" (UID: \"ea7d5a92-5fad-43db-b210-956555ed01e8\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wvfwn" Jul 12 18:18:30.309966 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.309828 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/04f3aab6-b724-4e1c-95b4-1a85691a5cca-root\") pod \"node-exporter-22qtq\" (UID: \"04f3aab6-b724-4e1c-95b4-1a85691a5cca\") " pod="openshift-monitoring/node-exporter-22qtq" Jul 12 18:18:30.309966 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.309853 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/04f3aab6-b724-4e1c-95b4-1a85691a5cca-node-exporter-tls\") pod \"node-exporter-22qtq\" (UID: \"04f3aab6-b724-4e1c-95b4-1a85691a5cca\") " pod="openshift-monitoring/node-exporter-22qtq" Jul 12 18:18:30.310228 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.309970 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/04f3aab6-b724-4e1c-95b4-1a85691a5cca-sys\") pod \"node-exporter-22qtq\" (UID: \"04f3aab6-b724-4e1c-95b4-1a85691a5cca\") " pod="openshift-monitoring/node-exporter-22qtq" Jul 12 18:18:30.310228 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.310014 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ea7d5a92-5fad-43db-b210-956555ed01e8-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-wvfwn\" (UID: \"ea7d5a92-5fad-43db-b210-956555ed01e8\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wvfwn" Jul 12 18:18:30.310228 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.310044 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4x82n\" (UniqueName: \"kubernetes.io/projected/ea7d5a92-5fad-43db-b210-956555ed01e8-kube-api-access-4x82n\") pod \"kube-state-metrics-785778ff74-wvfwn\" (UID: \"ea7d5a92-5fad-43db-b210-956555ed01e8\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wvfwn" Jul 12 18:18:30.310228 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.310108 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/04f3aab6-b724-4e1c-95b4-1a85691a5cca-node-exporter-accelerators-collector-config\") pod \"node-exporter-22qtq\" (UID: \"04f3aab6-b724-4e1c-95b4-1a85691a5cca\") " pod="openshift-monitoring/node-exporter-22qtq" Jul 12 18:18:30.310228 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.310138 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/04f3aab6-b724-4e1c-95b4-1a85691a5cca-node-exporter-wtmp\") pod \"node-exporter-22qtq\" (UID: \"04f3aab6-b724-4e1c-95b4-1a85691a5cca\") " pod="openshift-monitoring/node-exporter-22qtq" Jul 12 18:18:30.310228 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.310168 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/04f3aab6-b724-4e1c-95b4-1a85691a5cca-node-exporter-textfile\") pod \"node-exporter-22qtq\" (UID: \"04f3aab6-b724-4e1c-95b4-1a85691a5cca\") " pod="openshift-monitoring/node-exporter-22qtq" Jul 12 18:18:30.310228 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.310194 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/04f3aab6-b724-4e1c-95b4-1a85691a5cca-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-22qtq\" (UID: \"04f3aab6-b724-4e1c-95b4-1a85691a5cca\") " pod="openshift-monitoring/node-exporter-22qtq" Jul 12 18:18:30.310497 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.310230 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ea7d5a92-5fad-43db-b210-956555ed01e8-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-wvfwn\" (UID: \"ea7d5a92-5fad-43db-b210-956555ed01e8\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wvfwn" Jul 12 18:18:30.411479 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.411441 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ea7d5a92-5fad-43db-b210-956555ed01e8-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-wvfwn\" (UID: \"ea7d5a92-5fad-43db-b210-956555ed01e8\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wvfwn" Jul 12 18:18:30.411668 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.411486 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4x82n\" (UniqueName: \"kubernetes.io/projected/ea7d5a92-5fad-43db-b210-956555ed01e8-kube-api-access-4x82n\") pod \"kube-state-metrics-785778ff74-wvfwn\" (UID: \"ea7d5a92-5fad-43db-b210-956555ed01e8\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wvfwn" Jul 12 18:18:30.411668 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.411524 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/04f3aab6-b724-4e1c-95b4-1a85691a5cca-node-exporter-accelerators-collector-config\") pod \"node-exporter-22qtq\" (UID: \"04f3aab6-b724-4e1c-95b4-1a85691a5cca\") " pod="openshift-monitoring/node-exporter-22qtq" Jul 12 18:18:30.411668 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.411569 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/04f3aab6-b724-4e1c-95b4-1a85691a5cca-node-exporter-wtmp\") pod \"node-exporter-22qtq\" (UID: \"04f3aab6-b724-4e1c-95b4-1a85691a5cca\") " pod="openshift-monitoring/node-exporter-22qtq" Jul 12 18:18:30.411668 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.411600 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/04f3aab6-b724-4e1c-95b4-1a85691a5cca-node-exporter-textfile\") pod \"node-exporter-22qtq\" (UID: \"04f3aab6-b724-4e1c-95b4-1a85691a5cca\") " pod="openshift-monitoring/node-exporter-22qtq" Jul 12 18:18:30.411668 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.411660 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/04f3aab6-b724-4e1c-95b4-1a85691a5cca-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-22qtq\" (UID: \"04f3aab6-b724-4e1c-95b4-1a85691a5cca\") " pod="openshift-monitoring/node-exporter-22qtq" Jul 12 18:18:30.411933 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.411694 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ea7d5a92-5fad-43db-b210-956555ed01e8-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-wvfwn\" (UID: \"ea7d5a92-5fad-43db-b210-956555ed01e8\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wvfwn" Jul 12 18:18:30.411933 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.411726 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ea7d5a92-5fad-43db-b210-956555ed01e8-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-wvfwn\" (UID: \"ea7d5a92-5fad-43db-b210-956555ed01e8\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wvfwn" Jul 12 18:18:30.411933 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.411748 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ea7d5a92-5fad-43db-b210-956555ed01e8-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-wvfwn\" (UID: \"ea7d5a92-5fad-43db-b210-956555ed01e8\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wvfwn" Jul 12 18:18:30.411933 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.411773 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhbhf\" (UniqueName: \"kubernetes.io/projected/04f3aab6-b724-4e1c-95b4-1a85691a5cca-kube-api-access-dhbhf\") pod \"node-exporter-22qtq\" (UID: \"04f3aab6-b724-4e1c-95b4-1a85691a5cca\") " pod="openshift-monitoring/node-exporter-22qtq" Jul 12 18:18:30.411933 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.411803 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/04f3aab6-b724-4e1c-95b4-1a85691a5cca-metrics-client-ca\") pod \"node-exporter-22qtq\" (UID: \"04f3aab6-b724-4e1c-95b4-1a85691a5cca\") " pod="openshift-monitoring/node-exporter-22qtq" Jul 12 18:18:30.411933 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.411829 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ea7d5a92-5fad-43db-b210-956555ed01e8-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-wvfwn\" (UID: \"ea7d5a92-5fad-43db-b210-956555ed01e8\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wvfwn" Jul 12 18:18:30.411933 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.411850 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/04f3aab6-b724-4e1c-95b4-1a85691a5cca-root\") pod \"node-exporter-22qtq\" (UID: \"04f3aab6-b724-4e1c-95b4-1a85691a5cca\") " pod="openshift-monitoring/node-exporter-22qtq" Jul 12 18:18:30.411933 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.411910 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/04f3aab6-b724-4e1c-95b4-1a85691a5cca-node-exporter-tls\") pod \"node-exporter-22qtq\" (UID: \"04f3aab6-b724-4e1c-95b4-1a85691a5cca\") " pod="openshift-monitoring/node-exporter-22qtq" Jul 12 18:18:30.412319 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.411942 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/04f3aab6-b724-4e1c-95b4-1a85691a5cca-sys\") pod \"node-exporter-22qtq\" (UID: \"04f3aab6-b724-4e1c-95b4-1a85691a5cca\") " pod="openshift-monitoring/node-exporter-22qtq" Jul 12 18:18:30.412319 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.412026 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/04f3aab6-b724-4e1c-95b4-1a85691a5cca-sys\") pod \"node-exporter-22qtq\" (UID: \"04f3aab6-b724-4e1c-95b4-1a85691a5cca\") " pod="openshift-monitoring/node-exporter-22qtq" Jul 12 18:18:30.412791 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.412767 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ea7d5a92-5fad-43db-b210-956555ed01e8-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-wvfwn\" (UID: \"ea7d5a92-5fad-43db-b210-956555ed01e8\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wvfwn" Jul 12 18:18:30.413606 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.413585 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/04f3aab6-b724-4e1c-95b4-1a85691a5cca-node-exporter-accelerators-collector-config\") pod \"node-exporter-22qtq\" (UID: \"04f3aab6-b724-4e1c-95b4-1a85691a5cca\") " pod="openshift-monitoring/node-exporter-22qtq" Jul 12 18:18:30.413753 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.413737 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/04f3aab6-b724-4e1c-95b4-1a85691a5cca-node-exporter-wtmp\") pod \"node-exporter-22qtq\" (UID: \"04f3aab6-b724-4e1c-95b4-1a85691a5cca\") " pod="openshift-monitoring/node-exporter-22qtq" Jul 12 18:18:30.413972 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.413955 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/04f3aab6-b724-4e1c-95b4-1a85691a5cca-node-exporter-textfile\") pod \"node-exporter-22qtq\" (UID: \"04f3aab6-b724-4e1c-95b4-1a85691a5cca\") " pod="openshift-monitoring/node-exporter-22qtq" Jul 12 18:18:30.415466 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:18:30.414446 2577 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 12 18:18:30.415466 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:18:30.414516 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea7d5a92-5fad-43db-b210-956555ed01e8-kube-state-metrics-tls podName:ea7d5a92-5fad-43db-b210-956555ed01e8 nodeName:}" failed. No retries permitted until 2026-07-12 18:18:30.914497107 +0000 UTC m=+84.041401894 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/ea7d5a92-5fad-43db-b210-956555ed01e8-kube-state-metrics-tls") pod "kube-state-metrics-785778ff74-wvfwn" (UID: "ea7d5a92-5fad-43db-b210-956555ed01e8") : secret "kube-state-metrics-tls" not found Jul 12 18:18:30.415466 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.415155 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/04f3aab6-b724-4e1c-95b4-1a85691a5cca-metrics-client-ca\") pod \"node-exporter-22qtq\" (UID: \"04f3aab6-b724-4e1c-95b4-1a85691a5cca\") " pod="openshift-monitoring/node-exporter-22qtq" Jul 12 18:18:30.415466 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.415217 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/04f3aab6-b724-4e1c-95b4-1a85691a5cca-root\") pod \"node-exporter-22qtq\" (UID: \"04f3aab6-b724-4e1c-95b4-1a85691a5cca\") " pod="openshift-monitoring/node-exporter-22qtq" Jul 12 18:18:30.415772 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.415494 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ea7d5a92-5fad-43db-b210-956555ed01e8-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-wvfwn\" (UID: \"ea7d5a92-5fad-43db-b210-956555ed01e8\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wvfwn" Jul 12 18:18:30.416134 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.416097 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ea7d5a92-5fad-43db-b210-956555ed01e8-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-wvfwn\" (UID: \"ea7d5a92-5fad-43db-b210-956555ed01e8\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wvfwn" Jul 12 18:18:30.419064 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.418717 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/04f3aab6-b724-4e1c-95b4-1a85691a5cca-node-exporter-tls\") pod \"node-exporter-22qtq\" (UID: \"04f3aab6-b724-4e1c-95b4-1a85691a5cca\") " pod="openshift-monitoring/node-exporter-22qtq" Jul 12 18:18:30.420257 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.420235 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/04f3aab6-b724-4e1c-95b4-1a85691a5cca-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-22qtq\" (UID: \"04f3aab6-b724-4e1c-95b4-1a85691a5cca\") " pod="openshift-monitoring/node-exporter-22qtq" Jul 12 18:18:30.422043 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.421982 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4x82n\" (UniqueName: \"kubernetes.io/projected/ea7d5a92-5fad-43db-b210-956555ed01e8-kube-api-access-4x82n\") pod \"kube-state-metrics-785778ff74-wvfwn\" (UID: \"ea7d5a92-5fad-43db-b210-956555ed01e8\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wvfwn" Jul 12 18:18:30.423374 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.423329 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhbhf\" (UniqueName: \"kubernetes.io/projected/04f3aab6-b724-4e1c-95b4-1a85691a5cca-kube-api-access-dhbhf\") pod \"node-exporter-22qtq\" (UID: \"04f3aab6-b724-4e1c-95b4-1a85691a5cca\") " pod="openshift-monitoring/node-exporter-22qtq" Jul 12 18:18:30.437291 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.437268 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ea7d5a92-5fad-43db-b210-956555ed01e8-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-wvfwn\" (UID: \"ea7d5a92-5fad-43db-b210-956555ed01e8\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wvfwn" Jul 12 18:18:30.498321 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.498264 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-22qtq" Jul 12 18:18:30.514900 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:18:30.514869 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04f3aab6_b724_4e1c_95b4_1a85691a5cca.slice/crio-50349bdc1425ffee2686a5945465a071884a9d968010afb818c310d03ed0c187 WatchSource:0}: Error finding container 50349bdc1425ffee2686a5945465a071884a9d968010afb818c310d03ed0c187: Status 404 returned error can't find the container with id 50349bdc1425ffee2686a5945465a071884a9d968010afb818c310d03ed0c187 Jul 12 18:18:30.548293 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.548221 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-22qtq" event={"ID":"04f3aab6-b724-4e1c-95b4-1a85691a5cca","Type":"ContainerStarted","Data":"50349bdc1425ffee2686a5945465a071884a9d968010afb818c310d03ed0c187"} Jul 12 18:18:30.551482 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.551451 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6s2lc" event={"ID":"1cd288a5-920c-4b68-ad97-27d449622108","Type":"ContainerStarted","Data":"2d22ba1d8ac89acf68011b117a426a1d7ea230e87bc030e925eb72d8e11f1340"} Jul 12 18:18:30.916723 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.916622 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ea7d5a92-5fad-43db-b210-956555ed01e8-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-wvfwn\" (UID: \"ea7d5a92-5fad-43db-b210-956555ed01e8\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wvfwn" Jul 12 18:18:30.919614 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:30.919582 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ea7d5a92-5fad-43db-b210-956555ed01e8-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-wvfwn\" (UID: \"ea7d5a92-5fad-43db-b210-956555ed01e8\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wvfwn" Jul 12 18:18:31.128803 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:31.128775 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-wvfwn" Jul 12 18:18:31.309267 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:31.309235 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-wvfwn"] Jul 12 18:18:31.517109 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:31.517038 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-ld5vx" Jul 12 18:18:32.337104 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:18:32.337070 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea7d5a92_5fad_43db_b210_956555ed01e8.slice/crio-4e252ffad45b0bfbedc7789059b70f96f0570018820929afce15cd9776ab5d7f WatchSource:0}: Error finding container 4e252ffad45b0bfbedc7789059b70f96f0570018820929afce15cd9776ab5d7f: Status 404 returned error can't find the container with id 4e252ffad45b0bfbedc7789059b70f96f0570018820929afce15cd9776ab5d7f Jul 12 18:18:32.559704 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:32.559673 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-wvfwn" event={"ID":"ea7d5a92-5fad-43db-b210-956555ed01e8","Type":"ContainerStarted","Data":"4e252ffad45b0bfbedc7789059b70f96f0570018820929afce15cd9776ab5d7f"} Jul 12 18:18:33.565727 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:33.565680 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6s2lc" event={"ID":"1cd288a5-920c-4b68-ad97-27d449622108","Type":"ContainerStarted","Data":"f15cf548c03fc817ceb1dd8a4affd6f7aa3fa558ec5cee3589ddd74eba03f277"} Jul 12 18:18:33.567225 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:33.567200 2577 generic.go:356] "Generic (PLEG): container finished" podID="04f3aab6-b724-4e1c-95b4-1a85691a5cca" containerID="d91c9d75aae33590fafac91b03eba87d18d4694b9b71d89e43beaa23a1bf6be1" exitCode=0 Jul 12 18:18:33.567343 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:33.567234 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-22qtq" event={"ID":"04f3aab6-b724-4e1c-95b4-1a85691a5cca","Type":"ContainerDied","Data":"d91c9d75aae33590fafac91b03eba87d18d4694b9b71d89e43beaa23a1bf6be1"} Jul 12 18:18:33.583369 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:33.583328 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-6s2lc" podStartSLOduration=33.505616331 podStartE2EDuration="40.583312131s" totalImagesPullingTime="7.0776958s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 18:17:53 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:18:33.581927157 +0000 UTC m=+86.708831963" watchObservedRunningTime="2026-07-12 18:18:33.583312131 +0000 UTC m=+86.710216935" Jul 12 18:18:34.473160 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:34.473123 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-59bb7dbd4-wj8zl"] Jul 12 18:18:34.476343 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:34.476323 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-59bb7dbd4-wj8zl" Jul 12 18:18:34.479067 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:34.479034 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 12 18:18:34.479969 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:34.479953 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-1oso69m921h2n\"" Jul 12 18:18:34.480046 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:34.479966 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 12 18:18:34.480153 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:34.480140 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 12 18:18:34.480272 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:34.480254 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-2pccp\"" Jul 12 18:18:34.480331 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:34.480294 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 12 18:18:34.484373 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:34.484351 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-59bb7dbd4-wj8zl"] Jul 12 18:18:34.572406 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:34.572367 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-wvfwn" event={"ID":"ea7d5a92-5fad-43db-b210-956555ed01e8","Type":"ContainerStarted","Data":"95f3731de6c9eaafe4e5de638abe246aaeaed3686797f69b3b1ff88b3eb45581"} Jul 12 18:18:34.572859 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:34.572434 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-wvfwn" event={"ID":"ea7d5a92-5fad-43db-b210-956555ed01e8","Type":"ContainerStarted","Data":"944d67fc2eda911e69f4a6f44f37c72f0e136f6b4ab282c16deeebe819235ab1"} Jul 12 18:18:34.572859 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:34.572466 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-wvfwn" event={"ID":"ea7d5a92-5fad-43db-b210-956555ed01e8","Type":"ContainerStarted","Data":"542653779aafeb46d87e0a999ce9ef971dc58d201ab84308af778bb2c4ef6135"} Jul 12 18:18:34.574494 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:34.574472 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-22qtq" event={"ID":"04f3aab6-b724-4e1c-95b4-1a85691a5cca","Type":"ContainerStarted","Data":"95fd87b414482ebec4f2f530f9cb306d45f342205461f35c96483e1e501a2e67"} Jul 12 18:18:34.574622 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:34.574511 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-22qtq" event={"ID":"04f3aab6-b724-4e1c-95b4-1a85691a5cca","Type":"ContainerStarted","Data":"b701352e3462f4f6ce85ee9210577c7cd33f14db8c832dc3d44d79506225bc1f"} Jul 12 18:18:34.592929 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:34.592879 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-wvfwn" podStartSLOduration=3.315240262 podStartE2EDuration="4.592862771s" totalImagesPullingTime="1.277622509s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:18:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:18:34.591964827 +0000 UTC m=+87.718869633" watchObservedRunningTime="2026-07-12 18:18:34.592862771 +0000 UTC m=+87.719767577" Jul 12 18:18:34.610030 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:34.609983 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-22qtq" podStartSLOduration=2.482201602 podStartE2EDuration="4.609956342s" totalImagesPullingTime="2.063571735s" totalInitContainerRuntime="64.183005ms" isStatefulPod=true podCreationTimestamp="2026-07-12 18:18:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:18:34.608757221 +0000 UTC m=+87.735662027" watchObservedRunningTime="2026-07-12 18:18:34.609956342 +0000 UTC m=+87.736861177" Jul 12 18:18:34.648513 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:34.648466 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c2b013fb-cf46-487b-9ee4-0109234e9fd1-secret-metrics-server-tls\") pod \"metrics-server-59bb7dbd4-wj8zl\" (UID: \"c2b013fb-cf46-487b-9ee4-0109234e9fd1\") " pod="openshift-monitoring/metrics-server-59bb7dbd4-wj8zl" Jul 12 18:18:34.648702 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:34.648561 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c2b013fb-cf46-487b-9ee4-0109234e9fd1-secret-metrics-server-client-certs\") pod \"metrics-server-59bb7dbd4-wj8zl\" (UID: \"c2b013fb-cf46-487b-9ee4-0109234e9fd1\") " pod="openshift-monitoring/metrics-server-59bb7dbd4-wj8zl" Jul 12 18:18:34.648702 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:34.648579 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c2b013fb-cf46-487b-9ee4-0109234e9fd1-metrics-server-audit-profiles\") pod \"metrics-server-59bb7dbd4-wj8zl\" (UID: \"c2b013fb-cf46-487b-9ee4-0109234e9fd1\") " pod="openshift-monitoring/metrics-server-59bb7dbd4-wj8zl" Jul 12 18:18:34.648702 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:34.648611 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2b013fb-cf46-487b-9ee4-0109234e9fd1-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-59bb7dbd4-wj8zl\" (UID: \"c2b013fb-cf46-487b-9ee4-0109234e9fd1\") " pod="openshift-monitoring/metrics-server-59bb7dbd4-wj8zl" Jul 12 18:18:34.648813 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:34.648747 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkc68\" (UniqueName: \"kubernetes.io/projected/c2b013fb-cf46-487b-9ee4-0109234e9fd1-kube-api-access-zkc68\") pod \"metrics-server-59bb7dbd4-wj8zl\" (UID: \"c2b013fb-cf46-487b-9ee4-0109234e9fd1\") " pod="openshift-monitoring/metrics-server-59bb7dbd4-wj8zl" Jul 12 18:18:34.648813 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:34.648796 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c2b013fb-cf46-487b-9ee4-0109234e9fd1-client-ca-bundle\") pod \"metrics-server-59bb7dbd4-wj8zl\" (UID: \"c2b013fb-cf46-487b-9ee4-0109234e9fd1\") " pod="openshift-monitoring/metrics-server-59bb7dbd4-wj8zl" Jul 12 18:18:34.648930 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:34.648829 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c2b013fb-cf46-487b-9ee4-0109234e9fd1-audit-log\") pod \"metrics-server-59bb7dbd4-wj8zl\" (UID: \"c2b013fb-cf46-487b-9ee4-0109234e9fd1\") " pod="openshift-monitoring/metrics-server-59bb7dbd4-wj8zl" Jul 12 18:18:34.749871 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:34.749786 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c2b013fb-cf46-487b-9ee4-0109234e9fd1-secret-metrics-server-client-certs\") pod \"metrics-server-59bb7dbd4-wj8zl\" (UID: \"c2b013fb-cf46-487b-9ee4-0109234e9fd1\") " pod="openshift-monitoring/metrics-server-59bb7dbd4-wj8zl" Jul 12 18:18:34.749871 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:34.749833 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c2b013fb-cf46-487b-9ee4-0109234e9fd1-metrics-server-audit-profiles\") pod \"metrics-server-59bb7dbd4-wj8zl\" (UID: \"c2b013fb-cf46-487b-9ee4-0109234e9fd1\") " pod="openshift-monitoring/metrics-server-59bb7dbd4-wj8zl" Jul 12 18:18:34.750094 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:34.749881 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2b013fb-cf46-487b-9ee4-0109234e9fd1-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-59bb7dbd4-wj8zl\" (UID: \"c2b013fb-cf46-487b-9ee4-0109234e9fd1\") " pod="openshift-monitoring/metrics-server-59bb7dbd4-wj8zl" Jul 12 18:18:34.750094 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:34.749919 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkc68\" (UniqueName: \"kubernetes.io/projected/c2b013fb-cf46-487b-9ee4-0109234e9fd1-kube-api-access-zkc68\") pod \"metrics-server-59bb7dbd4-wj8zl\" (UID: \"c2b013fb-cf46-487b-9ee4-0109234e9fd1\") " pod="openshift-monitoring/metrics-server-59bb7dbd4-wj8zl" Jul 12 18:18:34.750094 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:34.749947 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c2b013fb-cf46-487b-9ee4-0109234e9fd1-client-ca-bundle\") pod \"metrics-server-59bb7dbd4-wj8zl\" (UID: \"c2b013fb-cf46-487b-9ee4-0109234e9fd1\") " pod="openshift-monitoring/metrics-server-59bb7dbd4-wj8zl" Jul 12 18:18:34.750094 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:34.749977 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c2b013fb-cf46-487b-9ee4-0109234e9fd1-audit-log\") pod \"metrics-server-59bb7dbd4-wj8zl\" (UID: \"c2b013fb-cf46-487b-9ee4-0109234e9fd1\") " pod="openshift-monitoring/metrics-server-59bb7dbd4-wj8zl" Jul 12 18:18:34.750094 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:34.750055 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c2b013fb-cf46-487b-9ee4-0109234e9fd1-secret-metrics-server-tls\") pod \"metrics-server-59bb7dbd4-wj8zl\" (UID: \"c2b013fb-cf46-487b-9ee4-0109234e9fd1\") " pod="openshift-monitoring/metrics-server-59bb7dbd4-wj8zl" Jul 12 18:18:34.750899 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:34.750710 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c2b013fb-cf46-487b-9ee4-0109234e9fd1-audit-log\") pod \"metrics-server-59bb7dbd4-wj8zl\" (UID: \"c2b013fb-cf46-487b-9ee4-0109234e9fd1\") " pod="openshift-monitoring/metrics-server-59bb7dbd4-wj8zl" Jul 12 18:18:34.750899 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:34.750707 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2b013fb-cf46-487b-9ee4-0109234e9fd1-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-59bb7dbd4-wj8zl\" (UID: \"c2b013fb-cf46-487b-9ee4-0109234e9fd1\") " pod="openshift-monitoring/metrics-server-59bb7dbd4-wj8zl" Jul 12 18:18:34.751099 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:34.750987 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c2b013fb-cf46-487b-9ee4-0109234e9fd1-metrics-server-audit-profiles\") pod \"metrics-server-59bb7dbd4-wj8zl\" (UID: \"c2b013fb-cf46-487b-9ee4-0109234e9fd1\") " pod="openshift-monitoring/metrics-server-59bb7dbd4-wj8zl" Jul 12 18:18:34.752414 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:34.752391 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c2b013fb-cf46-487b-9ee4-0109234e9fd1-secret-metrics-server-client-certs\") pod \"metrics-server-59bb7dbd4-wj8zl\" (UID: \"c2b013fb-cf46-487b-9ee4-0109234e9fd1\") " pod="openshift-monitoring/metrics-server-59bb7dbd4-wj8zl" Jul 12 18:18:34.752557 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:34.752539 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c2b013fb-cf46-487b-9ee4-0109234e9fd1-client-ca-bundle\") pod \"metrics-server-59bb7dbd4-wj8zl\" (UID: \"c2b013fb-cf46-487b-9ee4-0109234e9fd1\") " pod="openshift-monitoring/metrics-server-59bb7dbd4-wj8zl" Jul 12 18:18:34.752761 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:34.752741 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c2b013fb-cf46-487b-9ee4-0109234e9fd1-secret-metrics-server-tls\") pod \"metrics-server-59bb7dbd4-wj8zl\" (UID: \"c2b013fb-cf46-487b-9ee4-0109234e9fd1\") " pod="openshift-monitoring/metrics-server-59bb7dbd4-wj8zl" Jul 12 18:18:34.758057 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:34.758031 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkc68\" (UniqueName: \"kubernetes.io/projected/c2b013fb-cf46-487b-9ee4-0109234e9fd1-kube-api-access-zkc68\") pod \"metrics-server-59bb7dbd4-wj8zl\" (UID: \"c2b013fb-cf46-487b-9ee4-0109234e9fd1\") " pod="openshift-monitoring/metrics-server-59bb7dbd4-wj8zl" Jul 12 18:18:34.786024 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:34.785989 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-59bb7dbd4-wj8zl" Jul 12 18:18:34.940695 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:34.940665 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-59bb7dbd4-wj8zl"] Jul 12 18:18:34.944308 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:18:34.944272 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2b013fb_cf46_487b_9ee4_0109234e9fd1.slice/crio-03f17d21ec0a37fd58fb835f8daf127b12697ba43daef61f1e260ee3e45c6c44 WatchSource:0}: Error finding container 03f17d21ec0a37fd58fb835f8daf127b12697ba43daef61f1e260ee3e45c6c44: Status 404 returned error can't find the container with id 03f17d21ec0a37fd58fb835f8daf127b12697ba43daef61f1e260ee3e45c6c44 Jul 12 18:18:35.579077 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:35.579028 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-59bb7dbd4-wj8zl" event={"ID":"c2b013fb-cf46-487b-9ee4-0109234e9fd1","Type":"ContainerStarted","Data":"03f17d21ec0a37fd58fb835f8daf127b12697ba43daef61f1e260ee3e45c6c44"} Jul 12 18:18:36.584447 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:36.584356 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-59bb7dbd4-wj8zl" event={"ID":"c2b013fb-cf46-487b-9ee4-0109234e9fd1","Type":"ContainerStarted","Data":"fef3ff5390e8182b176ce7045ea82dd1ed28820658caf9d1cf9d28d0660c6b11"} Jul 12 18:18:36.600514 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:36.600468 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-59bb7dbd4-wj8zl" podStartSLOduration=1.247521302 podStartE2EDuration="2.600450847s" totalImagesPullingTime="1.352929545s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:18:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:18:36.599778855 +0000 UTC m=+89.726683659" watchObservedRunningTime="2026-07-12 18:18:36.600450847 +0000 UTC m=+89.727355652" Jul 12 18:18:39.503171 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:39.503144 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7bc8fbd9f8-mc6xx" Jul 12 18:18:39.696940 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:39.696910 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7945976fcb-jlwss"] Jul 12 18:18:39.701206 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:39.701189 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7945976fcb-jlwss" Jul 12 18:18:39.704279 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:39.704260 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 12 18:18:39.704369 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:39.704266 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 12 18:18:39.704369 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:39.704324 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-srn25\"" Jul 12 18:18:39.704711 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:39.704696 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 12 18:18:39.704802 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:39.704787 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 12 18:18:39.704854 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:39.704814 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 12 18:18:39.704890 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:39.704826 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 12 18:18:39.705411 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:39.705398 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 12 18:18:39.709608 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:39.709586 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7945976fcb-jlwss"] Jul 12 18:18:39.789531 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:39.789448 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c-console-config\") pod \"console-7945976fcb-jlwss\" (UID: \"6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c\") " pod="openshift-console/console-7945976fcb-jlwss" Jul 12 18:18:39.789531 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:39.789482 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c-oauth-serving-cert\") pod \"console-7945976fcb-jlwss\" (UID: \"6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c\") " pod="openshift-console/console-7945976fcb-jlwss" Jul 12 18:18:39.789531 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:39.789511 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c-console-serving-cert\") pod \"console-7945976fcb-jlwss\" (UID: \"6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c\") " pod="openshift-console/console-7945976fcb-jlwss" Jul 12 18:18:39.789531 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:39.789526 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrdkx\" (UniqueName: \"kubernetes.io/projected/6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c-kube-api-access-xrdkx\") pod \"console-7945976fcb-jlwss\" (UID: \"6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c\") " pod="openshift-console/console-7945976fcb-jlwss" Jul 12 18:18:39.789826 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:39.789653 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c-console-oauth-config\") pod \"console-7945976fcb-jlwss\" (UID: \"6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c\") " pod="openshift-console/console-7945976fcb-jlwss" Jul 12 18:18:39.789826 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:39.789688 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c-service-ca\") pod \"console-7945976fcb-jlwss\" (UID: \"6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c\") " pod="openshift-console/console-7945976fcb-jlwss" Jul 12 18:18:39.890835 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:39.890801 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c-console-serving-cert\") pod \"console-7945976fcb-jlwss\" (UID: \"6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c\") " pod="openshift-console/console-7945976fcb-jlwss" Jul 12 18:18:39.890835 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:39.890838 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrdkx\" (UniqueName: \"kubernetes.io/projected/6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c-kube-api-access-xrdkx\") pod \"console-7945976fcb-jlwss\" (UID: \"6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c\") " pod="openshift-console/console-7945976fcb-jlwss" Jul 12 18:18:39.891037 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:39.890899 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c-console-oauth-config\") pod \"console-7945976fcb-jlwss\" (UID: \"6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c\") " pod="openshift-console/console-7945976fcb-jlwss" Jul 12 18:18:39.891037 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:39.890929 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c-service-ca\") pod \"console-7945976fcb-jlwss\" (UID: \"6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c\") " pod="openshift-console/console-7945976fcb-jlwss" Jul 12 18:18:39.891037 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:39.890973 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c-console-config\") pod \"console-7945976fcb-jlwss\" (UID: \"6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c\") " pod="openshift-console/console-7945976fcb-jlwss" Jul 12 18:18:39.891037 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:39.890999 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c-oauth-serving-cert\") pod \"console-7945976fcb-jlwss\" (UID: \"6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c\") " pod="openshift-console/console-7945976fcb-jlwss" Jul 12 18:18:39.891741 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:39.891719 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c-console-config\") pod \"console-7945976fcb-jlwss\" (UID: \"6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c\") " pod="openshift-console/console-7945976fcb-jlwss" Jul 12 18:18:39.891856 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:39.891721 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c-oauth-serving-cert\") pod \"console-7945976fcb-jlwss\" (UID: \"6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c\") " pod="openshift-console/console-7945976fcb-jlwss" Jul 12 18:18:39.891856 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:39.891765 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c-service-ca\") pod \"console-7945976fcb-jlwss\" (UID: \"6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c\") " pod="openshift-console/console-7945976fcb-jlwss" Jul 12 18:18:39.893204 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:39.893182 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c-console-oauth-config\") pod \"console-7945976fcb-jlwss\" (UID: \"6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c\") " pod="openshift-console/console-7945976fcb-jlwss" Jul 12 18:18:39.893315 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:39.893261 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c-console-serving-cert\") pod \"console-7945976fcb-jlwss\" (UID: \"6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c\") " pod="openshift-console/console-7945976fcb-jlwss" Jul 12 18:18:39.898640 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:39.898599 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrdkx\" (UniqueName: \"kubernetes.io/projected/6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c-kube-api-access-xrdkx\") pod \"console-7945976fcb-jlwss\" (UID: \"6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c\") " pod="openshift-console/console-7945976fcb-jlwss" Jul 12 18:18:40.010933 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:40.010894 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7945976fcb-jlwss" Jul 12 18:18:40.139340 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:40.139306 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7945976fcb-jlwss"] Jul 12 18:18:40.142765 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:18:40.142736 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dcb77ba_dfe8_4dc3_b9b6_2ccd045c570c.slice/crio-2f62bb6b9b9d893957bb1a7adce364ed98466a802eb3cf45b9ca3c3ff07f4c33 WatchSource:0}: Error finding container 2f62bb6b9b9d893957bb1a7adce364ed98466a802eb3cf45b9ca3c3ff07f4c33: Status 404 returned error can't find the container with id 2f62bb6b9b9d893957bb1a7adce364ed98466a802eb3cf45b9ca3c3ff07f4c33 Jul 12 18:18:40.599172 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:40.599133 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7945976fcb-jlwss" event={"ID":"6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c","Type":"ContainerStarted","Data":"2f62bb6b9b9d893957bb1a7adce364ed98466a802eb3cf45b9ca3c3ff07f4c33"} Jul 12 18:18:43.611692 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:43.611657 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7945976fcb-jlwss" event={"ID":"6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c","Type":"ContainerStarted","Data":"461d79066514fc1c3d53b8899404daa2da435e6fe155e837e4b6a4188319b6b4"} Jul 12 18:18:43.628492 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:43.628402 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7945976fcb-jlwss" podStartSLOduration=1.3930594539999999 podStartE2EDuration="4.628386235s" totalImagesPullingTime="3.235326781s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:18:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:18:43.627619273 +0000 UTC m=+96.754524335" watchObservedRunningTime="2026-07-12 18:18:43.628386235 +0000 UTC m=+96.755291076" Jul 12 18:18:44.522453 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:44.522404 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7bc8fbd9f8-mc6xx" podUID="d5113b21-a815-40a5-8b61-0c3b0e8c58ca" containerName="registry" containerID="cri-o://e74014e3c73427ea56855d5c95ab81c8336b333edd462927ac300a9f9a8d16cb" gracePeriod=30 Jul 12 18:18:44.761125 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:44.761103 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7bc8fbd9f8-mc6xx" Jul 12 18:18:44.931953 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:44.931868 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-trgdn\" (UniqueName: \"kubernetes.io/projected/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-kube-api-access-trgdn\") pod \"d5113b21-a815-40a5-8b61-0c3b0e8c58ca\" (UID: \"d5113b21-a815-40a5-8b61-0c3b0e8c58ca\") " Jul 12 18:18:44.931953 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:44.931926 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-ca-trust-extracted\") pod \"d5113b21-a815-40a5-8b61-0c3b0e8c58ca\" (UID: \"d5113b21-a815-40a5-8b61-0c3b0e8c58ca\") " Jul 12 18:18:44.931953 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:44.931946 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-registry-tls\") pod \"d5113b21-a815-40a5-8b61-0c3b0e8c58ca\" (UID: \"d5113b21-a815-40a5-8b61-0c3b0e8c58ca\") " Jul 12 18:18:44.932211 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:44.931972 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-registry-certificates\") pod \"d5113b21-a815-40a5-8b61-0c3b0e8c58ca\" (UID: \"d5113b21-a815-40a5-8b61-0c3b0e8c58ca\") " Jul 12 18:18:44.932211 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:44.931997 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-trusted-ca\") pod \"d5113b21-a815-40a5-8b61-0c3b0e8c58ca\" (UID: \"d5113b21-a815-40a5-8b61-0c3b0e8c58ca\") " Jul 12 18:18:44.932211 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:44.932030 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-bound-sa-token\") pod \"d5113b21-a815-40a5-8b61-0c3b0e8c58ca\" (UID: \"d5113b21-a815-40a5-8b61-0c3b0e8c58ca\") " Jul 12 18:18:44.932211 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:44.932067 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-installation-pull-secrets\") pod \"d5113b21-a815-40a5-8b61-0c3b0e8c58ca\" (UID: \"d5113b21-a815-40a5-8b61-0c3b0e8c58ca\") " Jul 12 18:18:44.932211 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:44.932104 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-image-registry-private-configuration\") pod \"d5113b21-a815-40a5-8b61-0c3b0e8c58ca\" (UID: \"d5113b21-a815-40a5-8b61-0c3b0e8c58ca\") " Jul 12 18:18:44.932788 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:44.932701 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "d5113b21-a815-40a5-8b61-0c3b0e8c58ca" (UID: "d5113b21-a815-40a5-8b61-0c3b0e8c58ca"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:18:44.933166 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:44.933127 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "d5113b21-a815-40a5-8b61-0c3b0e8c58ca" (UID: "d5113b21-a815-40a5-8b61-0c3b0e8c58ca"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:18:44.934536 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:44.934491 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "d5113b21-a815-40a5-8b61-0c3b0e8c58ca" (UID: "d5113b21-a815-40a5-8b61-0c3b0e8c58ca"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 18:18:44.934894 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:44.934859 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-kube-api-access-trgdn" (OuterVolumeSpecName: "kube-api-access-trgdn") pod "d5113b21-a815-40a5-8b61-0c3b0e8c58ca" (UID: "d5113b21-a815-40a5-8b61-0c3b0e8c58ca"). InnerVolumeSpecName "kube-api-access-trgdn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:18:44.935010 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:44.934931 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "d5113b21-a815-40a5-8b61-0c3b0e8c58ca" (UID: "d5113b21-a815-40a5-8b61-0c3b0e8c58ca"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 18:18:44.935010 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:44.934968 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "d5113b21-a815-40a5-8b61-0c3b0e8c58ca" (UID: "d5113b21-a815-40a5-8b61-0c3b0e8c58ca"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:18:44.935149 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:44.935027 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "d5113b21-a815-40a5-8b61-0c3b0e8c58ca" (UID: "d5113b21-a815-40a5-8b61-0c3b0e8c58ca"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:18:44.943107 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:44.943075 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "d5113b21-a815-40a5-8b61-0c3b0e8c58ca" (UID: "d5113b21-a815-40a5-8b61-0c3b0e8c58ca"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:18:45.033157 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:45.033122 2577 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-bound-sa-token\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:18:45.033157 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:45.033152 2577 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-installation-pull-secrets\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:18:45.033157 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:45.033164 2577 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-image-registry-private-configuration\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:18:45.033380 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:45.033174 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-trgdn\" (UniqueName: \"kubernetes.io/projected/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-kube-api-access-trgdn\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:18:45.033380 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:45.033183 2577 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-ca-trust-extracted\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:18:45.033380 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:45.033191 2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-registry-tls\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:18:45.033380 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:45.033199 2577 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-registry-certificates\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:18:45.033380 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:45.033207 2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d5113b21-a815-40a5-8b61-0c3b0e8c58ca-trusted-ca\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:18:45.621169 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:45.621133 2577 generic.go:356] "Generic (PLEG): container finished" podID="d5113b21-a815-40a5-8b61-0c3b0e8c58ca" containerID="e74014e3c73427ea56855d5c95ab81c8336b333edd462927ac300a9f9a8d16cb" exitCode=0 Jul 12 18:18:45.621360 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:45.621193 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7bc8fbd9f8-mc6xx" Jul 12 18:18:45.621360 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:45.621210 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7bc8fbd9f8-mc6xx" event={"ID":"d5113b21-a815-40a5-8b61-0c3b0e8c58ca","Type":"ContainerDied","Data":"e74014e3c73427ea56855d5c95ab81c8336b333edd462927ac300a9f9a8d16cb"} Jul 12 18:18:45.621360 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:45.621258 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7bc8fbd9f8-mc6xx" event={"ID":"d5113b21-a815-40a5-8b61-0c3b0e8c58ca","Type":"ContainerDied","Data":"bab75a3ca2bf00ca3bd9bb67a9b9ec844bc8723ff970cde9ecb8fe56ec245985"} Jul 12 18:18:45.621360 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:45.621284 2577 scope.go:117] "RemoveContainer" containerID="e74014e3c73427ea56855d5c95ab81c8336b333edd462927ac300a9f9a8d16cb" Jul 12 18:18:45.629269 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:45.629250 2577 scope.go:117] "RemoveContainer" containerID="e74014e3c73427ea56855d5c95ab81c8336b333edd462927ac300a9f9a8d16cb" Jul 12 18:18:45.629534 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:18:45.629509 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e74014e3c73427ea56855d5c95ab81c8336b333edd462927ac300a9f9a8d16cb\": container with ID starting with e74014e3c73427ea56855d5c95ab81c8336b333edd462927ac300a9f9a8d16cb not found: ID does not exist" containerID="e74014e3c73427ea56855d5c95ab81c8336b333edd462927ac300a9f9a8d16cb" Jul 12 18:18:45.629585 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:45.629544 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e74014e3c73427ea56855d5c95ab81c8336b333edd462927ac300a9f9a8d16cb"} err="failed to get container status \"e74014e3c73427ea56855d5c95ab81c8336b333edd462927ac300a9f9a8d16cb\": rpc error: code = NotFound desc = could not find container \"e74014e3c73427ea56855d5c95ab81c8336b333edd462927ac300a9f9a8d16cb\": container with ID starting with e74014e3c73427ea56855d5c95ab81c8336b333edd462927ac300a9f9a8d16cb not found: ID does not exist" Jul 12 18:18:45.637464 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:45.637442 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7bc8fbd9f8-mc6xx"] Jul 12 18:18:45.641314 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:45.641295 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7bc8fbd9f8-mc6xx"] Jul 12 18:18:47.074692 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:47.074660 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5113b21-a815-40a5-8b61-0c3b0e8c58ca" path="/var/lib/kubelet/pods/d5113b21-a815-40a5-8b61-0c3b0e8c58ca/volumes" Jul 12 18:18:48.484383 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:48.484310 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8554b68bb5-rsshs"] Jul 12 18:18:48.484752 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:48.484550 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5113b21-a815-40a5-8b61-0c3b0e8c58ca" containerName="registry" Jul 12 18:18:48.484752 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:48.484563 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5113b21-a815-40a5-8b61-0c3b0e8c58ca" containerName="registry" Jul 12 18:18:48.484752 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:48.484607 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5113b21-a815-40a5-8b61-0c3b0e8c58ca" containerName="registry" Jul 12 18:18:48.527448 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:48.527414 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8554b68bb5-rsshs"] Jul 12 18:18:48.527599 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:48.527510 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8554b68bb5-rsshs" Jul 12 18:18:48.536044 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:48.536022 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 12 18:18:48.558569 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:48.558536 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cdea5bab-e53e-47ff-b1bf-b5b8319ddeea-console-serving-cert\") pod \"console-8554b68bb5-rsshs\" (UID: \"cdea5bab-e53e-47ff-b1bf-b5b8319ddeea\") " pod="openshift-console/console-8554b68bb5-rsshs" Jul 12 18:18:48.558721 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:48.558583 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cdea5bab-e53e-47ff-b1bf-b5b8319ddeea-oauth-serving-cert\") pod \"console-8554b68bb5-rsshs\" (UID: \"cdea5bab-e53e-47ff-b1bf-b5b8319ddeea\") " pod="openshift-console/console-8554b68bb5-rsshs" Jul 12 18:18:48.558721 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:48.558650 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdea5bab-e53e-47ff-b1bf-b5b8319ddeea-trusted-ca-bundle\") pod \"console-8554b68bb5-rsshs\" (UID: \"cdea5bab-e53e-47ff-b1bf-b5b8319ddeea\") " pod="openshift-console/console-8554b68bb5-rsshs" Jul 12 18:18:48.558721 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:48.558705 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cdea5bab-e53e-47ff-b1bf-b5b8319ddeea-console-oauth-config\") pod \"console-8554b68bb5-rsshs\" (UID: \"cdea5bab-e53e-47ff-b1bf-b5b8319ddeea\") " pod="openshift-console/console-8554b68bb5-rsshs" Jul 12 18:18:48.558882 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:48.558790 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cdea5bab-e53e-47ff-b1bf-b5b8319ddeea-console-config\") pod \"console-8554b68bb5-rsshs\" (UID: \"cdea5bab-e53e-47ff-b1bf-b5b8319ddeea\") " pod="openshift-console/console-8554b68bb5-rsshs" Jul 12 18:18:48.558882 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:48.558835 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcm9d\" (UniqueName: \"kubernetes.io/projected/cdea5bab-e53e-47ff-b1bf-b5b8319ddeea-kube-api-access-gcm9d\") pod \"console-8554b68bb5-rsshs\" (UID: \"cdea5bab-e53e-47ff-b1bf-b5b8319ddeea\") " pod="openshift-console/console-8554b68bb5-rsshs" Jul 12 18:18:48.558949 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:48.558879 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cdea5bab-e53e-47ff-b1bf-b5b8319ddeea-service-ca\") pod \"console-8554b68bb5-rsshs\" (UID: \"cdea5bab-e53e-47ff-b1bf-b5b8319ddeea\") " pod="openshift-console/console-8554b68bb5-rsshs" Jul 12 18:18:48.659425 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:48.659392 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdea5bab-e53e-47ff-b1bf-b5b8319ddeea-trusted-ca-bundle\") pod \"console-8554b68bb5-rsshs\" (UID: \"cdea5bab-e53e-47ff-b1bf-b5b8319ddeea\") " pod="openshift-console/console-8554b68bb5-rsshs" Jul 12 18:18:48.659425 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:48.659425 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cdea5bab-e53e-47ff-b1bf-b5b8319ddeea-console-oauth-config\") pod \"console-8554b68bb5-rsshs\" (UID: \"cdea5bab-e53e-47ff-b1bf-b5b8319ddeea\") " pod="openshift-console/console-8554b68bb5-rsshs" Jul 12 18:18:48.659685 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:48.659443 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cdea5bab-e53e-47ff-b1bf-b5b8319ddeea-console-config\") pod \"console-8554b68bb5-rsshs\" (UID: \"cdea5bab-e53e-47ff-b1bf-b5b8319ddeea\") " pod="openshift-console/console-8554b68bb5-rsshs" Jul 12 18:18:48.659685 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:48.659467 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcm9d\" (UniqueName: \"kubernetes.io/projected/cdea5bab-e53e-47ff-b1bf-b5b8319ddeea-kube-api-access-gcm9d\") pod \"console-8554b68bb5-rsshs\" (UID: \"cdea5bab-e53e-47ff-b1bf-b5b8319ddeea\") " pod="openshift-console/console-8554b68bb5-rsshs" Jul 12 18:18:48.659685 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:48.659484 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cdea5bab-e53e-47ff-b1bf-b5b8319ddeea-service-ca\") pod \"console-8554b68bb5-rsshs\" (UID: \"cdea5bab-e53e-47ff-b1bf-b5b8319ddeea\") " pod="openshift-console/console-8554b68bb5-rsshs" Jul 12 18:18:48.659685 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:48.659512 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cdea5bab-e53e-47ff-b1bf-b5b8319ddeea-console-serving-cert\") pod \"console-8554b68bb5-rsshs\" (UID: \"cdea5bab-e53e-47ff-b1bf-b5b8319ddeea\") " pod="openshift-console/console-8554b68bb5-rsshs" Jul 12 18:18:48.659685 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:48.659543 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cdea5bab-e53e-47ff-b1bf-b5b8319ddeea-oauth-serving-cert\") pod \"console-8554b68bb5-rsshs\" (UID: \"cdea5bab-e53e-47ff-b1bf-b5b8319ddeea\") " pod="openshift-console/console-8554b68bb5-rsshs" Jul 12 18:18:48.660308 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:48.660257 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cdea5bab-e53e-47ff-b1bf-b5b8319ddeea-service-ca\") pod \"console-8554b68bb5-rsshs\" (UID: \"cdea5bab-e53e-47ff-b1bf-b5b8319ddeea\") " pod="openshift-console/console-8554b68bb5-rsshs" Jul 12 18:18:48.660308 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:48.660301 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cdea5bab-e53e-47ff-b1bf-b5b8319ddeea-console-config\") pod \"console-8554b68bb5-rsshs\" (UID: \"cdea5bab-e53e-47ff-b1bf-b5b8319ddeea\") " pod="openshift-console/console-8554b68bb5-rsshs" Jul 12 18:18:48.660486 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:48.660315 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cdea5bab-e53e-47ff-b1bf-b5b8319ddeea-oauth-serving-cert\") pod \"console-8554b68bb5-rsshs\" (UID: \"cdea5bab-e53e-47ff-b1bf-b5b8319ddeea\") " pod="openshift-console/console-8554b68bb5-rsshs" Jul 12 18:18:48.660486 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:48.660358 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdea5bab-e53e-47ff-b1bf-b5b8319ddeea-trusted-ca-bundle\") pod \"console-8554b68bb5-rsshs\" (UID: \"cdea5bab-e53e-47ff-b1bf-b5b8319ddeea\") " pod="openshift-console/console-8554b68bb5-rsshs" Jul 12 18:18:48.661840 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:48.661820 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cdea5bab-e53e-47ff-b1bf-b5b8319ddeea-console-oauth-config\") pod \"console-8554b68bb5-rsshs\" (UID: \"cdea5bab-e53e-47ff-b1bf-b5b8319ddeea\") " pod="openshift-console/console-8554b68bb5-rsshs" Jul 12 18:18:48.662122 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:48.662103 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cdea5bab-e53e-47ff-b1bf-b5b8319ddeea-console-serving-cert\") pod \"console-8554b68bb5-rsshs\" (UID: \"cdea5bab-e53e-47ff-b1bf-b5b8319ddeea\") " pod="openshift-console/console-8554b68bb5-rsshs" Jul 12 18:18:48.667113 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:48.667092 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcm9d\" (UniqueName: \"kubernetes.io/projected/cdea5bab-e53e-47ff-b1bf-b5b8319ddeea-kube-api-access-gcm9d\") pod \"console-8554b68bb5-rsshs\" (UID: \"cdea5bab-e53e-47ff-b1bf-b5b8319ddeea\") " pod="openshift-console/console-8554b68bb5-rsshs" Jul 12 18:18:48.837270 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:48.837183 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8554b68bb5-rsshs" Jul 12 18:18:48.995657 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:48.995605 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8554b68bb5-rsshs"] Jul 12 18:18:48.999023 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:18:48.998995 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdea5bab_e53e_47ff_b1bf_b5b8319ddeea.slice/crio-d86b8464941aa63e98e07276f4e40811bb4d35772de568fda51d19561c388266 WatchSource:0}: Error finding container d86b8464941aa63e98e07276f4e40811bb4d35772de568fda51d19561c388266: Status 404 returned error can't find the container with id d86b8464941aa63e98e07276f4e40811bb4d35772de568fda51d19561c388266 Jul 12 18:18:49.637714 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:49.637677 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8554b68bb5-rsshs" event={"ID":"cdea5bab-e53e-47ff-b1bf-b5b8319ddeea","Type":"ContainerStarted","Data":"dc5ac5b0bcd65ad1e61e129c9eac71f13d10fa6a3d6a68ae37ab272f91b89496"} Jul 12 18:18:49.638061 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:49.637733 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8554b68bb5-rsshs" event={"ID":"cdea5bab-e53e-47ff-b1bf-b5b8319ddeea","Type":"ContainerStarted","Data":"d86b8464941aa63e98e07276f4e40811bb4d35772de568fda51d19561c388266"} Jul 12 18:18:49.655276 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:49.655226 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8554b68bb5-rsshs" podStartSLOduration=1.655209682 podStartE2EDuration="1.655209682s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:18:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:18:49.654364484 +0000 UTC m=+102.781269289" watchObservedRunningTime="2026-07-12 18:18:49.655209682 +0000 UTC m=+102.782114487" Jul 12 18:18:50.011030 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:50.010997 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7945976fcb-jlwss" Jul 12 18:18:50.011213 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:50.011088 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7945976fcb-jlwss" Jul 12 18:18:50.015944 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:50.015923 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7945976fcb-jlwss" Jul 12 18:18:50.506603 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:50.506565 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-lfnkm" Jul 12 18:18:50.645513 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:50.645488 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7945976fcb-jlwss" Jul 12 18:18:54.786939 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:54.786883 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-59bb7dbd4-wj8zl" Jul 12 18:18:54.787413 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:54.786962 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-59bb7dbd4-wj8zl" Jul 12 18:18:58.837761 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:58.837713 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8554b68bb5-rsshs" Jul 12 18:18:58.838221 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:58.837783 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8554b68bb5-rsshs" Jul 12 18:18:58.842414 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:58.842393 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8554b68bb5-rsshs" Jul 12 18:18:59.676160 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:59.676127 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8554b68bb5-rsshs" Jul 12 18:18:59.722615 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:18:59.722582 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7945976fcb-jlwss"] Jul 12 18:19:14.791400 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:14.791365 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-59bb7dbd4-wj8zl" Jul 12 18:19:14.795124 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:14.795094 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-59bb7dbd4-wj8zl" Jul 12 18:19:24.743169 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:24.743098 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7945976fcb-jlwss" podUID="6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c" containerName="console" containerID="cri-o://461d79066514fc1c3d53b8899404daa2da435e6fe155e837e4b6a4188319b6b4" gracePeriod=15 Jul 12 18:19:24.992429 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:24.992405 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7945976fcb-jlwss" Jul 12 18:19:25.135092 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:25.135003 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c-console-serving-cert\") pod \"6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c\" (UID: \"6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c\") " Jul 12 18:19:25.135092 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:25.135036 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xrdkx\" (UniqueName: \"kubernetes.io/projected/6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c-kube-api-access-xrdkx\") pod \"6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c\" (UID: \"6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c\") " Jul 12 18:19:25.135092 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:25.135065 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c-console-config\") pod \"6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c\" (UID: \"6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c\") " Jul 12 18:19:25.135092 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:25.135085 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c-oauth-serving-cert\") pod \"6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c\" (UID: \"6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c\") " Jul 12 18:19:25.135419 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:25.135145 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c-service-ca\") pod \"6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c\" (UID: \"6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c\") " Jul 12 18:19:25.135419 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:25.135170 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c-console-oauth-config\") pod \"6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c\" (UID: \"6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c\") " Jul 12 18:19:25.135561 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:25.135531 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c" (UID: "6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:19:25.135677 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:25.135540 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c-console-config" (OuterVolumeSpecName: "console-config") pod "6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c" (UID: "6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:19:25.135677 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:25.135542 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c-service-ca" (OuterVolumeSpecName: "service-ca") pod "6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c" (UID: "6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:19:25.137398 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:25.137373 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c-kube-api-access-xrdkx" (OuterVolumeSpecName: "kube-api-access-xrdkx") pod "6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c" (UID: "6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c"). InnerVolumeSpecName "kube-api-access-xrdkx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:19:25.137488 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:25.137422 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c" (UID: "6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 18:19:25.137488 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:25.137446 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c" (UID: "6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 18:19:25.235789 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:25.235756 2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c-service-ca\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:19:25.235789 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:25.235784 2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c-console-oauth-config\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:19:25.235789 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:25.235799 2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c-console-serving-cert\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:19:25.236011 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:25.235812 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xrdkx\" (UniqueName: \"kubernetes.io/projected/6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c-kube-api-access-xrdkx\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:19:25.236011 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:25.235823 2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c-console-config\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:19:25.236011 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:25.235835 2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c-oauth-serving-cert\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:19:25.760446 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:25.760411 2577 generic.go:356] "Generic (PLEG): container finished" podID="6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c" containerID="461d79066514fc1c3d53b8899404daa2da435e6fe155e837e4b6a4188319b6b4" exitCode=0 Jul 12 18:19:25.760929 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:25.760462 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7945976fcb-jlwss" Jul 12 18:19:25.760929 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:25.760499 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7945976fcb-jlwss" event={"ID":"6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c","Type":"ContainerDied","Data":"461d79066514fc1c3d53b8899404daa2da435e6fe155e837e4b6a4188319b6b4"} Jul 12 18:19:25.760929 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:25.760548 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7945976fcb-jlwss" event={"ID":"6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c","Type":"ContainerDied","Data":"2f62bb6b9b9d893957bb1a7adce364ed98466a802eb3cf45b9ca3c3ff07f4c33"} Jul 12 18:19:25.760929 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:25.760569 2577 scope.go:117] "RemoveContainer" containerID="461d79066514fc1c3d53b8899404daa2da435e6fe155e837e4b6a4188319b6b4" Jul 12 18:19:25.768495 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:25.768472 2577 scope.go:117] "RemoveContainer" containerID="461d79066514fc1c3d53b8899404daa2da435e6fe155e837e4b6a4188319b6b4" Jul 12 18:19:25.768767 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:19:25.768749 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"461d79066514fc1c3d53b8899404daa2da435e6fe155e837e4b6a4188319b6b4\": container with ID starting with 461d79066514fc1c3d53b8899404daa2da435e6fe155e837e4b6a4188319b6b4 not found: ID does not exist" containerID="461d79066514fc1c3d53b8899404daa2da435e6fe155e837e4b6a4188319b6b4" Jul 12 18:19:25.768825 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:25.768775 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"461d79066514fc1c3d53b8899404daa2da435e6fe155e837e4b6a4188319b6b4"} err="failed to get container status \"461d79066514fc1c3d53b8899404daa2da435e6fe155e837e4b6a4188319b6b4\": rpc error: code = NotFound desc = could not find container \"461d79066514fc1c3d53b8899404daa2da435e6fe155e837e4b6a4188319b6b4\": container with ID starting with 461d79066514fc1c3d53b8899404daa2da435e6fe155e837e4b6a4188319b6b4 not found: ID does not exist" Jul 12 18:19:25.779444 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:25.779424 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7945976fcb-jlwss"] Jul 12 18:19:25.783468 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:25.783448 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7945976fcb-jlwss"] Jul 12 18:19:27.074256 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:27.074224 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c" path="/var/lib/kubelet/pods/6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c/volumes" Jul 12 18:19:52.596844 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:52.596804 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 12 18:19:52.597362 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:52.597350 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c" containerName="console" Jul 12 18:19:52.597420 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:52.597368 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c" containerName="console" Jul 12 18:19:52.597521 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:52.597505 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6dcb77ba-dfe8-4dc3-b9b6-2ccd045c570c" containerName="console" Jul 12 18:19:52.603881 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:52.603847 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 12 18:19:52.606197 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:52.606177 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 12 18:19:52.606376 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:52.606358 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 12 18:19:52.606598 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:52.606577 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 12 18:19:52.606692 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:52.606665 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-rn9kx\"" Jul 12 18:19:52.606692 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:52.606672 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 12 18:19:52.606815 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:52.606703 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 12 18:19:52.606815 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:52.606582 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 12 18:19:52.607069 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:52.607051 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 12 18:19:52.607163 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:52.607089 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 12 18:19:52.611365 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:52.611342 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 12 18:19:52.611783 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:52.611763 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 12 18:19:52.746119 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:52.746080 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdtdt\" (UniqueName: \"kubernetes.io/projected/21e7f811-7d95-4335-a3c7-f10528ce7a0a-kube-api-access-tdtdt\") pod \"alertmanager-main-0\" (UID: \"21e7f811-7d95-4335-a3c7-f10528ce7a0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 18:19:52.746311 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:52.746130 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/21e7f811-7d95-4335-a3c7-f10528ce7a0a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"21e7f811-7d95-4335-a3c7-f10528ce7a0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 18:19:52.746311 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:52.746162 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/21e7f811-7d95-4335-a3c7-f10528ce7a0a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"21e7f811-7d95-4335-a3c7-f10528ce7a0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 18:19:52.746311 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:52.746183 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/21e7f811-7d95-4335-a3c7-f10528ce7a0a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"21e7f811-7d95-4335-a3c7-f10528ce7a0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 18:19:52.746311 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:52.746205 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/21e7f811-7d95-4335-a3c7-f10528ce7a0a-config-out\") pod \"alertmanager-main-0\" (UID: \"21e7f811-7d95-4335-a3c7-f10528ce7a0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 18:19:52.746311 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:52.746229 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/21e7f811-7d95-4335-a3c7-f10528ce7a0a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"21e7f811-7d95-4335-a3c7-f10528ce7a0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 18:19:52.746311 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:52.746271 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/21e7f811-7d95-4335-a3c7-f10528ce7a0a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"21e7f811-7d95-4335-a3c7-f10528ce7a0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 18:19:52.746311 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:52.746294 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/21e7f811-7d95-4335-a3c7-f10528ce7a0a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"21e7f811-7d95-4335-a3c7-f10528ce7a0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 18:19:52.746605 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:52.746315 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/21e7f811-7d95-4335-a3c7-f10528ce7a0a-web-config\") pod \"alertmanager-main-0\" (UID: \"21e7f811-7d95-4335-a3c7-f10528ce7a0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 18:19:52.746605 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:52.746338 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/21e7f811-7d95-4335-a3c7-f10528ce7a0a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"21e7f811-7d95-4335-a3c7-f10528ce7a0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 18:19:52.746605 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:52.746364 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/21e7f811-7d95-4335-a3c7-f10528ce7a0a-config-volume\") pod \"alertmanager-main-0\" (UID: \"21e7f811-7d95-4335-a3c7-f10528ce7a0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 18:19:52.746605 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:52.746390 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21e7f811-7d95-4335-a3c7-f10528ce7a0a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"21e7f811-7d95-4335-a3c7-f10528ce7a0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 18:19:52.746605 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:52.746430 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/21e7f811-7d95-4335-a3c7-f10528ce7a0a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"21e7f811-7d95-4335-a3c7-f10528ce7a0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 18:19:52.847485 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:52.847386 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/21e7f811-7d95-4335-a3c7-f10528ce7a0a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"21e7f811-7d95-4335-a3c7-f10528ce7a0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 18:19:52.847485 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:52.847437 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/21e7f811-7d95-4335-a3c7-f10528ce7a0a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"21e7f811-7d95-4335-a3c7-f10528ce7a0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 18:19:52.847485 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:52.847454 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/21e7f811-7d95-4335-a3c7-f10528ce7a0a-web-config\") pod \"alertmanager-main-0\" (UID: \"21e7f811-7d95-4335-a3c7-f10528ce7a0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 18:19:52.847485 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:52.847472 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/21e7f811-7d95-4335-a3c7-f10528ce7a0a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"21e7f811-7d95-4335-a3c7-f10528ce7a0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 18:19:52.847485 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:52.847498 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/21e7f811-7d95-4335-a3c7-f10528ce7a0a-config-volume\") pod \"alertmanager-main-0\" (UID: \"21e7f811-7d95-4335-a3c7-f10528ce7a0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 18:19:52.847944 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:52.847610 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21e7f811-7d95-4335-a3c7-f10528ce7a0a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"21e7f811-7d95-4335-a3c7-f10528ce7a0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 18:19:52.847944 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:52.847703 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/21e7f811-7d95-4335-a3c7-f10528ce7a0a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"21e7f811-7d95-4335-a3c7-f10528ce7a0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 18:19:52.847944 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:52.847771 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdtdt\" (UniqueName: \"kubernetes.io/projected/21e7f811-7d95-4335-a3c7-f10528ce7a0a-kube-api-access-tdtdt\") pod \"alertmanager-main-0\" (UID: \"21e7f811-7d95-4335-a3c7-f10528ce7a0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 18:19:52.847944 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:52.847815 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/21e7f811-7d95-4335-a3c7-f10528ce7a0a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"21e7f811-7d95-4335-a3c7-f10528ce7a0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 18:19:52.847944 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:52.847851 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/21e7f811-7d95-4335-a3c7-f10528ce7a0a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"21e7f811-7d95-4335-a3c7-f10528ce7a0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 18:19:52.847944 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:52.847876 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/21e7f811-7d95-4335-a3c7-f10528ce7a0a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"21e7f811-7d95-4335-a3c7-f10528ce7a0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 18:19:52.847944 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:52.847902 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/21e7f811-7d95-4335-a3c7-f10528ce7a0a-config-out\") pod \"alertmanager-main-0\" (UID: \"21e7f811-7d95-4335-a3c7-f10528ce7a0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 18:19:52.847944 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:52.847932 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/21e7f811-7d95-4335-a3c7-f10528ce7a0a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"21e7f811-7d95-4335-a3c7-f10528ce7a0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 18:19:52.848344 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:52.848266 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/21e7f811-7d95-4335-a3c7-f10528ce7a0a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"21e7f811-7d95-4335-a3c7-f10528ce7a0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 18:19:52.848505 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:52.848479 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21e7f811-7d95-4335-a3c7-f10528ce7a0a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"21e7f811-7d95-4335-a3c7-f10528ce7a0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 18:19:52.849388 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:52.849358 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/21e7f811-7d95-4335-a3c7-f10528ce7a0a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"21e7f811-7d95-4335-a3c7-f10528ce7a0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 18:19:52.850673 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:52.850603 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/21e7f811-7d95-4335-a3c7-f10528ce7a0a-config-volume\") pod \"alertmanager-main-0\" (UID: \"21e7f811-7d95-4335-a3c7-f10528ce7a0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 18:19:52.850673 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:52.850661 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/21e7f811-7d95-4335-a3c7-f10528ce7a0a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"21e7f811-7d95-4335-a3c7-f10528ce7a0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 18:19:52.850853 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:52.850827 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/21e7f811-7d95-4335-a3c7-f10528ce7a0a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"21e7f811-7d95-4335-a3c7-f10528ce7a0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 18:19:52.851087 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:52.851046 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/21e7f811-7d95-4335-a3c7-f10528ce7a0a-web-config\") pod \"alertmanager-main-0\" (UID: \"21e7f811-7d95-4335-a3c7-f10528ce7a0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 18:19:52.851415 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:52.851384 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/21e7f811-7d95-4335-a3c7-f10528ce7a0a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"21e7f811-7d95-4335-a3c7-f10528ce7a0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 18:19:52.851555 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:52.851536 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/21e7f811-7d95-4335-a3c7-f10528ce7a0a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"21e7f811-7d95-4335-a3c7-f10528ce7a0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 18:19:52.851735 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:52.851714 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/21e7f811-7d95-4335-a3c7-f10528ce7a0a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"21e7f811-7d95-4335-a3c7-f10528ce7a0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 18:19:52.851877 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:52.851858 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/21e7f811-7d95-4335-a3c7-f10528ce7a0a-config-out\") pod \"alertmanager-main-0\" (UID: \"21e7f811-7d95-4335-a3c7-f10528ce7a0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 18:19:52.852405 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:52.852382 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/21e7f811-7d95-4335-a3c7-f10528ce7a0a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"21e7f811-7d95-4335-a3c7-f10528ce7a0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 18:19:52.857515 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:52.857490 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdtdt\" (UniqueName: \"kubernetes.io/projected/21e7f811-7d95-4335-a3c7-f10528ce7a0a-kube-api-access-tdtdt\") pod \"alertmanager-main-0\" (UID: \"21e7f811-7d95-4335-a3c7-f10528ce7a0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 18:19:52.916296 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:52.916258 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 12 18:19:53.052424 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:53.052389 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 12 18:19:53.057151 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:19:53.057115 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21e7f811_7d95_4335_a3c7_f10528ce7a0a.slice/crio-d173c9ce831e4308e56b04e8f43c666eb079d13fb4f715f1b68ae5a32a4f12b1 WatchSource:0}: Error finding container d173c9ce831e4308e56b04e8f43c666eb079d13fb4f715f1b68ae5a32a4f12b1: Status 404 returned error can't find the container with id d173c9ce831e4308e56b04e8f43c666eb079d13fb4f715f1b68ae5a32a4f12b1 Jul 12 18:19:53.861991 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:53.861941 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"21e7f811-7d95-4335-a3c7-f10528ce7a0a","Type":"ContainerStarted","Data":"d173c9ce831e4308e56b04e8f43c666eb079d13fb4f715f1b68ae5a32a4f12b1"} Jul 12 18:19:54.448527 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:54.448494 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5d45bc54b5-m5nrq"] Jul 12 18:19:54.451655 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:54.451619 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5d45bc54b5-m5nrq" Jul 12 18:19:54.454272 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:54.454251 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 12 18:19:54.454272 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:54.454269 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-xbj2p\"" Jul 12 18:19:54.454446 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:54.454432 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 12 18:19:54.454506 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:54.454474 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 12 18:19:54.454562 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:54.454520 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 12 18:19:54.454562 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:54.454519 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 12 18:19:54.459080 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:54.459063 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 12 18:19:54.464352 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:54.464331 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5d45bc54b5-m5nrq"] Jul 12 18:19:54.561529 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:54.561491 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/56a707ab-2a08-46c0-8ded-547b26ad7da8-metrics-client-ca\") pod \"telemeter-client-5d45bc54b5-m5nrq\" (UID: \"56a707ab-2a08-46c0-8ded-547b26ad7da8\") " pod="openshift-monitoring/telemeter-client-5d45bc54b5-m5nrq" Jul 12 18:19:54.561722 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:54.561539 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fw4m\" (UniqueName: \"kubernetes.io/projected/56a707ab-2a08-46c0-8ded-547b26ad7da8-kube-api-access-2fw4m\") pod \"telemeter-client-5d45bc54b5-m5nrq\" (UID: \"56a707ab-2a08-46c0-8ded-547b26ad7da8\") " pod="openshift-monitoring/telemeter-client-5d45bc54b5-m5nrq" Jul 12 18:19:54.561722 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:54.561618 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/56a707ab-2a08-46c0-8ded-547b26ad7da8-telemeter-client-tls\") pod \"telemeter-client-5d45bc54b5-m5nrq\" (UID: \"56a707ab-2a08-46c0-8ded-547b26ad7da8\") " pod="openshift-monitoring/telemeter-client-5d45bc54b5-m5nrq" Jul 12 18:19:54.561722 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:54.561696 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56a707ab-2a08-46c0-8ded-547b26ad7da8-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5d45bc54b5-m5nrq\" (UID: \"56a707ab-2a08-46c0-8ded-547b26ad7da8\") " pod="openshift-monitoring/telemeter-client-5d45bc54b5-m5nrq" Jul 12 18:19:54.561825 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:54.561727 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/56a707ab-2a08-46c0-8ded-547b26ad7da8-secret-telemeter-client\") pod \"telemeter-client-5d45bc54b5-m5nrq\" (UID: \"56a707ab-2a08-46c0-8ded-547b26ad7da8\") " pod="openshift-monitoring/telemeter-client-5d45bc54b5-m5nrq" Jul 12 18:19:54.561825 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:54.561749 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56a707ab-2a08-46c0-8ded-547b26ad7da8-serving-certs-ca-bundle\") pod \"telemeter-client-5d45bc54b5-m5nrq\" (UID: \"56a707ab-2a08-46c0-8ded-547b26ad7da8\") " pod="openshift-monitoring/telemeter-client-5d45bc54b5-m5nrq" Jul 12 18:19:54.561825 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:54.561770 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/56a707ab-2a08-46c0-8ded-547b26ad7da8-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5d45bc54b5-m5nrq\" (UID: \"56a707ab-2a08-46c0-8ded-547b26ad7da8\") " pod="openshift-monitoring/telemeter-client-5d45bc54b5-m5nrq" Jul 12 18:19:54.561825 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:54.561790 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/56a707ab-2a08-46c0-8ded-547b26ad7da8-federate-client-tls\") pod \"telemeter-client-5d45bc54b5-m5nrq\" (UID: \"56a707ab-2a08-46c0-8ded-547b26ad7da8\") " pod="openshift-monitoring/telemeter-client-5d45bc54b5-m5nrq" Jul 12 18:19:54.662431 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:54.662349 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/56a707ab-2a08-46c0-8ded-547b26ad7da8-secret-telemeter-client\") pod \"telemeter-client-5d45bc54b5-m5nrq\" (UID: \"56a707ab-2a08-46c0-8ded-547b26ad7da8\") " pod="openshift-monitoring/telemeter-client-5d45bc54b5-m5nrq" Jul 12 18:19:54.662431 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:54.662387 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56a707ab-2a08-46c0-8ded-547b26ad7da8-serving-certs-ca-bundle\") pod \"telemeter-client-5d45bc54b5-m5nrq\" (UID: \"56a707ab-2a08-46c0-8ded-547b26ad7da8\") " pod="openshift-monitoring/telemeter-client-5d45bc54b5-m5nrq" Jul 12 18:19:54.662431 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:54.662405 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/56a707ab-2a08-46c0-8ded-547b26ad7da8-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5d45bc54b5-m5nrq\" (UID: \"56a707ab-2a08-46c0-8ded-547b26ad7da8\") " pod="openshift-monitoring/telemeter-client-5d45bc54b5-m5nrq" Jul 12 18:19:54.662431 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:54.662427 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/56a707ab-2a08-46c0-8ded-547b26ad7da8-federate-client-tls\") pod \"telemeter-client-5d45bc54b5-m5nrq\" (UID: \"56a707ab-2a08-46c0-8ded-547b26ad7da8\") " pod="openshift-monitoring/telemeter-client-5d45bc54b5-m5nrq" Jul 12 18:19:54.662756 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:54.662453 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/56a707ab-2a08-46c0-8ded-547b26ad7da8-metrics-client-ca\") pod \"telemeter-client-5d45bc54b5-m5nrq\" (UID: \"56a707ab-2a08-46c0-8ded-547b26ad7da8\") " pod="openshift-monitoring/telemeter-client-5d45bc54b5-m5nrq" Jul 12 18:19:54.662756 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:54.662492 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fw4m\" (UniqueName: \"kubernetes.io/projected/56a707ab-2a08-46c0-8ded-547b26ad7da8-kube-api-access-2fw4m\") pod \"telemeter-client-5d45bc54b5-m5nrq\" (UID: \"56a707ab-2a08-46c0-8ded-547b26ad7da8\") " pod="openshift-monitoring/telemeter-client-5d45bc54b5-m5nrq" Jul 12 18:19:54.662756 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:54.662525 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/56a707ab-2a08-46c0-8ded-547b26ad7da8-telemeter-client-tls\") pod \"telemeter-client-5d45bc54b5-m5nrq\" (UID: \"56a707ab-2a08-46c0-8ded-547b26ad7da8\") " pod="openshift-monitoring/telemeter-client-5d45bc54b5-m5nrq" Jul 12 18:19:54.662756 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:54.662564 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56a707ab-2a08-46c0-8ded-547b26ad7da8-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5d45bc54b5-m5nrq\" (UID: \"56a707ab-2a08-46c0-8ded-547b26ad7da8\") " pod="openshift-monitoring/telemeter-client-5d45bc54b5-m5nrq" Jul 12 18:19:54.663274 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:54.663249 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56a707ab-2a08-46c0-8ded-547b26ad7da8-serving-certs-ca-bundle\") pod \"telemeter-client-5d45bc54b5-m5nrq\" (UID: \"56a707ab-2a08-46c0-8ded-547b26ad7da8\") " pod="openshift-monitoring/telemeter-client-5d45bc54b5-m5nrq" Jul 12 18:19:54.663390 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:54.663290 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/56a707ab-2a08-46c0-8ded-547b26ad7da8-metrics-client-ca\") pod \"telemeter-client-5d45bc54b5-m5nrq\" (UID: \"56a707ab-2a08-46c0-8ded-547b26ad7da8\") " pod="openshift-monitoring/telemeter-client-5d45bc54b5-m5nrq" Jul 12 18:19:54.663790 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:54.663762 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/56a707ab-2a08-46c0-8ded-547b26ad7da8-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5d45bc54b5-m5nrq\" (UID: \"56a707ab-2a08-46c0-8ded-547b26ad7da8\") " pod="openshift-monitoring/telemeter-client-5d45bc54b5-m5nrq" Jul 12 18:19:54.665187 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:54.665162 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/56a707ab-2a08-46c0-8ded-547b26ad7da8-telemeter-client-tls\") pod \"telemeter-client-5d45bc54b5-m5nrq\" (UID: \"56a707ab-2a08-46c0-8ded-547b26ad7da8\") " pod="openshift-monitoring/telemeter-client-5d45bc54b5-m5nrq" Jul 12 18:19:54.665253 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:54.665162 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/56a707ab-2a08-46c0-8ded-547b26ad7da8-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5d45bc54b5-m5nrq\" (UID: \"56a707ab-2a08-46c0-8ded-547b26ad7da8\") " pod="openshift-monitoring/telemeter-client-5d45bc54b5-m5nrq" Jul 12 18:19:54.665327 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:54.665308 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/56a707ab-2a08-46c0-8ded-547b26ad7da8-federate-client-tls\") pod \"telemeter-client-5d45bc54b5-m5nrq\" (UID: \"56a707ab-2a08-46c0-8ded-547b26ad7da8\") " pod="openshift-monitoring/telemeter-client-5d45bc54b5-m5nrq" Jul 12 18:19:54.665363 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:54.665308 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/56a707ab-2a08-46c0-8ded-547b26ad7da8-secret-telemeter-client\") pod \"telemeter-client-5d45bc54b5-m5nrq\" (UID: \"56a707ab-2a08-46c0-8ded-547b26ad7da8\") " pod="openshift-monitoring/telemeter-client-5d45bc54b5-m5nrq" Jul 12 18:19:54.670487 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:54.670461 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fw4m\" (UniqueName: \"kubernetes.io/projected/56a707ab-2a08-46c0-8ded-547b26ad7da8-kube-api-access-2fw4m\") pod \"telemeter-client-5d45bc54b5-m5nrq\" (UID: \"56a707ab-2a08-46c0-8ded-547b26ad7da8\") " pod="openshift-monitoring/telemeter-client-5d45bc54b5-m5nrq" Jul 12 18:19:54.795816 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:54.795790 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5d45bc54b5-m5nrq" Jul 12 18:19:54.867413 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:54.867380 2577 generic.go:356] "Generic (PLEG): container finished" podID="21e7f811-7d95-4335-a3c7-f10528ce7a0a" containerID="74969b95ba7d92c0739c2eebe60df2ca5b8bcaaa079be736e089e33ec4bc9d75" exitCode=0 Jul 12 18:19:54.867813 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:54.867454 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"21e7f811-7d95-4335-a3c7-f10528ce7a0a","Type":"ContainerDied","Data":"74969b95ba7d92c0739c2eebe60df2ca5b8bcaaa079be736e089e33ec4bc9d75"} Jul 12 18:19:54.928758 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:54.928682 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5d45bc54b5-m5nrq"] Jul 12 18:19:54.931470 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:19:54.931446 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56a707ab_2a08_46c0_8ded_547b26ad7da8.slice/crio-3c673828a8a1f625db86aec898edf5943f4363ccd05ac35af9c6e478cc8ab2f6 WatchSource:0}: Error finding container 3c673828a8a1f625db86aec898edf5943f4363ccd05ac35af9c6e478cc8ab2f6: Status 404 returned error can't find the container with id 3c673828a8a1f625db86aec898edf5943f4363ccd05ac35af9c6e478cc8ab2f6 Jul 12 18:19:55.873100 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:55.873056 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5d45bc54b5-m5nrq" event={"ID":"56a707ab-2a08-46c0-8ded-547b26ad7da8","Type":"ContainerStarted","Data":"3c673828a8a1f625db86aec898edf5943f4363ccd05ac35af9c6e478cc8ab2f6"} Jul 12 18:19:57.885362 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:57.885314 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5d45bc54b5-m5nrq" event={"ID":"56a707ab-2a08-46c0-8ded-547b26ad7da8","Type":"ContainerStarted","Data":"669fab5de5cfaf6e6048ef90eff55f490aa9df457d6e91b715d4c1ac97e0b5db"} Jul 12 18:19:57.885865 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:57.885377 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5d45bc54b5-m5nrq" event={"ID":"56a707ab-2a08-46c0-8ded-547b26ad7da8","Type":"ContainerStarted","Data":"482aa22f6a61610bf6e698b52d327e3568b2a924e315edce76e13a565bb9453d"} Jul 12 18:19:57.885865 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:57.885407 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5d45bc54b5-m5nrq" event={"ID":"56a707ab-2a08-46c0-8ded-547b26ad7da8","Type":"ContainerStarted","Data":"35e2cbca453fd08c36332adc376a1b9b25920f51dbff4f6656920fa5fc2572dc"} Jul 12 18:19:57.888996 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:57.888965 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"21e7f811-7d95-4335-a3c7-f10528ce7a0a","Type":"ContainerStarted","Data":"0c0b290bfa219a68103b2fa75a5bb441efa83e275f374a2cb36a811b26791a9a"} Jul 12 18:19:57.889118 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:57.889043 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"21e7f811-7d95-4335-a3c7-f10528ce7a0a","Type":"ContainerStarted","Data":"5d81cf2b6145f267db4c861de801f4f7135e3099e277144a27a51213c39c754e"} Jul 12 18:19:57.889118 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:57.889086 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"21e7f811-7d95-4335-a3c7-f10528ce7a0a","Type":"ContainerStarted","Data":"16ca0a27b8405c1b33a0df755ba8741a1aab2b408998a773776e372eca4ee7e1"} Jul 12 18:19:57.889192 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:57.889128 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"21e7f811-7d95-4335-a3c7-f10528ce7a0a","Type":"ContainerStarted","Data":"28c5ba95250da3e9c0137abac321208c62c15bd7f546adf58cf8036ab015a831"} Jul 12 18:19:57.889192 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:57.889167 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"21e7f811-7d95-4335-a3c7-f10528ce7a0a","Type":"ContainerStarted","Data":"f73509b93aa872fbb30d2cde0173efdaa331cde2a70ae9ecb7efd1129dc1a7a5"} Jul 12 18:19:57.907960 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:57.907885 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5d45bc54b5-m5nrq" podStartSLOduration=1.812720205 podStartE2EDuration="3.90786295s" totalImagesPullingTime="2.095142745s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:19:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:19:57.906110509 +0000 UTC m=+171.033015317" watchObservedRunningTime="2026-07-12 18:19:57.90786295 +0000 UTC m=+171.034767956" Jul 12 18:19:58.484062 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:58.484032 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-65d988bc4f-jj9js"] Jul 12 18:19:58.492194 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:58.492165 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65d988bc4f-jj9js" Jul 12 18:19:58.496492 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:58.496463 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-65d988bc4f-jj9js"] Jul 12 18:19:58.590894 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:58.590855 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc99a43a-4854-4f4c-8f5b-3df24052e69d-trusted-ca-bundle\") pod \"console-65d988bc4f-jj9js\" (UID: \"cc99a43a-4854-4f4c-8f5b-3df24052e69d\") " pod="openshift-console/console-65d988bc4f-jj9js" Jul 12 18:19:58.590894 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:58.590895 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cc99a43a-4854-4f4c-8f5b-3df24052e69d-console-config\") pod \"console-65d988bc4f-jj9js\" (UID: \"cc99a43a-4854-4f4c-8f5b-3df24052e69d\") " pod="openshift-console/console-65d988bc4f-jj9js" Jul 12 18:19:58.591103 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:58.590917 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cc99a43a-4854-4f4c-8f5b-3df24052e69d-console-oauth-config\") pod \"console-65d988bc4f-jj9js\" (UID: \"cc99a43a-4854-4f4c-8f5b-3df24052e69d\") " pod="openshift-console/console-65d988bc4f-jj9js" Jul 12 18:19:58.591103 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:58.590974 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cc99a43a-4854-4f4c-8f5b-3df24052e69d-service-ca\") pod \"console-65d988bc4f-jj9js\" (UID: \"cc99a43a-4854-4f4c-8f5b-3df24052e69d\") " pod="openshift-console/console-65d988bc4f-jj9js" Jul 12 18:19:58.591103 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:58.591005 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hh4xw\" (UniqueName: \"kubernetes.io/projected/cc99a43a-4854-4f4c-8f5b-3df24052e69d-kube-api-access-hh4xw\") pod \"console-65d988bc4f-jj9js\" (UID: \"cc99a43a-4854-4f4c-8f5b-3df24052e69d\") " pod="openshift-console/console-65d988bc4f-jj9js" Jul 12 18:19:58.591103 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:58.591042 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cc99a43a-4854-4f4c-8f5b-3df24052e69d-oauth-serving-cert\") pod \"console-65d988bc4f-jj9js\" (UID: \"cc99a43a-4854-4f4c-8f5b-3df24052e69d\") " pod="openshift-console/console-65d988bc4f-jj9js" Jul 12 18:19:58.591103 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:58.591068 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cc99a43a-4854-4f4c-8f5b-3df24052e69d-console-serving-cert\") pod \"console-65d988bc4f-jj9js\" (UID: \"cc99a43a-4854-4f4c-8f5b-3df24052e69d\") " pod="openshift-console/console-65d988bc4f-jj9js" Jul 12 18:19:58.691925 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:58.691889 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cc99a43a-4854-4f4c-8f5b-3df24052e69d-oauth-serving-cert\") pod \"console-65d988bc4f-jj9js\" (UID: \"cc99a43a-4854-4f4c-8f5b-3df24052e69d\") " pod="openshift-console/console-65d988bc4f-jj9js" Jul 12 18:19:58.692106 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:58.691935 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cc99a43a-4854-4f4c-8f5b-3df24052e69d-console-serving-cert\") pod \"console-65d988bc4f-jj9js\" (UID: \"cc99a43a-4854-4f4c-8f5b-3df24052e69d\") " pod="openshift-console/console-65d988bc4f-jj9js" Jul 12 18:19:58.692106 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:58.691989 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc99a43a-4854-4f4c-8f5b-3df24052e69d-trusted-ca-bundle\") pod \"console-65d988bc4f-jj9js\" (UID: \"cc99a43a-4854-4f4c-8f5b-3df24052e69d\") " pod="openshift-console/console-65d988bc4f-jj9js" Jul 12 18:19:58.692106 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:58.692017 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cc99a43a-4854-4f4c-8f5b-3df24052e69d-console-config\") pod \"console-65d988bc4f-jj9js\" (UID: \"cc99a43a-4854-4f4c-8f5b-3df24052e69d\") " pod="openshift-console/console-65d988bc4f-jj9js" Jul 12 18:19:58.692106 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:58.692045 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cc99a43a-4854-4f4c-8f5b-3df24052e69d-console-oauth-config\") pod \"console-65d988bc4f-jj9js\" (UID: \"cc99a43a-4854-4f4c-8f5b-3df24052e69d\") " pod="openshift-console/console-65d988bc4f-jj9js" Jul 12 18:19:58.692106 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:58.692076 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cc99a43a-4854-4f4c-8f5b-3df24052e69d-service-ca\") pod \"console-65d988bc4f-jj9js\" (UID: \"cc99a43a-4854-4f4c-8f5b-3df24052e69d\") " pod="openshift-console/console-65d988bc4f-jj9js" Jul 12 18:19:58.692106 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:58.692100 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hh4xw\" (UniqueName: \"kubernetes.io/projected/cc99a43a-4854-4f4c-8f5b-3df24052e69d-kube-api-access-hh4xw\") pod \"console-65d988bc4f-jj9js\" (UID: \"cc99a43a-4854-4f4c-8f5b-3df24052e69d\") " pod="openshift-console/console-65d988bc4f-jj9js" Jul 12 18:19:58.692762 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:58.692732 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cc99a43a-4854-4f4c-8f5b-3df24052e69d-oauth-serving-cert\") pod \"console-65d988bc4f-jj9js\" (UID: \"cc99a43a-4854-4f4c-8f5b-3df24052e69d\") " pod="openshift-console/console-65d988bc4f-jj9js" Jul 12 18:19:58.692881 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:58.692803 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cc99a43a-4854-4f4c-8f5b-3df24052e69d-console-config\") pod \"console-65d988bc4f-jj9js\" (UID: \"cc99a43a-4854-4f4c-8f5b-3df24052e69d\") " pod="openshift-console/console-65d988bc4f-jj9js" Jul 12 18:19:58.692943 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:58.692876 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cc99a43a-4854-4f4c-8f5b-3df24052e69d-service-ca\") pod \"console-65d988bc4f-jj9js\" (UID: \"cc99a43a-4854-4f4c-8f5b-3df24052e69d\") " pod="openshift-console/console-65d988bc4f-jj9js" Jul 12 18:19:58.692943 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:58.692909 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc99a43a-4854-4f4c-8f5b-3df24052e69d-trusted-ca-bundle\") pod \"console-65d988bc4f-jj9js\" (UID: \"cc99a43a-4854-4f4c-8f5b-3df24052e69d\") " pod="openshift-console/console-65d988bc4f-jj9js" Jul 12 18:19:58.694387 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:58.694369 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cc99a43a-4854-4f4c-8f5b-3df24052e69d-console-serving-cert\") pod \"console-65d988bc4f-jj9js\" (UID: \"cc99a43a-4854-4f4c-8f5b-3df24052e69d\") " pod="openshift-console/console-65d988bc4f-jj9js" Jul 12 18:19:58.694461 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:58.694402 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cc99a43a-4854-4f4c-8f5b-3df24052e69d-console-oauth-config\") pod \"console-65d988bc4f-jj9js\" (UID: \"cc99a43a-4854-4f4c-8f5b-3df24052e69d\") " pod="openshift-console/console-65d988bc4f-jj9js" Jul 12 18:19:58.699856 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:58.699828 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hh4xw\" (UniqueName: \"kubernetes.io/projected/cc99a43a-4854-4f4c-8f5b-3df24052e69d-kube-api-access-hh4xw\") pod \"console-65d988bc4f-jj9js\" (UID: \"cc99a43a-4854-4f4c-8f5b-3df24052e69d\") " pod="openshift-console/console-65d988bc4f-jj9js" Jul 12 18:19:58.802459 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:58.802425 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65d988bc4f-jj9js" Jul 12 18:19:58.897517 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:58.897429 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"21e7f811-7d95-4335-a3c7-f10528ce7a0a","Type":"ContainerStarted","Data":"271df126f72c3a1ae29320e8a9687784cc1f2564425da8c113dbea47e588810d"} Jul 12 18:19:58.927310 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:58.927249 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.426602881 podStartE2EDuration="6.927225314s" totalImagesPullingTime="4.4792194s" totalInitContainerRuntime="21.403033ms" isStatefulPod=false podCreationTimestamp="2026-07-12 18:19:52 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:19:58.92314077 +0000 UTC m=+172.050045579" watchObservedRunningTime="2026-07-12 18:19:58.927225314 +0000 UTC m=+172.054130120" Jul 12 18:19:58.948678 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:58.948577 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-65d988bc4f-jj9js"] Jul 12 18:19:58.952082 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:19:58.952055 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc99a43a_4854_4f4c_8f5b_3df24052e69d.slice/crio-3656497f5dc3c916f4d01bd9cee8dc08d5abf883da639b3abd93031f0cba4257 WatchSource:0}: Error finding container 3656497f5dc3c916f4d01bd9cee8dc08d5abf883da639b3abd93031f0cba4257: Status 404 returned error can't find the container with id 3656497f5dc3c916f4d01bd9cee8dc08d5abf883da639b3abd93031f0cba4257 Jul 12 18:19:59.902877 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:59.902840 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65d988bc4f-jj9js" event={"ID":"cc99a43a-4854-4f4c-8f5b-3df24052e69d","Type":"ContainerStarted","Data":"7b1f0f4a12d3b3765c5fbec21b799a44de0a450b49a15fc395ace6e93e3bf10d"} Jul 12 18:19:59.903274 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:59.902894 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65d988bc4f-jj9js" event={"ID":"cc99a43a-4854-4f4c-8f5b-3df24052e69d","Type":"ContainerStarted","Data":"3656497f5dc3c916f4d01bd9cee8dc08d5abf883da639b3abd93031f0cba4257"} Jul 12 18:19:59.918775 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:19:59.918722 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-65d988bc4f-jj9js" podStartSLOduration=1.918706963 podStartE2EDuration="1.918706963s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:19:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:19:59.917671446 +0000 UTC m=+173.044576255" watchObservedRunningTime="2026-07-12 18:19:59.918706963 +0000 UTC m=+173.045611767" Jul 12 18:20:08.803077 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:20:08.803029 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-65d988bc4f-jj9js" Jul 12 18:20:08.803488 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:20:08.803107 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-65d988bc4f-jj9js" Jul 12 18:20:08.807724 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:20:08.807702 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-65d988bc4f-jj9js" Jul 12 18:20:08.938209 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:20:08.938184 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-65d988bc4f-jj9js" Jul 12 18:20:08.981394 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:20:08.981366 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8554b68bb5-rsshs"] Jul 12 18:20:34.003646 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:20:34.003590 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-8554b68bb5-rsshs" podUID="cdea5bab-e53e-47ff-b1bf-b5b8319ddeea" containerName="console" containerID="cri-o://dc5ac5b0bcd65ad1e61e129c9eac71f13d10fa6a3d6a68ae37ab272f91b89496" gracePeriod=15 Jul 12 18:20:34.247577 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:20:34.247551 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8554b68bb5-rsshs" Jul 12 18:20:34.381299 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:20:34.381217 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cdea5bab-e53e-47ff-b1bf-b5b8319ddeea-console-config\") pod \"cdea5bab-e53e-47ff-b1bf-b5b8319ddeea\" (UID: \"cdea5bab-e53e-47ff-b1bf-b5b8319ddeea\") " Jul 12 18:20:34.381299 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:20:34.381251 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cdea5bab-e53e-47ff-b1bf-b5b8319ddeea-console-oauth-config\") pod \"cdea5bab-e53e-47ff-b1bf-b5b8319ddeea\" (UID: \"cdea5bab-e53e-47ff-b1bf-b5b8319ddeea\") " Jul 12 18:20:34.381299 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:20:34.381282 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdea5bab-e53e-47ff-b1bf-b5b8319ddeea-trusted-ca-bundle\") pod \"cdea5bab-e53e-47ff-b1bf-b5b8319ddeea\" (UID: \"cdea5bab-e53e-47ff-b1bf-b5b8319ddeea\") " Jul 12 18:20:34.381557 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:20:34.381311 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cdea5bab-e53e-47ff-b1bf-b5b8319ddeea-console-serving-cert\") pod \"cdea5bab-e53e-47ff-b1bf-b5b8319ddeea\" (UID: \"cdea5bab-e53e-47ff-b1bf-b5b8319ddeea\") " Jul 12 18:20:34.381557 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:20:34.381331 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cdea5bab-e53e-47ff-b1bf-b5b8319ddeea-oauth-serving-cert\") pod \"cdea5bab-e53e-47ff-b1bf-b5b8319ddeea\" (UID: \"cdea5bab-e53e-47ff-b1bf-b5b8319ddeea\") " Jul 12 18:20:34.381557 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:20:34.381348 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cdea5bab-e53e-47ff-b1bf-b5b8319ddeea-service-ca\") pod \"cdea5bab-e53e-47ff-b1bf-b5b8319ddeea\" (UID: \"cdea5bab-e53e-47ff-b1bf-b5b8319ddeea\") " Jul 12 18:20:34.381557 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:20:34.381384 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gcm9d\" (UniqueName: \"kubernetes.io/projected/cdea5bab-e53e-47ff-b1bf-b5b8319ddeea-kube-api-access-gcm9d\") pod \"cdea5bab-e53e-47ff-b1bf-b5b8319ddeea\" (UID: \"cdea5bab-e53e-47ff-b1bf-b5b8319ddeea\") " Jul 12 18:20:34.381806 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:20:34.381779 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cdea5bab-e53e-47ff-b1bf-b5b8319ddeea-console-config" (OuterVolumeSpecName: "console-config") pod "cdea5bab-e53e-47ff-b1bf-b5b8319ddeea" (UID: "cdea5bab-e53e-47ff-b1bf-b5b8319ddeea"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:20:34.381868 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:20:34.381828 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cdea5bab-e53e-47ff-b1bf-b5b8319ddeea-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "cdea5bab-e53e-47ff-b1bf-b5b8319ddeea" (UID: "cdea5bab-e53e-47ff-b1bf-b5b8319ddeea"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:20:34.381942 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:20:34.381917 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cdea5bab-e53e-47ff-b1bf-b5b8319ddeea-service-ca" (OuterVolumeSpecName: "service-ca") pod "cdea5bab-e53e-47ff-b1bf-b5b8319ddeea" (UID: "cdea5bab-e53e-47ff-b1bf-b5b8319ddeea"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:20:34.381986 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:20:34.381923 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cdea5bab-e53e-47ff-b1bf-b5b8319ddeea-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "cdea5bab-e53e-47ff-b1bf-b5b8319ddeea" (UID: "cdea5bab-e53e-47ff-b1bf-b5b8319ddeea"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:20:34.383596 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:20:34.383567 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cdea5bab-e53e-47ff-b1bf-b5b8319ddeea-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "cdea5bab-e53e-47ff-b1bf-b5b8319ddeea" (UID: "cdea5bab-e53e-47ff-b1bf-b5b8319ddeea"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 18:20:34.383596 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:20:34.383581 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cdea5bab-e53e-47ff-b1bf-b5b8319ddeea-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "cdea5bab-e53e-47ff-b1bf-b5b8319ddeea" (UID: "cdea5bab-e53e-47ff-b1bf-b5b8319ddeea"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 18:20:34.383786 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:20:34.383719 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdea5bab-e53e-47ff-b1bf-b5b8319ddeea-kube-api-access-gcm9d" (OuterVolumeSpecName: "kube-api-access-gcm9d") pod "cdea5bab-e53e-47ff-b1bf-b5b8319ddeea" (UID: "cdea5bab-e53e-47ff-b1bf-b5b8319ddeea"). InnerVolumeSpecName "kube-api-access-gcm9d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:20:34.482006 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:20:34.481968 2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cdea5bab-e53e-47ff-b1bf-b5b8319ddeea-console-config\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:20:34.482006 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:20:34.481999 2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cdea5bab-e53e-47ff-b1bf-b5b8319ddeea-console-oauth-config\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:20:34.482006 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:20:34.482009 2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cdea5bab-e53e-47ff-b1bf-b5b8319ddeea-trusted-ca-bundle\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:20:34.482006 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:20:34.482018 2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cdea5bab-e53e-47ff-b1bf-b5b8319ddeea-console-serving-cert\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:20:34.482257 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:20:34.482027 2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cdea5bab-e53e-47ff-b1bf-b5b8319ddeea-oauth-serving-cert\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:20:34.482257 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:20:34.482036 2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cdea5bab-e53e-47ff-b1bf-b5b8319ddeea-service-ca\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:20:34.482257 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:20:34.482044 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gcm9d\" (UniqueName: \"kubernetes.io/projected/cdea5bab-e53e-47ff-b1bf-b5b8319ddeea-kube-api-access-gcm9d\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:20:35.026826 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:20:35.026784 2577 generic.go:356] "Generic (PLEG): container finished" podID="cdea5bab-e53e-47ff-b1bf-b5b8319ddeea" containerID="dc5ac5b0bcd65ad1e61e129c9eac71f13d10fa6a3d6a68ae37ab272f91b89496" exitCode=0 Jul 12 18:20:35.027234 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:20:35.026838 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8554b68bb5-rsshs" Jul 12 18:20:35.027234 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:20:35.026864 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8554b68bb5-rsshs" event={"ID":"cdea5bab-e53e-47ff-b1bf-b5b8319ddeea","Type":"ContainerDied","Data":"dc5ac5b0bcd65ad1e61e129c9eac71f13d10fa6a3d6a68ae37ab272f91b89496"} Jul 12 18:20:35.027234 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:20:35.026912 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8554b68bb5-rsshs" event={"ID":"cdea5bab-e53e-47ff-b1bf-b5b8319ddeea","Type":"ContainerDied","Data":"d86b8464941aa63e98e07276f4e40811bb4d35772de568fda51d19561c388266"} Jul 12 18:20:35.027234 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:20:35.026932 2577 scope.go:117] "RemoveContainer" containerID="dc5ac5b0bcd65ad1e61e129c9eac71f13d10fa6a3d6a68ae37ab272f91b89496" Jul 12 18:20:35.035187 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:20:35.035167 2577 scope.go:117] "RemoveContainer" containerID="dc5ac5b0bcd65ad1e61e129c9eac71f13d10fa6a3d6a68ae37ab272f91b89496" Jul 12 18:20:35.035462 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:20:35.035441 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc5ac5b0bcd65ad1e61e129c9eac71f13d10fa6a3d6a68ae37ab272f91b89496\": container with ID starting with dc5ac5b0bcd65ad1e61e129c9eac71f13d10fa6a3d6a68ae37ab272f91b89496 not found: ID does not exist" containerID="dc5ac5b0bcd65ad1e61e129c9eac71f13d10fa6a3d6a68ae37ab272f91b89496" Jul 12 18:20:35.035511 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:20:35.035470 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc5ac5b0bcd65ad1e61e129c9eac71f13d10fa6a3d6a68ae37ab272f91b89496"} err="failed to get container status \"dc5ac5b0bcd65ad1e61e129c9eac71f13d10fa6a3d6a68ae37ab272f91b89496\": rpc error: code = NotFound desc = could not find container \"dc5ac5b0bcd65ad1e61e129c9eac71f13d10fa6a3d6a68ae37ab272f91b89496\": container with ID starting with dc5ac5b0bcd65ad1e61e129c9eac71f13d10fa6a3d6a68ae37ab272f91b89496 not found: ID does not exist" Jul 12 18:20:35.046425 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:20:35.046404 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8554b68bb5-rsshs"] Jul 12 18:20:35.050339 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:20:35.050319 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8554b68bb5-rsshs"] Jul 12 18:20:35.075046 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:20:35.075012 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cdea5bab-e53e-47ff-b1bf-b5b8319ddeea" path="/var/lib/kubelet/pods/cdea5bab-e53e-47ff-b1bf-b5b8319ddeea/volumes" Jul 12 18:21:07.548190 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:07.548155 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-686b8f579b-j6tn5"] Jul 12 18:21:07.548681 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:07.548419 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdea5bab-e53e-47ff-b1bf-b5b8319ddeea" containerName="console" Jul 12 18:21:07.548681 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:07.548430 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdea5bab-e53e-47ff-b1bf-b5b8319ddeea" containerName="console" Jul 12 18:21:07.548681 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:07.548488 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="cdea5bab-e53e-47ff-b1bf-b5b8319ddeea" containerName="console" Jul 12 18:21:07.551460 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:07.551437 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-686b8f579b-j6tn5" Jul 12 18:21:07.563297 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:07.563270 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-686b8f579b-j6tn5"] Jul 12 18:21:07.636354 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:07.636322 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cd14dd7d-4812-4fb7-9a42-067285985d2b-console-config\") pod \"console-686b8f579b-j6tn5\" (UID: \"cd14dd7d-4812-4fb7-9a42-067285985d2b\") " pod="openshift-console/console-686b8f579b-j6tn5" Jul 12 18:21:07.636354 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:07.636358 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cd14dd7d-4812-4fb7-9a42-067285985d2b-console-oauth-config\") pod \"console-686b8f579b-j6tn5\" (UID: \"cd14dd7d-4812-4fb7-9a42-067285985d2b\") " pod="openshift-console/console-686b8f579b-j6tn5" Jul 12 18:21:07.636655 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:07.636390 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2llb\" (UniqueName: \"kubernetes.io/projected/cd14dd7d-4812-4fb7-9a42-067285985d2b-kube-api-access-m2llb\") pod \"console-686b8f579b-j6tn5\" (UID: \"cd14dd7d-4812-4fb7-9a42-067285985d2b\") " pod="openshift-console/console-686b8f579b-j6tn5" Jul 12 18:21:07.636655 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:07.636492 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cd14dd7d-4812-4fb7-9a42-067285985d2b-service-ca\") pod \"console-686b8f579b-j6tn5\" (UID: \"cd14dd7d-4812-4fb7-9a42-067285985d2b\") " pod="openshift-console/console-686b8f579b-j6tn5" Jul 12 18:21:07.636655 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:07.636530 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cd14dd7d-4812-4fb7-9a42-067285985d2b-oauth-serving-cert\") pod \"console-686b8f579b-j6tn5\" (UID: \"cd14dd7d-4812-4fb7-9a42-067285985d2b\") " pod="openshift-console/console-686b8f579b-j6tn5" Jul 12 18:21:07.636655 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:07.636556 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd14dd7d-4812-4fb7-9a42-067285985d2b-trusted-ca-bundle\") pod \"console-686b8f579b-j6tn5\" (UID: \"cd14dd7d-4812-4fb7-9a42-067285985d2b\") " pod="openshift-console/console-686b8f579b-j6tn5" Jul 12 18:21:07.636655 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:07.636586 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cd14dd7d-4812-4fb7-9a42-067285985d2b-console-serving-cert\") pod \"console-686b8f579b-j6tn5\" (UID: \"cd14dd7d-4812-4fb7-9a42-067285985d2b\") " pod="openshift-console/console-686b8f579b-j6tn5" Jul 12 18:21:07.737527 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:07.737484 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cd14dd7d-4812-4fb7-9a42-067285985d2b-console-config\") pod \"console-686b8f579b-j6tn5\" (UID: \"cd14dd7d-4812-4fb7-9a42-067285985d2b\") " pod="openshift-console/console-686b8f579b-j6tn5" Jul 12 18:21:07.737717 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:07.737543 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cd14dd7d-4812-4fb7-9a42-067285985d2b-console-oauth-config\") pod \"console-686b8f579b-j6tn5\" (UID: \"cd14dd7d-4812-4fb7-9a42-067285985d2b\") " pod="openshift-console/console-686b8f579b-j6tn5" Jul 12 18:21:07.737717 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:07.737582 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2llb\" (UniqueName: \"kubernetes.io/projected/cd14dd7d-4812-4fb7-9a42-067285985d2b-kube-api-access-m2llb\") pod \"console-686b8f579b-j6tn5\" (UID: \"cd14dd7d-4812-4fb7-9a42-067285985d2b\") " pod="openshift-console/console-686b8f579b-j6tn5" Jul 12 18:21:07.737717 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:07.737652 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cd14dd7d-4812-4fb7-9a42-067285985d2b-service-ca\") pod \"console-686b8f579b-j6tn5\" (UID: \"cd14dd7d-4812-4fb7-9a42-067285985d2b\") " pod="openshift-console/console-686b8f579b-j6tn5" Jul 12 18:21:07.737717 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:07.737685 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cd14dd7d-4812-4fb7-9a42-067285985d2b-oauth-serving-cert\") pod \"console-686b8f579b-j6tn5\" (UID: \"cd14dd7d-4812-4fb7-9a42-067285985d2b\") " pod="openshift-console/console-686b8f579b-j6tn5" Jul 12 18:21:07.737717 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:07.737714 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd14dd7d-4812-4fb7-9a42-067285985d2b-trusted-ca-bundle\") pod \"console-686b8f579b-j6tn5\" (UID: \"cd14dd7d-4812-4fb7-9a42-067285985d2b\") " pod="openshift-console/console-686b8f579b-j6tn5" Jul 12 18:21:07.737989 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:07.737750 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cd14dd7d-4812-4fb7-9a42-067285985d2b-console-serving-cert\") pod \"console-686b8f579b-j6tn5\" (UID: \"cd14dd7d-4812-4fb7-9a42-067285985d2b\") " pod="openshift-console/console-686b8f579b-j6tn5" Jul 12 18:21:07.738233 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:07.738208 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cd14dd7d-4812-4fb7-9a42-067285985d2b-console-config\") pod \"console-686b8f579b-j6tn5\" (UID: \"cd14dd7d-4812-4fb7-9a42-067285985d2b\") " pod="openshift-console/console-686b8f579b-j6tn5" Jul 12 18:21:07.738370 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:07.738338 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cd14dd7d-4812-4fb7-9a42-067285985d2b-service-ca\") pod \"console-686b8f579b-j6tn5\" (UID: \"cd14dd7d-4812-4fb7-9a42-067285985d2b\") " pod="openshift-console/console-686b8f579b-j6tn5" Jul 12 18:21:07.738473 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:07.738345 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cd14dd7d-4812-4fb7-9a42-067285985d2b-oauth-serving-cert\") pod \"console-686b8f579b-j6tn5\" (UID: \"cd14dd7d-4812-4fb7-9a42-067285985d2b\") " pod="openshift-console/console-686b8f579b-j6tn5" Jul 12 18:21:07.738615 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:07.738591 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd14dd7d-4812-4fb7-9a42-067285985d2b-trusted-ca-bundle\") pod \"console-686b8f579b-j6tn5\" (UID: \"cd14dd7d-4812-4fb7-9a42-067285985d2b\") " pod="openshift-console/console-686b8f579b-j6tn5" Jul 12 18:21:07.740012 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:07.739990 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cd14dd7d-4812-4fb7-9a42-067285985d2b-console-oauth-config\") pod \"console-686b8f579b-j6tn5\" (UID: \"cd14dd7d-4812-4fb7-9a42-067285985d2b\") " pod="openshift-console/console-686b8f579b-j6tn5" Jul 12 18:21:07.740131 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:07.740110 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cd14dd7d-4812-4fb7-9a42-067285985d2b-console-serving-cert\") pod \"console-686b8f579b-j6tn5\" (UID: \"cd14dd7d-4812-4fb7-9a42-067285985d2b\") " pod="openshift-console/console-686b8f579b-j6tn5" Jul 12 18:21:07.746424 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:07.746403 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2llb\" (UniqueName: \"kubernetes.io/projected/cd14dd7d-4812-4fb7-9a42-067285985d2b-kube-api-access-m2llb\") pod \"console-686b8f579b-j6tn5\" (UID: \"cd14dd7d-4812-4fb7-9a42-067285985d2b\") " pod="openshift-console/console-686b8f579b-j6tn5" Jul 12 18:21:07.860696 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:07.860574 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-686b8f579b-j6tn5" Jul 12 18:21:07.987206 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:07.987176 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-686b8f579b-j6tn5"] Jul 12 18:21:07.989963 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:21:07.989937 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd14dd7d_4812_4fb7_9a42_067285985d2b.slice/crio-1c29013701373a8575949c306a7e95f318dd8be66b6044369f8b1d51dc8708ce WatchSource:0}: Error finding container 1c29013701373a8575949c306a7e95f318dd8be66b6044369f8b1d51dc8708ce: Status 404 returned error can't find the container with id 1c29013701373a8575949c306a7e95f318dd8be66b6044369f8b1d51dc8708ce Jul 12 18:21:08.147487 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:08.147409 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-686b8f579b-j6tn5" event={"ID":"cd14dd7d-4812-4fb7-9a42-067285985d2b","Type":"ContainerStarted","Data":"deecdda4012a9aa371f32c8ab030bfee758061b2c283100296cf591fc5f6d0d2"} Jul 12 18:21:08.147487 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:08.147457 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-686b8f579b-j6tn5" event={"ID":"cd14dd7d-4812-4fb7-9a42-067285985d2b","Type":"ContainerStarted","Data":"1c29013701373a8575949c306a7e95f318dd8be66b6044369f8b1d51dc8708ce"} Jul 12 18:21:08.163916 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:08.163867 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-686b8f579b-j6tn5" podStartSLOduration=1.1638506259999999 podStartE2EDuration="1.163850626s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:21:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:21:08.163110395 +0000 UTC m=+241.290015200" watchObservedRunningTime="2026-07-12 18:21:08.163850626 +0000 UTC m=+241.290755491" Jul 12 18:21:17.860912 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:17.860876 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-686b8f579b-j6tn5" Jul 12 18:21:17.861385 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:17.860961 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-686b8f579b-j6tn5" Jul 12 18:21:17.865466 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:17.865442 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-686b8f579b-j6tn5" Jul 12 18:21:18.188419 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:18.188399 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-686b8f579b-j6tn5" Jul 12 18:21:18.229936 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:18.229901 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-65d988bc4f-jj9js"] Jul 12 18:21:43.251184 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:43.251079 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-65d988bc4f-jj9js" podUID="cc99a43a-4854-4f4c-8f5b-3df24052e69d" containerName="console" containerID="cri-o://7b1f0f4a12d3b3765c5fbec21b799a44de0a450b49a15fc395ace6e93e3bf10d" gracePeriod=15 Jul 12 18:21:43.487151 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:43.487126 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65d988bc4f-jj9js" Jul 12 18:21:43.553450 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:43.553370 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cc99a43a-4854-4f4c-8f5b-3df24052e69d-console-serving-cert\") pod \"cc99a43a-4854-4f4c-8f5b-3df24052e69d\" (UID: \"cc99a43a-4854-4f4c-8f5b-3df24052e69d\") " Jul 12 18:21:43.553450 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:43.553407 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc99a43a-4854-4f4c-8f5b-3df24052e69d-trusted-ca-bundle\") pod \"cc99a43a-4854-4f4c-8f5b-3df24052e69d\" (UID: \"cc99a43a-4854-4f4c-8f5b-3df24052e69d\") " Jul 12 18:21:43.553450 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:43.553430 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cc99a43a-4854-4f4c-8f5b-3df24052e69d-console-oauth-config\") pod \"cc99a43a-4854-4f4c-8f5b-3df24052e69d\" (UID: \"cc99a43a-4854-4f4c-8f5b-3df24052e69d\") " Jul 12 18:21:43.553728 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:43.553461 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hh4xw\" (UniqueName: \"kubernetes.io/projected/cc99a43a-4854-4f4c-8f5b-3df24052e69d-kube-api-access-hh4xw\") pod \"cc99a43a-4854-4f4c-8f5b-3df24052e69d\" (UID: \"cc99a43a-4854-4f4c-8f5b-3df24052e69d\") " Jul 12 18:21:43.553728 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:43.553645 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cc99a43a-4854-4f4c-8f5b-3df24052e69d-console-config\") pod \"cc99a43a-4854-4f4c-8f5b-3df24052e69d\" (UID: \"cc99a43a-4854-4f4c-8f5b-3df24052e69d\") " Jul 12 18:21:43.553728 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:43.553694 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cc99a43a-4854-4f4c-8f5b-3df24052e69d-oauth-serving-cert\") pod \"cc99a43a-4854-4f4c-8f5b-3df24052e69d\" (UID: \"cc99a43a-4854-4f4c-8f5b-3df24052e69d\") " Jul 12 18:21:43.553886 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:43.553744 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cc99a43a-4854-4f4c-8f5b-3df24052e69d-service-ca\") pod \"cc99a43a-4854-4f4c-8f5b-3df24052e69d\" (UID: \"cc99a43a-4854-4f4c-8f5b-3df24052e69d\") " Jul 12 18:21:43.554029 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:43.554003 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc99a43a-4854-4f4c-8f5b-3df24052e69d-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "cc99a43a-4854-4f4c-8f5b-3df24052e69d" (UID: "cc99a43a-4854-4f4c-8f5b-3df24052e69d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:21:43.554029 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:43.554015 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc99a43a-4854-4f4c-8f5b-3df24052e69d-console-config" (OuterVolumeSpecName: "console-config") pod "cc99a43a-4854-4f4c-8f5b-3df24052e69d" (UID: "cc99a43a-4854-4f4c-8f5b-3df24052e69d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:21:43.554176 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:43.554140 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc99a43a-4854-4f4c-8f5b-3df24052e69d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "cc99a43a-4854-4f4c-8f5b-3df24052e69d" (UID: "cc99a43a-4854-4f4c-8f5b-3df24052e69d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:21:43.554239 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:43.554219 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc99a43a-4854-4f4c-8f5b-3df24052e69d-service-ca" (OuterVolumeSpecName: "service-ca") pod "cc99a43a-4854-4f4c-8f5b-3df24052e69d" (UID: "cc99a43a-4854-4f4c-8f5b-3df24052e69d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:21:43.555612 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:43.555588 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc99a43a-4854-4f4c-8f5b-3df24052e69d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "cc99a43a-4854-4f4c-8f5b-3df24052e69d" (UID: "cc99a43a-4854-4f4c-8f5b-3df24052e69d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 18:21:43.555967 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:43.555948 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc99a43a-4854-4f4c-8f5b-3df24052e69d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "cc99a43a-4854-4f4c-8f5b-3df24052e69d" (UID: "cc99a43a-4854-4f4c-8f5b-3df24052e69d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 18:21:43.555967 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:43.555954 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc99a43a-4854-4f4c-8f5b-3df24052e69d-kube-api-access-hh4xw" (OuterVolumeSpecName: "kube-api-access-hh4xw") pod "cc99a43a-4854-4f4c-8f5b-3df24052e69d" (UID: "cc99a43a-4854-4f4c-8f5b-3df24052e69d"). InnerVolumeSpecName "kube-api-access-hh4xw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:21:43.654521 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:43.654490 2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cc99a43a-4854-4f4c-8f5b-3df24052e69d-console-serving-cert\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:21:43.654521 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:43.654518 2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc99a43a-4854-4f4c-8f5b-3df24052e69d-trusted-ca-bundle\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:21:43.654521 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:43.654527 2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cc99a43a-4854-4f4c-8f5b-3df24052e69d-console-oauth-config\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:21:43.654755 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:43.654536 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hh4xw\" (UniqueName: \"kubernetes.io/projected/cc99a43a-4854-4f4c-8f5b-3df24052e69d-kube-api-access-hh4xw\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:21:43.654755 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:43.654545 2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cc99a43a-4854-4f4c-8f5b-3df24052e69d-console-config\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:21:43.654755 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:43.654556 2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cc99a43a-4854-4f4c-8f5b-3df24052e69d-oauth-serving-cert\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:21:43.654755 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:43.654565 2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cc99a43a-4854-4f4c-8f5b-3df24052e69d-service-ca\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:21:44.277902 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:44.277866 2577 generic.go:356] "Generic (PLEG): container finished" podID="cc99a43a-4854-4f4c-8f5b-3df24052e69d" containerID="7b1f0f4a12d3b3765c5fbec21b799a44de0a450b49a15fc395ace6e93e3bf10d" exitCode=0 Jul 12 18:21:44.278286 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:44.277933 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65d988bc4f-jj9js" Jul 12 18:21:44.278286 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:44.277934 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65d988bc4f-jj9js" event={"ID":"cc99a43a-4854-4f4c-8f5b-3df24052e69d","Type":"ContainerDied","Data":"7b1f0f4a12d3b3765c5fbec21b799a44de0a450b49a15fc395ace6e93e3bf10d"} Jul 12 18:21:44.278286 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:44.278043 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65d988bc4f-jj9js" event={"ID":"cc99a43a-4854-4f4c-8f5b-3df24052e69d","Type":"ContainerDied","Data":"3656497f5dc3c916f4d01bd9cee8dc08d5abf883da639b3abd93031f0cba4257"} Jul 12 18:21:44.278286 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:44.278063 2577 scope.go:117] "RemoveContainer" containerID="7b1f0f4a12d3b3765c5fbec21b799a44de0a450b49a15fc395ace6e93e3bf10d" Jul 12 18:21:44.286263 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:44.286244 2577 scope.go:117] "RemoveContainer" containerID="7b1f0f4a12d3b3765c5fbec21b799a44de0a450b49a15fc395ace6e93e3bf10d" Jul 12 18:21:44.286577 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:21:44.286558 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b1f0f4a12d3b3765c5fbec21b799a44de0a450b49a15fc395ace6e93e3bf10d\": container with ID starting with 7b1f0f4a12d3b3765c5fbec21b799a44de0a450b49a15fc395ace6e93e3bf10d not found: ID does not exist" containerID="7b1f0f4a12d3b3765c5fbec21b799a44de0a450b49a15fc395ace6e93e3bf10d" Jul 12 18:21:44.286621 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:44.286586 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b1f0f4a12d3b3765c5fbec21b799a44de0a450b49a15fc395ace6e93e3bf10d"} err="failed to get container status \"7b1f0f4a12d3b3765c5fbec21b799a44de0a450b49a15fc395ace6e93e3bf10d\": rpc error: code = NotFound desc = could not find container \"7b1f0f4a12d3b3765c5fbec21b799a44de0a450b49a15fc395ace6e93e3bf10d\": container with ID starting with 7b1f0f4a12d3b3765c5fbec21b799a44de0a450b49a15fc395ace6e93e3bf10d not found: ID does not exist" Jul 12 18:21:44.299563 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:44.299535 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-65d988bc4f-jj9js"] Jul 12 18:21:44.302543 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:44.302507 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-65d988bc4f-jj9js"] Jul 12 18:21:45.074124 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:45.074093 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc99a43a-4854-4f4c-8f5b-3df24052e69d" path="/var/lib/kubelet/pods/cc99a43a-4854-4f4c-8f5b-3df24052e69d/volumes" Jul 12 18:21:51.920527 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:51.920493 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jfvjq"] Jul 12 18:21:51.920988 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:51.920778 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc99a43a-4854-4f4c-8f5b-3df24052e69d" containerName="console" Jul 12 18:21:51.920988 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:51.920789 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc99a43a-4854-4f4c-8f5b-3df24052e69d" containerName="console" Jul 12 18:21:51.920988 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:51.920871 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc99a43a-4854-4f4c-8f5b-3df24052e69d" containerName="console" Jul 12 18:21:51.925205 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:51.925188 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jfvjq" Jul 12 18:21:51.928140 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:51.928114 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 12 18:21:51.928259 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:51.928118 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-wg4d9\"" Jul 12 18:21:51.928865 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:51.928849 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 12 18:21:51.932234 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:51.932131 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jfvjq"] Jul 12 18:21:52.020959 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:52.020928 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d22nq\" (UniqueName: \"kubernetes.io/projected/5156f0ac-f2ab-49d1-acf6-425169aac537-kube-api-access-d22nq\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jfvjq\" (UID: \"5156f0ac-f2ab-49d1-acf6-425169aac537\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jfvjq" Jul 12 18:21:52.021144 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:52.020977 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5156f0ac-f2ab-49d1-acf6-425169aac537-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jfvjq\" (UID: \"5156f0ac-f2ab-49d1-acf6-425169aac537\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jfvjq" Jul 12 18:21:52.021144 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:52.021001 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5156f0ac-f2ab-49d1-acf6-425169aac537-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jfvjq\" (UID: \"5156f0ac-f2ab-49d1-acf6-425169aac537\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jfvjq" Jul 12 18:21:52.122008 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:52.121976 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d22nq\" (UniqueName: \"kubernetes.io/projected/5156f0ac-f2ab-49d1-acf6-425169aac537-kube-api-access-d22nq\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jfvjq\" (UID: \"5156f0ac-f2ab-49d1-acf6-425169aac537\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jfvjq" Jul 12 18:21:52.122120 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:52.122025 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5156f0ac-f2ab-49d1-acf6-425169aac537-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jfvjq\" (UID: \"5156f0ac-f2ab-49d1-acf6-425169aac537\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jfvjq" Jul 12 18:21:52.122120 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:52.122049 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5156f0ac-f2ab-49d1-acf6-425169aac537-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jfvjq\" (UID: \"5156f0ac-f2ab-49d1-acf6-425169aac537\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jfvjq" Jul 12 18:21:52.122393 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:52.122378 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5156f0ac-f2ab-49d1-acf6-425169aac537-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jfvjq\" (UID: \"5156f0ac-f2ab-49d1-acf6-425169aac537\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jfvjq" Jul 12 18:21:52.122436 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:52.122407 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5156f0ac-f2ab-49d1-acf6-425169aac537-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jfvjq\" (UID: \"5156f0ac-f2ab-49d1-acf6-425169aac537\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jfvjq" Jul 12 18:21:52.132310 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:52.132282 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d22nq\" (UniqueName: \"kubernetes.io/projected/5156f0ac-f2ab-49d1-acf6-425169aac537-kube-api-access-d22nq\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jfvjq\" (UID: \"5156f0ac-f2ab-49d1-acf6-425169aac537\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jfvjq" Jul 12 18:21:52.234882 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:52.234855 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jfvjq" Jul 12 18:21:52.359886 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:52.359833 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jfvjq"] Jul 12 18:21:52.362446 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:21:52.362414 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5156f0ac_f2ab_49d1_acf6_425169aac537.slice/crio-abd4dc8b569fc34746a4f84d00fd11bacb6d0b5a3cfb43c88d35e5acfa5753d3 WatchSource:0}: Error finding container abd4dc8b569fc34746a4f84d00fd11bacb6d0b5a3cfb43c88d35e5acfa5753d3: Status 404 returned error can't find the container with id abd4dc8b569fc34746a4f84d00fd11bacb6d0b5a3cfb43c88d35e5acfa5753d3 Jul 12 18:21:53.316665 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:21:53.316610 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jfvjq" event={"ID":"5156f0ac-f2ab-49d1-acf6-425169aac537","Type":"ContainerStarted","Data":"abd4dc8b569fc34746a4f84d00fd11bacb6d0b5a3cfb43c88d35e5acfa5753d3"} Jul 12 18:22:00.346585 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:00.346546 2577 generic.go:356] "Generic (PLEG): container finished" podID="5156f0ac-f2ab-49d1-acf6-425169aac537" containerID="06949c419642521c51a5568a43860bed727f81fe6e5d92f8ab12be79b2496cb2" exitCode=0 Jul 12 18:22:00.346585 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:00.346589 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jfvjq" event={"ID":"5156f0ac-f2ab-49d1-acf6-425169aac537","Type":"ContainerDied","Data":"06949c419642521c51a5568a43860bed727f81fe6e5d92f8ab12be79b2496cb2"} Jul 12 18:22:02.355962 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:02.355931 2577 generic.go:356] "Generic (PLEG): container finished" podID="5156f0ac-f2ab-49d1-acf6-425169aac537" containerID="5bda3faedd80d8576810f3089b3bac3421f7c1951988748da2d85e8f88eb3d7e" exitCode=0 Jul 12 18:22:02.356318 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:02.356000 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jfvjq" event={"ID":"5156f0ac-f2ab-49d1-acf6-425169aac537","Type":"ContainerDied","Data":"5bda3faedd80d8576810f3089b3bac3421f7c1951988748da2d85e8f88eb3d7e"} Jul 12 18:22:09.822389 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:09.822360 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-spzqd_ed115460-b220-476d-b9d5-65f95ec3d059/ovn-acl-logging/0.log" Jul 12 18:22:09.822830 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:09.822360 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-spzqd_ed115460-b220-476d-b9d5-65f95ec3d059/ovn-acl-logging/0.log" Jul 12 18:22:09.827985 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:09.827967 2577 kubelet.go:1648] "Image garbage collection succeeded" Jul 12 18:22:10.388705 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:10.388674 2577 generic.go:356] "Generic (PLEG): container finished" podID="5156f0ac-f2ab-49d1-acf6-425169aac537" containerID="a25627e0c7c7ed279a25fa8bd71d427ebb2feea7548768ab7fe9f4a612ac0777" exitCode=0 Jul 12 18:22:10.388876 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:10.388712 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jfvjq" event={"ID":"5156f0ac-f2ab-49d1-acf6-425169aac537","Type":"ContainerDied","Data":"a25627e0c7c7ed279a25fa8bd71d427ebb2feea7548768ab7fe9f4a612ac0777"} Jul 12 18:22:11.513214 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:11.513188 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jfvjq" Jul 12 18:22:11.593214 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:11.593170 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5156f0ac-f2ab-49d1-acf6-425169aac537-util\") pod \"5156f0ac-f2ab-49d1-acf6-425169aac537\" (UID: \"5156f0ac-f2ab-49d1-acf6-425169aac537\") " Jul 12 18:22:11.593214 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:11.593211 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5156f0ac-f2ab-49d1-acf6-425169aac537-bundle\") pod \"5156f0ac-f2ab-49d1-acf6-425169aac537\" (UID: \"5156f0ac-f2ab-49d1-acf6-425169aac537\") " Jul 12 18:22:11.593415 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:11.593265 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d22nq\" (UniqueName: \"kubernetes.io/projected/5156f0ac-f2ab-49d1-acf6-425169aac537-kube-api-access-d22nq\") pod \"5156f0ac-f2ab-49d1-acf6-425169aac537\" (UID: \"5156f0ac-f2ab-49d1-acf6-425169aac537\") " Jul 12 18:22:11.593941 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:11.593919 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5156f0ac-f2ab-49d1-acf6-425169aac537-bundle" (OuterVolumeSpecName: "bundle") pod "5156f0ac-f2ab-49d1-acf6-425169aac537" (UID: "5156f0ac-f2ab-49d1-acf6-425169aac537"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:22:11.595470 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:11.595446 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5156f0ac-f2ab-49d1-acf6-425169aac537-kube-api-access-d22nq" (OuterVolumeSpecName: "kube-api-access-d22nq") pod "5156f0ac-f2ab-49d1-acf6-425169aac537" (UID: "5156f0ac-f2ab-49d1-acf6-425169aac537"). InnerVolumeSpecName "kube-api-access-d22nq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:22:11.597665 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:11.597642 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5156f0ac-f2ab-49d1-acf6-425169aac537-util" (OuterVolumeSpecName: "util") pod "5156f0ac-f2ab-49d1-acf6-425169aac537" (UID: "5156f0ac-f2ab-49d1-acf6-425169aac537"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:22:11.694506 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:11.694467 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5156f0ac-f2ab-49d1-acf6-425169aac537-util\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:22:11.694506 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:11.694501 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5156f0ac-f2ab-49d1-acf6-425169aac537-bundle\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:22:11.694741 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:11.694517 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d22nq\" (UniqueName: \"kubernetes.io/projected/5156f0ac-f2ab-49d1-acf6-425169aac537-kube-api-access-d22nq\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:22:12.397997 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:12.397964 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jfvjq" event={"ID":"5156f0ac-f2ab-49d1-acf6-425169aac537","Type":"ContainerDied","Data":"abd4dc8b569fc34746a4f84d00fd11bacb6d0b5a3cfb43c88d35e5acfa5753d3"} Jul 12 18:22:12.398160 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:12.398005 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jfvjq" Jul 12 18:22:12.398160 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:12.398014 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="abd4dc8b569fc34746a4f84d00fd11bacb6d0b5a3cfb43c88d35e5acfa5753d3" Jul 12 18:22:19.888537 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:19.888501 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4vnrq"] Jul 12 18:22:19.888940 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:19.888819 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5156f0ac-f2ab-49d1-acf6-425169aac537" containerName="util" Jul 12 18:22:19.888940 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:19.888831 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5156f0ac-f2ab-49d1-acf6-425169aac537" containerName="util" Jul 12 18:22:19.888940 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:19.888838 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5156f0ac-f2ab-49d1-acf6-425169aac537" containerName="pull" Jul 12 18:22:19.888940 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:19.888843 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5156f0ac-f2ab-49d1-acf6-425169aac537" containerName="pull" Jul 12 18:22:19.888940 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:19.888857 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5156f0ac-f2ab-49d1-acf6-425169aac537" containerName="extract" Jul 12 18:22:19.888940 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:19.888864 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5156f0ac-f2ab-49d1-acf6-425169aac537" containerName="extract" Jul 12 18:22:19.888940 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:19.888909 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="5156f0ac-f2ab-49d1-acf6-425169aac537" containerName="extract" Jul 12 18:22:19.939701 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:19.939670 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4vnrq"] Jul 12 18:22:19.939857 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:19.939769 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4vnrq" Jul 12 18:22:19.942419 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:19.942400 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 12 18:22:19.942577 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:19.942557 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-6nz5x\"" Jul 12 18:22:19.942687 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:19.942590 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 12 18:22:20.062524 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:20.062481 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e09d226b-564e-4357-87da-2c3f9147508d-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-4vnrq\" (UID: \"e09d226b-564e-4357-87da-2c3f9147508d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4vnrq" Jul 12 18:22:20.062693 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:20.062590 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6fhn\" (UniqueName: \"kubernetes.io/projected/e09d226b-564e-4357-87da-2c3f9147508d-kube-api-access-x6fhn\") pod \"cert-manager-operator-controller-manager-55c444bf86-4vnrq\" (UID: \"e09d226b-564e-4357-87da-2c3f9147508d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4vnrq" Jul 12 18:22:20.163707 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:20.163620 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e09d226b-564e-4357-87da-2c3f9147508d-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-4vnrq\" (UID: \"e09d226b-564e-4357-87da-2c3f9147508d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4vnrq" Jul 12 18:22:20.163832 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:20.163720 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6fhn\" (UniqueName: \"kubernetes.io/projected/e09d226b-564e-4357-87da-2c3f9147508d-kube-api-access-x6fhn\") pod \"cert-manager-operator-controller-manager-55c444bf86-4vnrq\" (UID: \"e09d226b-564e-4357-87da-2c3f9147508d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4vnrq" Jul 12 18:22:20.164011 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:20.163994 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e09d226b-564e-4357-87da-2c3f9147508d-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-4vnrq\" (UID: \"e09d226b-564e-4357-87da-2c3f9147508d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4vnrq" Jul 12 18:22:20.171862 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:20.171838 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6fhn\" (UniqueName: \"kubernetes.io/projected/e09d226b-564e-4357-87da-2c3f9147508d-kube-api-access-x6fhn\") pod \"cert-manager-operator-controller-manager-55c444bf86-4vnrq\" (UID: \"e09d226b-564e-4357-87da-2c3f9147508d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4vnrq" Jul 12 18:22:20.248708 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:20.248676 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4vnrq" Jul 12 18:22:20.398125 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:20.398097 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4vnrq"] Jul 12 18:22:20.399780 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:22:20.399752 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode09d226b_564e_4357_87da_2c3f9147508d.slice/crio-36d5174f1ee181e5ba52e000cd98d3bd108da9e58113f91a6450154572a2cb44 WatchSource:0}: Error finding container 36d5174f1ee181e5ba52e000cd98d3bd108da9e58113f91a6450154572a2cb44: Status 404 returned error can't find the container with id 36d5174f1ee181e5ba52e000cd98d3bd108da9e58113f91a6450154572a2cb44 Jul 12 18:22:20.402413 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:20.402397 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 18:22:20.427475 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:20.427408 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4vnrq" event={"ID":"e09d226b-564e-4357-87da-2c3f9147508d","Type":"ContainerStarted","Data":"36d5174f1ee181e5ba52e000cd98d3bd108da9e58113f91a6450154572a2cb44"} Jul 12 18:22:22.439618 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:22.439565 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4vnrq" event={"ID":"e09d226b-564e-4357-87da-2c3f9147508d","Type":"ContainerStarted","Data":"e34d8cf7bd2a9b44009c5c16785e9ee35f57662c3643cfef7c07b790531c6ba5"} Jul 12 18:22:22.478002 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:22.477947 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4vnrq" podStartSLOduration=2.055214792 podStartE2EDuration="3.477930397s" totalImagesPullingTime="1.422715605s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:22:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:22:22.475494248 +0000 UTC m=+315.602399064" watchObservedRunningTime="2026-07-12 18:22:22.477930397 +0000 UTC m=+315.604835202" Jul 12 18:22:23.637184 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:23.637153 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl88fb"] Jul 12 18:22:23.639433 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:23.639413 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl88fb" Jul 12 18:22:23.642036 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:23.642018 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 12 18:22:23.642283 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:23.642269 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 12 18:22:23.642331 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:23.642276 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-wg4d9\"" Jul 12 18:22:23.654328 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:23.654307 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl88fb"] Jul 12 18:22:23.697785 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:23.697758 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-78lmg"] Jul 12 18:22:23.699722 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:23.699704 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-78lmg" Jul 12 18:22:23.702111 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:23.702092 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-l4mb6\"" Jul 12 18:22:23.702111 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:23.702103 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 12 18:22:23.702253 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:23.702124 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 12 18:22:23.710308 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:23.710286 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-78lmg"] Jul 12 18:22:23.797038 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:23.796998 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2pwx\" (UniqueName: \"kubernetes.io/projected/9433ef06-b1bb-4af3-83bd-7ace8effcada-kube-api-access-h2pwx\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl88fb\" (UID: \"9433ef06-b1bb-4af3-83bd-7ace8effcada\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl88fb" Jul 12 18:22:23.797251 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:23.797067 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9433ef06-b1bb-4af3-83bd-7ace8effcada-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl88fb\" (UID: \"9433ef06-b1bb-4af3-83bd-7ace8effcada\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl88fb" Jul 12 18:22:23.797251 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:23.797096 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0cbf9f6c-a2e4-4e42-b2cf-7cc97a01d321-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-78lmg\" (UID: \"0cbf9f6c-a2e4-4e42-b2cf-7cc97a01d321\") " pod="cert-manager/cert-manager-webhook-6486ffd997-78lmg" Jul 12 18:22:23.797251 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:23.797121 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9433ef06-b1bb-4af3-83bd-7ace8effcada-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl88fb\" (UID: \"9433ef06-b1bb-4af3-83bd-7ace8effcada\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl88fb" Jul 12 18:22:23.797251 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:23.797174 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tslp7\" (UniqueName: \"kubernetes.io/projected/0cbf9f6c-a2e4-4e42-b2cf-7cc97a01d321-kube-api-access-tslp7\") pod \"cert-manager-webhook-6486ffd997-78lmg\" (UID: \"0cbf9f6c-a2e4-4e42-b2cf-7cc97a01d321\") " pod="cert-manager/cert-manager-webhook-6486ffd997-78lmg" Jul 12 18:22:23.898060 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:23.897978 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2pwx\" (UniqueName: \"kubernetes.io/projected/9433ef06-b1bb-4af3-83bd-7ace8effcada-kube-api-access-h2pwx\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl88fb\" (UID: \"9433ef06-b1bb-4af3-83bd-7ace8effcada\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl88fb" Jul 12 18:22:23.898060 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:23.898024 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9433ef06-b1bb-4af3-83bd-7ace8effcada-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl88fb\" (UID: \"9433ef06-b1bb-4af3-83bd-7ace8effcada\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl88fb" Jul 12 18:22:23.898235 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:23.898141 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0cbf9f6c-a2e4-4e42-b2cf-7cc97a01d321-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-78lmg\" (UID: \"0cbf9f6c-a2e4-4e42-b2cf-7cc97a01d321\") " pod="cert-manager/cert-manager-webhook-6486ffd997-78lmg" Jul 12 18:22:23.898235 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:23.898181 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9433ef06-b1bb-4af3-83bd-7ace8effcada-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl88fb\" (UID: \"9433ef06-b1bb-4af3-83bd-7ace8effcada\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl88fb" Jul 12 18:22:23.898235 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:23.898212 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tslp7\" (UniqueName: \"kubernetes.io/projected/0cbf9f6c-a2e4-4e42-b2cf-7cc97a01d321-kube-api-access-tslp7\") pod \"cert-manager-webhook-6486ffd997-78lmg\" (UID: \"0cbf9f6c-a2e4-4e42-b2cf-7cc97a01d321\") " pod="cert-manager/cert-manager-webhook-6486ffd997-78lmg" Jul 12 18:22:23.898572 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:23.898555 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9433ef06-b1bb-4af3-83bd-7ace8effcada-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl88fb\" (UID: \"9433ef06-b1bb-4af3-83bd-7ace8effcada\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl88fb" Jul 12 18:22:23.898615 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:23.898558 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9433ef06-b1bb-4af3-83bd-7ace8effcada-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl88fb\" (UID: \"9433ef06-b1bb-4af3-83bd-7ace8effcada\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl88fb" Jul 12 18:22:23.906816 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:23.906775 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0cbf9f6c-a2e4-4e42-b2cf-7cc97a01d321-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-78lmg\" (UID: \"0cbf9f6c-a2e4-4e42-b2cf-7cc97a01d321\") " pod="cert-manager/cert-manager-webhook-6486ffd997-78lmg" Jul 12 18:22:23.906929 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:23.906899 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tslp7\" (UniqueName: \"kubernetes.io/projected/0cbf9f6c-a2e4-4e42-b2cf-7cc97a01d321-kube-api-access-tslp7\") pod \"cert-manager-webhook-6486ffd997-78lmg\" (UID: \"0cbf9f6c-a2e4-4e42-b2cf-7cc97a01d321\") " pod="cert-manager/cert-manager-webhook-6486ffd997-78lmg" Jul 12 18:22:23.907035 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:23.907014 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2pwx\" (UniqueName: \"kubernetes.io/projected/9433ef06-b1bb-4af3-83bd-7ace8effcada-kube-api-access-h2pwx\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl88fb\" (UID: \"9433ef06-b1bb-4af3-83bd-7ace8effcada\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl88fb" Jul 12 18:22:23.953337 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:23.953293 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl88fb" Jul 12 18:22:24.020770 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:24.020744 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-78lmg" Jul 12 18:22:24.084238 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:24.084041 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl88fb"] Jul 12 18:22:24.086882 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:22:24.086850 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9433ef06_b1bb_4af3_83bd_7ace8effcada.slice/crio-109fca23f1ce083845e940fe71e6eb042a4d42fc1169800b050d91894b504aaa WatchSource:0}: Error finding container 109fca23f1ce083845e940fe71e6eb042a4d42fc1169800b050d91894b504aaa: Status 404 returned error can't find the container with id 109fca23f1ce083845e940fe71e6eb042a4d42fc1169800b050d91894b504aaa Jul 12 18:22:24.161791 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:24.161764 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-78lmg"] Jul 12 18:22:24.166146 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:22:24.166123 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0cbf9f6c_a2e4_4e42_b2cf_7cc97a01d321.slice/crio-2d6403edb7fa7cbfd8a697577222142f444bb40f38c35313c5c71f148acf30fc WatchSource:0}: Error finding container 2d6403edb7fa7cbfd8a697577222142f444bb40f38c35313c5c71f148acf30fc: Status 404 returned error can't find the container with id 2d6403edb7fa7cbfd8a697577222142f444bb40f38c35313c5c71f148acf30fc Jul 12 18:22:24.448651 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:24.448598 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-78lmg" event={"ID":"0cbf9f6c-a2e4-4e42-b2cf-7cc97a01d321","Type":"ContainerStarted","Data":"2d6403edb7fa7cbfd8a697577222142f444bb40f38c35313c5c71f148acf30fc"} Jul 12 18:22:24.450015 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:24.449984 2577 generic.go:356] "Generic (PLEG): container finished" podID="9433ef06-b1bb-4af3-83bd-7ace8effcada" containerID="3f04bf2f7a95bbbcca7e177a909b076df8af22d9ace9c8beb4c3df177dc23872" exitCode=0 Jul 12 18:22:24.450137 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:24.450028 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl88fb" event={"ID":"9433ef06-b1bb-4af3-83bd-7ace8effcada","Type":"ContainerDied","Data":"3f04bf2f7a95bbbcca7e177a909b076df8af22d9ace9c8beb4c3df177dc23872"} Jul 12 18:22:24.450137 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:24.450078 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl88fb" event={"ID":"9433ef06-b1bb-4af3-83bd-7ace8effcada","Type":"ContainerStarted","Data":"109fca23f1ce083845e940fe71e6eb042a4d42fc1169800b050d91894b504aaa"} Jul 12 18:22:26.416078 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:26.416044 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-vbxnc"] Jul 12 18:22:26.419159 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:26.419138 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-vbxnc" Jul 12 18:22:26.421516 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:26.421493 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-slbgl\"" Jul 12 18:22:26.427164 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:26.427115 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-vbxnc"] Jul 12 18:22:26.522779 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:26.522738 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5m66r\" (UniqueName: \"kubernetes.io/projected/273d875a-630e-4623-b25d-f483a12afd8b-kube-api-access-5m66r\") pod \"cert-manager-cainjector-559cdd8c87-vbxnc\" (UID: \"273d875a-630e-4623-b25d-f483a12afd8b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-vbxnc" Jul 12 18:22:26.522779 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:26.522791 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/273d875a-630e-4623-b25d-f483a12afd8b-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-vbxnc\" (UID: \"273d875a-630e-4623-b25d-f483a12afd8b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-vbxnc" Jul 12 18:22:26.623726 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:26.623687 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/273d875a-630e-4623-b25d-f483a12afd8b-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-vbxnc\" (UID: \"273d875a-630e-4623-b25d-f483a12afd8b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-vbxnc" Jul 12 18:22:26.623898 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:26.623800 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5m66r\" (UniqueName: \"kubernetes.io/projected/273d875a-630e-4623-b25d-f483a12afd8b-kube-api-access-5m66r\") pod \"cert-manager-cainjector-559cdd8c87-vbxnc\" (UID: \"273d875a-630e-4623-b25d-f483a12afd8b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-vbxnc" Jul 12 18:22:26.632124 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:26.632088 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/273d875a-630e-4623-b25d-f483a12afd8b-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-vbxnc\" (UID: \"273d875a-630e-4623-b25d-f483a12afd8b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-vbxnc" Jul 12 18:22:26.632247 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:26.632204 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5m66r\" (UniqueName: \"kubernetes.io/projected/273d875a-630e-4623-b25d-f483a12afd8b-kube-api-access-5m66r\") pod \"cert-manager-cainjector-559cdd8c87-vbxnc\" (UID: \"273d875a-630e-4623-b25d-f483a12afd8b\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-vbxnc" Jul 12 18:22:26.730836 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:26.730794 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-vbxnc" Jul 12 18:22:27.746465 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:27.746432 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-vbxnc"] Jul 12 18:22:27.750367 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:22:27.750326 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod273d875a_630e_4623_b25d_f483a12afd8b.slice/crio-1f80aed32e88f8964c71a5f0c30c8c99c969f9ab84462088ba0c8cc6bc3fed8f WatchSource:0}: Error finding container 1f80aed32e88f8964c71a5f0c30c8c99c969f9ab84462088ba0c8cc6bc3fed8f: Status 404 returned error can't find the container with id 1f80aed32e88f8964c71a5f0c30c8c99c969f9ab84462088ba0c8cc6bc3fed8f Jul 12 18:22:28.471234 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:28.471201 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-78lmg" event={"ID":"0cbf9f6c-a2e4-4e42-b2cf-7cc97a01d321","Type":"ContainerStarted","Data":"61bdb934a3890cb42e3e025520564afb1ce23313b215488cc86976bff6d63a2a"} Jul 12 18:22:28.471427 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:28.471352 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-78lmg" Jul 12 18:22:28.472955 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:28.472929 2577 generic.go:356] "Generic (PLEG): container finished" podID="9433ef06-b1bb-4af3-83bd-7ace8effcada" containerID="252aec796a7fdbb1e873c382dea91c61b0391bd260953e6c8e9274efa9a7c8da" exitCode=0 Jul 12 18:22:28.473065 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:28.473018 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl88fb" event={"ID":"9433ef06-b1bb-4af3-83bd-7ace8effcada","Type":"ContainerDied","Data":"252aec796a7fdbb1e873c382dea91c61b0391bd260953e6c8e9274efa9a7c8da"} Jul 12 18:22:28.474461 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:28.474437 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-vbxnc" event={"ID":"273d875a-630e-4623-b25d-f483a12afd8b","Type":"ContainerStarted","Data":"a36022fcdcbc2eff26833c85bd7373a1e549c3c487f357778698b91db8abee8c"} Jul 12 18:22:28.474531 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:28.474471 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-vbxnc" event={"ID":"273d875a-630e-4623-b25d-f483a12afd8b","Type":"ContainerStarted","Data":"1f80aed32e88f8964c71a5f0c30c8c99c969f9ab84462088ba0c8cc6bc3fed8f"} Jul 12 18:22:28.485743 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:28.485696 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-78lmg" podStartSLOduration=2.041331201 podStartE2EDuration="5.485680225s" totalImagesPullingTime="3.444349024s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:22:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:22:28.485576823 +0000 UTC m=+321.612481629" watchObservedRunningTime="2026-07-12 18:22:28.485680225 +0000 UTC m=+321.612585029" Jul 12 18:22:28.517302 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:28.517259 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-vbxnc" podStartSLOduration=2.517241029 podStartE2EDuration="2.517241029s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:22:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:22:28.515589862 +0000 UTC m=+321.642494666" watchObservedRunningTime="2026-07-12 18:22:28.517241029 +0000 UTC m=+321.644145833" Jul 12 18:22:29.481585 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:29.481552 2577 generic.go:356] "Generic (PLEG): container finished" podID="9433ef06-b1bb-4af3-83bd-7ace8effcada" containerID="e61d9d63aaec9deafcf77a07269dc63fc54fffad1494fc0fc836883213d9e909" exitCode=0 Jul 12 18:22:29.481989 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:29.481620 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl88fb" event={"ID":"9433ef06-b1bb-4af3-83bd-7ace8effcada","Type":"ContainerDied","Data":"e61d9d63aaec9deafcf77a07269dc63fc54fffad1494fc0fc836883213d9e909"} Jul 12 18:22:30.609377 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:30.609354 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl88fb" Jul 12 18:22:30.660064 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:30.660033 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9433ef06-b1bb-4af3-83bd-7ace8effcada-bundle\") pod \"9433ef06-b1bb-4af3-83bd-7ace8effcada\" (UID: \"9433ef06-b1bb-4af3-83bd-7ace8effcada\") " Jul 12 18:22:30.660218 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:30.660098 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h2pwx\" (UniqueName: \"kubernetes.io/projected/9433ef06-b1bb-4af3-83bd-7ace8effcada-kube-api-access-h2pwx\") pod \"9433ef06-b1bb-4af3-83bd-7ace8effcada\" (UID: \"9433ef06-b1bb-4af3-83bd-7ace8effcada\") " Jul 12 18:22:30.660218 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:30.660146 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9433ef06-b1bb-4af3-83bd-7ace8effcada-util\") pod \"9433ef06-b1bb-4af3-83bd-7ace8effcada\" (UID: \"9433ef06-b1bb-4af3-83bd-7ace8effcada\") " Jul 12 18:22:30.660514 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:30.660485 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9433ef06-b1bb-4af3-83bd-7ace8effcada-bundle" (OuterVolumeSpecName: "bundle") pod "9433ef06-b1bb-4af3-83bd-7ace8effcada" (UID: "9433ef06-b1bb-4af3-83bd-7ace8effcada"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:22:30.662255 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:30.662225 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9433ef06-b1bb-4af3-83bd-7ace8effcada-kube-api-access-h2pwx" (OuterVolumeSpecName: "kube-api-access-h2pwx") pod "9433ef06-b1bb-4af3-83bd-7ace8effcada" (UID: "9433ef06-b1bb-4af3-83bd-7ace8effcada"). InnerVolumeSpecName "kube-api-access-h2pwx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:22:30.664972 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:30.664946 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9433ef06-b1bb-4af3-83bd-7ace8effcada-util" (OuterVolumeSpecName: "util") pod "9433ef06-b1bb-4af3-83bd-7ace8effcada" (UID: "9433ef06-b1bb-4af3-83bd-7ace8effcada"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:22:30.761060 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:30.760985 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h2pwx\" (UniqueName: \"kubernetes.io/projected/9433ef06-b1bb-4af3-83bd-7ace8effcada-kube-api-access-h2pwx\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:22:30.761060 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:30.761011 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9433ef06-b1bb-4af3-83bd-7ace8effcada-util\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:22:30.761060 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:30.761020 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9433ef06-b1bb-4af3-83bd-7ace8effcada-bundle\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:22:31.491222 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:31.491195 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl88fb" Jul 12 18:22:31.491222 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:31.491214 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fl88fb" event={"ID":"9433ef06-b1bb-4af3-83bd-7ace8effcada","Type":"ContainerDied","Data":"109fca23f1ce083845e940fe71e6eb042a4d42fc1169800b050d91894b504aaa"} Jul 12 18:22:31.491421 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:31.491261 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="109fca23f1ce083845e940fe71e6eb042a4d42fc1169800b050d91894b504aaa" Jul 12 18:22:34.484186 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:34.484155 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-78lmg" Jul 12 18:22:36.497997 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:36.497961 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-92c98"] Jul 12 18:22:36.498358 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:36.498245 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9433ef06-b1bb-4af3-83bd-7ace8effcada" containerName="extract" Jul 12 18:22:36.498358 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:36.498255 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9433ef06-b1bb-4af3-83bd-7ace8effcada" containerName="extract" Jul 12 18:22:36.498358 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:36.498268 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9433ef06-b1bb-4af3-83bd-7ace8effcada" containerName="util" Jul 12 18:22:36.498358 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:36.498273 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9433ef06-b1bb-4af3-83bd-7ace8effcada" containerName="util" Jul 12 18:22:36.498358 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:36.498289 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9433ef06-b1bb-4af3-83bd-7ace8effcada" containerName="pull" Jul 12 18:22:36.498358 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:36.498295 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9433ef06-b1bb-4af3-83bd-7ace8effcada" containerName="pull" Jul 12 18:22:36.498358 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:36.498343 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9433ef06-b1bb-4af3-83bd-7ace8effcada" containerName="extract" Jul 12 18:22:36.501927 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:36.501906 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-92c98" Jul 12 18:22:36.504112 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:36.504097 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-cqkvg\"" Jul 12 18:22:36.511217 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:36.511196 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-92c98"] Jul 12 18:22:36.605281 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:36.605243 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1a1a61fe-3f6b-48e2-9d20-71c149ceed75-bound-sa-token\") pod \"cert-manager-55b4d954d5-92c98\" (UID: \"1a1a61fe-3f6b-48e2-9d20-71c149ceed75\") " pod="cert-manager/cert-manager-55b4d954d5-92c98" Jul 12 18:22:36.605458 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:36.605303 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8q6f\" (UniqueName: \"kubernetes.io/projected/1a1a61fe-3f6b-48e2-9d20-71c149ceed75-kube-api-access-r8q6f\") pod \"cert-manager-55b4d954d5-92c98\" (UID: \"1a1a61fe-3f6b-48e2-9d20-71c149ceed75\") " pod="cert-manager/cert-manager-55b4d954d5-92c98" Jul 12 18:22:36.706247 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:36.706210 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8q6f\" (UniqueName: \"kubernetes.io/projected/1a1a61fe-3f6b-48e2-9d20-71c149ceed75-kube-api-access-r8q6f\") pod \"cert-manager-55b4d954d5-92c98\" (UID: \"1a1a61fe-3f6b-48e2-9d20-71c149ceed75\") " pod="cert-manager/cert-manager-55b4d954d5-92c98" Jul 12 18:22:36.706438 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:36.706301 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1a1a61fe-3f6b-48e2-9d20-71c149ceed75-bound-sa-token\") pod \"cert-manager-55b4d954d5-92c98\" (UID: \"1a1a61fe-3f6b-48e2-9d20-71c149ceed75\") " pod="cert-manager/cert-manager-55b4d954d5-92c98" Jul 12 18:22:36.719271 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:36.719244 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1a1a61fe-3f6b-48e2-9d20-71c149ceed75-bound-sa-token\") pod \"cert-manager-55b4d954d5-92c98\" (UID: \"1a1a61fe-3f6b-48e2-9d20-71c149ceed75\") " pod="cert-manager/cert-manager-55b4d954d5-92c98" Jul 12 18:22:36.719439 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:36.719419 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8q6f\" (UniqueName: \"kubernetes.io/projected/1a1a61fe-3f6b-48e2-9d20-71c149ceed75-kube-api-access-r8q6f\") pod \"cert-manager-55b4d954d5-92c98\" (UID: \"1a1a61fe-3f6b-48e2-9d20-71c149ceed75\") " pod="cert-manager/cert-manager-55b4d954d5-92c98" Jul 12 18:22:36.810851 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:36.810743 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-92c98" Jul 12 18:22:36.945793 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:36.945767 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-92c98"] Jul 12 18:22:36.948159 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:22:36.948126 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1a1a61fe_3f6b_48e2_9d20_71c149ceed75.slice/crio-f9db2a0ec0891492eefa7cccc892e399d9115f941e63489637b6e72171ea0395 WatchSource:0}: Error finding container f9db2a0ec0891492eefa7cccc892e399d9115f941e63489637b6e72171ea0395: Status 404 returned error can't find the container with id f9db2a0ec0891492eefa7cccc892e399d9115f941e63489637b6e72171ea0395 Jul 12 18:22:37.520546 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:37.520515 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-92c98" event={"ID":"1a1a61fe-3f6b-48e2-9d20-71c149ceed75","Type":"ContainerStarted","Data":"c39d8cbe108e6b847d05a8c6b23745c1844376f9a1d461f541d8a89c38e009b7"} Jul 12 18:22:37.520970 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:37.520556 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-92c98" event={"ID":"1a1a61fe-3f6b-48e2-9d20-71c149ceed75","Type":"ContainerStarted","Data":"f9db2a0ec0891492eefa7cccc892e399d9115f941e63489637b6e72171ea0395"} Jul 12 18:22:37.540870 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:37.540822 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-92c98" podStartSLOduration=1.540807537 podStartE2EDuration="1.540807537s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:22:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:22:37.538739477 +0000 UTC m=+330.665644281" watchObservedRunningTime="2026-07-12 18:22:37.540807537 +0000 UTC m=+330.667712342" Jul 12 18:22:37.919515 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:37.919437 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wjrwc"] Jul 12 18:22:37.922927 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:37.922909 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wjrwc" Jul 12 18:22:37.925346 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:37.925324 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-4zr8v\"" Jul 12 18:22:37.925346 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:37.925341 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 12 18:22:37.926254 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:37.926240 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 12 18:22:37.930933 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:37.930911 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wjrwc"] Jul 12 18:22:38.015711 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:38.015676 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gq8kz\" (UniqueName: \"kubernetes.io/projected/ae75c702-3547-4324-86df-ada6e3b4b443-kube-api-access-gq8kz\") pod \"openshift-lws-operator-fd8ccff4c-wjrwc\" (UID: \"ae75c702-3547-4324-86df-ada6e3b4b443\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wjrwc" Jul 12 18:22:38.015894 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:38.015752 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ae75c702-3547-4324-86df-ada6e3b4b443-tmp\") pod \"openshift-lws-operator-fd8ccff4c-wjrwc\" (UID: \"ae75c702-3547-4324-86df-ada6e3b4b443\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wjrwc" Jul 12 18:22:38.116957 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:38.116923 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ae75c702-3547-4324-86df-ada6e3b4b443-tmp\") pod \"openshift-lws-operator-fd8ccff4c-wjrwc\" (UID: \"ae75c702-3547-4324-86df-ada6e3b4b443\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wjrwc" Jul 12 18:22:38.117130 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:38.116974 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gq8kz\" (UniqueName: \"kubernetes.io/projected/ae75c702-3547-4324-86df-ada6e3b4b443-kube-api-access-gq8kz\") pod \"openshift-lws-operator-fd8ccff4c-wjrwc\" (UID: \"ae75c702-3547-4324-86df-ada6e3b4b443\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wjrwc" Jul 12 18:22:38.117336 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:38.117312 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ae75c702-3547-4324-86df-ada6e3b4b443-tmp\") pod \"openshift-lws-operator-fd8ccff4c-wjrwc\" (UID: \"ae75c702-3547-4324-86df-ada6e3b4b443\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wjrwc" Jul 12 18:22:38.126365 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:38.126326 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gq8kz\" (UniqueName: \"kubernetes.io/projected/ae75c702-3547-4324-86df-ada6e3b4b443-kube-api-access-gq8kz\") pod \"openshift-lws-operator-fd8ccff4c-wjrwc\" (UID: \"ae75c702-3547-4324-86df-ada6e3b4b443\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wjrwc" Jul 12 18:22:38.232930 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:38.232897 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wjrwc" Jul 12 18:22:38.361330 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:38.361299 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wjrwc"] Jul 12 18:22:38.364262 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:22:38.364234 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podae75c702_3547_4324_86df_ada6e3b4b443.slice/crio-dc0e5374831957550b2ecc46c5f2e3e8b92b0112ef108fde5ba58e90ea1fa760 WatchSource:0}: Error finding container dc0e5374831957550b2ecc46c5f2e3e8b92b0112ef108fde5ba58e90ea1fa760: Status 404 returned error can't find the container with id dc0e5374831957550b2ecc46c5f2e3e8b92b0112ef108fde5ba58e90ea1fa760 Jul 12 18:22:38.526886 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:38.526791 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wjrwc" event={"ID":"ae75c702-3547-4324-86df-ada6e3b4b443","Type":"ContainerStarted","Data":"dc0e5374831957550b2ecc46c5f2e3e8b92b0112ef108fde5ba58e90ea1fa760"} Jul 12 18:22:40.536969 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:40.536935 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wjrwc" event={"ID":"ae75c702-3547-4324-86df-ada6e3b4b443","Type":"ContainerStarted","Data":"b016760257e46ec49f11e2f2a9956601a7d3059fea764dedeb5d0f7f2614e570"} Jul 12 18:22:40.553604 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:40.553557 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wjrwc" podStartSLOduration=1.558874403 podStartE2EDuration="3.553540661s" totalImagesPullingTime="1.994666258s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:22:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:22:40.552227545 +0000 UTC m=+333.679132388" watchObservedRunningTime="2026-07-12 18:22:40.553540661 +0000 UTC m=+333.680445466" Jul 12 18:22:43.210029 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:43.209993 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5dq8m"] Jul 12 18:22:43.212455 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:43.212434 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5dq8m" Jul 12 18:22:43.214961 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:43.214944 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 12 18:22:43.215045 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:43.215019 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-wg4d9\"" Jul 12 18:22:43.215097 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:43.215069 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 12 18:22:43.221041 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:43.221018 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5dq8m"] Jul 12 18:22:43.264401 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:43.264371 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/95c3481e-dece-4e71-be63-548359bd4289-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5dq8m\" (UID: \"95c3481e-dece-4e71-be63-548359bd4289\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5dq8m" Jul 12 18:22:43.264540 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:43.264421 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/95c3481e-dece-4e71-be63-548359bd4289-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5dq8m\" (UID: \"95c3481e-dece-4e71-be63-548359bd4289\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5dq8m" Jul 12 18:22:43.264540 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:43.264499 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8v22l\" (UniqueName: \"kubernetes.io/projected/95c3481e-dece-4e71-be63-548359bd4289-kube-api-access-8v22l\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5dq8m\" (UID: \"95c3481e-dece-4e71-be63-548359bd4289\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5dq8m" Jul 12 18:22:43.365538 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:43.365501 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8v22l\" (UniqueName: \"kubernetes.io/projected/95c3481e-dece-4e71-be63-548359bd4289-kube-api-access-8v22l\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5dq8m\" (UID: \"95c3481e-dece-4e71-be63-548359bd4289\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5dq8m" Jul 12 18:22:43.365722 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:43.365548 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/95c3481e-dece-4e71-be63-548359bd4289-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5dq8m\" (UID: \"95c3481e-dece-4e71-be63-548359bd4289\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5dq8m" Jul 12 18:22:43.365722 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:43.365602 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/95c3481e-dece-4e71-be63-548359bd4289-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5dq8m\" (UID: \"95c3481e-dece-4e71-be63-548359bd4289\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5dq8m" Jul 12 18:22:43.366004 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:43.365984 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/95c3481e-dece-4e71-be63-548359bd4289-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5dq8m\" (UID: \"95c3481e-dece-4e71-be63-548359bd4289\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5dq8m" Jul 12 18:22:43.366039 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:43.366023 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/95c3481e-dece-4e71-be63-548359bd4289-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5dq8m\" (UID: \"95c3481e-dece-4e71-be63-548359bd4289\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5dq8m" Jul 12 18:22:43.373385 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:43.373357 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8v22l\" (UniqueName: \"kubernetes.io/projected/95c3481e-dece-4e71-be63-548359bd4289-kube-api-access-8v22l\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5dq8m\" (UID: \"95c3481e-dece-4e71-be63-548359bd4289\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5dq8m" Jul 12 18:22:43.521609 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:43.521520 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5dq8m" Jul 12 18:22:43.649391 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:43.649365 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5dq8m"] Jul 12 18:22:43.651377 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:22:43.651351 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95c3481e_dece_4e71_be63_548359bd4289.slice/crio-84bb736f0050e756a6c90ec5e56a8cbeaf92684ad8207f6a9440b813ef95a02c WatchSource:0}: Error finding container 84bb736f0050e756a6c90ec5e56a8cbeaf92684ad8207f6a9440b813ef95a02c: Status 404 returned error can't find the container with id 84bb736f0050e756a6c90ec5e56a8cbeaf92684ad8207f6a9440b813ef95a02c Jul 12 18:22:44.559276 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:44.559240 2577 generic.go:356] "Generic (PLEG): container finished" podID="95c3481e-dece-4e71-be63-548359bd4289" containerID="1da326a93d5012adadfbfd9dc260266f97e93882ca33203ebe29d3d32b5c537c" exitCode=0 Jul 12 18:22:44.559776 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:44.559323 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5dq8m" event={"ID":"95c3481e-dece-4e71-be63-548359bd4289","Type":"ContainerDied","Data":"1da326a93d5012adadfbfd9dc260266f97e93882ca33203ebe29d3d32b5c537c"} Jul 12 18:22:44.559776 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:44.559373 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5dq8m" event={"ID":"95c3481e-dece-4e71-be63-548359bd4289","Type":"ContainerStarted","Data":"84bb736f0050e756a6c90ec5e56a8cbeaf92684ad8207f6a9440b813ef95a02c"} Jul 12 18:22:45.565343 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:45.565251 2577 generic.go:356] "Generic (PLEG): container finished" podID="95c3481e-dece-4e71-be63-548359bd4289" containerID="e75da36fbf1345336c925aa262eb30e9d9144b3f2cc9d0eddb3f40045a3f1473" exitCode=0 Jul 12 18:22:45.565781 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:45.565349 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5dq8m" event={"ID":"95c3481e-dece-4e71-be63-548359bd4289","Type":"ContainerDied","Data":"e75da36fbf1345336c925aa262eb30e9d9144b3f2cc9d0eddb3f40045a3f1473"} Jul 12 18:22:46.571963 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:46.571926 2577 generic.go:356] "Generic (PLEG): container finished" podID="95c3481e-dece-4e71-be63-548359bd4289" containerID="bbf5f8e0aa5e713944de128d1f398aebd6e2d019124160a92b6878c5d0938ed8" exitCode=0 Jul 12 18:22:46.572391 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:46.572003 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5dq8m" event={"ID":"95c3481e-dece-4e71-be63-548359bd4289","Type":"ContainerDied","Data":"bbf5f8e0aa5e713944de128d1f398aebd6e2d019124160a92b6878c5d0938ed8"} Jul 12 18:22:47.707922 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:47.707899 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5dq8m" Jul 12 18:22:47.802939 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:47.802906 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/95c3481e-dece-4e71-be63-548359bd4289-bundle\") pod \"95c3481e-dece-4e71-be63-548359bd4289\" (UID: \"95c3481e-dece-4e71-be63-548359bd4289\") " Jul 12 18:22:47.802939 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:47.802947 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/95c3481e-dece-4e71-be63-548359bd4289-util\") pod \"95c3481e-dece-4e71-be63-548359bd4289\" (UID: \"95c3481e-dece-4e71-be63-548359bd4289\") " Jul 12 18:22:47.803190 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:47.802989 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8v22l\" (UniqueName: \"kubernetes.io/projected/95c3481e-dece-4e71-be63-548359bd4289-kube-api-access-8v22l\") pod \"95c3481e-dece-4e71-be63-548359bd4289\" (UID: \"95c3481e-dece-4e71-be63-548359bd4289\") " Jul 12 18:22:47.803774 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:47.803738 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95c3481e-dece-4e71-be63-548359bd4289-bundle" (OuterVolumeSpecName: "bundle") pod "95c3481e-dece-4e71-be63-548359bd4289" (UID: "95c3481e-dece-4e71-be63-548359bd4289"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:22:47.805079 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:47.805051 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95c3481e-dece-4e71-be63-548359bd4289-kube-api-access-8v22l" (OuterVolumeSpecName: "kube-api-access-8v22l") pod "95c3481e-dece-4e71-be63-548359bd4289" (UID: "95c3481e-dece-4e71-be63-548359bd4289"). InnerVolumeSpecName "kube-api-access-8v22l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:22:47.808894 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:47.808860 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95c3481e-dece-4e71-be63-548359bd4289-util" (OuterVolumeSpecName: "util") pod "95c3481e-dece-4e71-be63-548359bd4289" (UID: "95c3481e-dece-4e71-be63-548359bd4289"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:22:47.904563 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:47.904479 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8v22l\" (UniqueName: \"kubernetes.io/projected/95c3481e-dece-4e71-be63-548359bd4289-kube-api-access-8v22l\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:22:47.904563 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:47.904505 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/95c3481e-dece-4e71-be63-548359bd4289-bundle\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:22:47.904563 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:47.904515 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/95c3481e-dece-4e71-be63-548359bd4289-util\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:22:48.582208 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:48.582129 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5dq8m" Jul 12 18:22:48.582208 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:48.582141 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af5dq8m" event={"ID":"95c3481e-dece-4e71-be63-548359bd4289","Type":"ContainerDied","Data":"84bb736f0050e756a6c90ec5e56a8cbeaf92684ad8207f6a9440b813ef95a02c"} Jul 12 18:22:48.582208 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:22:48.582188 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="84bb736f0050e756a6c90ec5e56a8cbeaf92684ad8207f6a9440b813ef95a02c" Jul 12 18:23:00.705471 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:00.705438 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-db78fd4c7-bq5pl"] Jul 12 18:23:00.705981 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:00.705755 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95c3481e-dece-4e71-be63-548359bd4289" containerName="pull" Jul 12 18:23:00.705981 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:00.705766 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="95c3481e-dece-4e71-be63-548359bd4289" containerName="pull" Jul 12 18:23:00.705981 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:00.705773 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95c3481e-dece-4e71-be63-548359bd4289" containerName="extract" Jul 12 18:23:00.705981 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:00.705779 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="95c3481e-dece-4e71-be63-548359bd4289" containerName="extract" Jul 12 18:23:00.705981 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:00.705799 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95c3481e-dece-4e71-be63-548359bd4289" containerName="util" Jul 12 18:23:00.705981 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:00.705804 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="95c3481e-dece-4e71-be63-548359bd4289" containerName="util" Jul 12 18:23:00.705981 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:00.705851 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="95c3481e-dece-4e71-be63-548359bd4289" containerName="extract" Jul 12 18:23:00.708056 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:00.708040 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-db78fd4c7-bq5pl" Jul 12 18:23:00.712963 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:00.712932 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 12 18:23:00.713245 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:00.713226 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-l2kpp\"" Jul 12 18:23:00.715694 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:00.715665 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 12 18:23:00.715920 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:00.715897 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 12 18:23:00.716764 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:00.716509 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 12 18:23:00.732293 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:00.732270 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-db78fd4c7-bq5pl"] Jul 12 18:23:00.803191 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:00.803142 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1ac29973-5a2e-4f0a-854e-e85369b8157a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-db78fd4c7-bq5pl\" (UID: \"1ac29973-5a2e-4f0a-854e-e85369b8157a\") " pod="opendatahub/opendatahub-operator-controller-manager-db78fd4c7-bq5pl" Jul 12 18:23:00.803350 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:00.803245 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1ac29973-5a2e-4f0a-854e-e85369b8157a-webhook-cert\") pod \"opendatahub-operator-controller-manager-db78fd4c7-bq5pl\" (UID: \"1ac29973-5a2e-4f0a-854e-e85369b8157a\") " pod="opendatahub/opendatahub-operator-controller-manager-db78fd4c7-bq5pl" Jul 12 18:23:00.803350 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:00.803306 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1ac29973-5a2e-4f0a-854e-e85369b8157a-tmp\") pod \"opendatahub-operator-controller-manager-db78fd4c7-bq5pl\" (UID: \"1ac29973-5a2e-4f0a-854e-e85369b8157a\") " pod="opendatahub/opendatahub-operator-controller-manager-db78fd4c7-bq5pl" Jul 12 18:23:00.803440 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:00.803370 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1ac29973-5a2e-4f0a-854e-e85369b8157a-manifests\") pod \"opendatahub-operator-controller-manager-db78fd4c7-bq5pl\" (UID: \"1ac29973-5a2e-4f0a-854e-e85369b8157a\") " pod="opendatahub/opendatahub-operator-controller-manager-db78fd4c7-bq5pl" Jul 12 18:23:00.803440 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:00.803403 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hctz5\" (UniqueName: \"kubernetes.io/projected/1ac29973-5a2e-4f0a-854e-e85369b8157a-kube-api-access-hctz5\") pod \"opendatahub-operator-controller-manager-db78fd4c7-bq5pl\" (UID: \"1ac29973-5a2e-4f0a-854e-e85369b8157a\") " pod="opendatahub/opendatahub-operator-controller-manager-db78fd4c7-bq5pl" Jul 12 18:23:00.904187 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:00.904144 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1ac29973-5a2e-4f0a-854e-e85369b8157a-manifests\") pod \"opendatahub-operator-controller-manager-db78fd4c7-bq5pl\" (UID: \"1ac29973-5a2e-4f0a-854e-e85369b8157a\") " pod="opendatahub/opendatahub-operator-controller-manager-db78fd4c7-bq5pl" Jul 12 18:23:00.904334 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:00.904198 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hctz5\" (UniqueName: \"kubernetes.io/projected/1ac29973-5a2e-4f0a-854e-e85369b8157a-kube-api-access-hctz5\") pod \"opendatahub-operator-controller-manager-db78fd4c7-bq5pl\" (UID: \"1ac29973-5a2e-4f0a-854e-e85369b8157a\") " pod="opendatahub/opendatahub-operator-controller-manager-db78fd4c7-bq5pl" Jul 12 18:23:00.904334 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:00.904282 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1ac29973-5a2e-4f0a-854e-e85369b8157a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-db78fd4c7-bq5pl\" (UID: \"1ac29973-5a2e-4f0a-854e-e85369b8157a\") " pod="opendatahub/opendatahub-operator-controller-manager-db78fd4c7-bq5pl" Jul 12 18:23:00.904334 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:00.904328 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1ac29973-5a2e-4f0a-854e-e85369b8157a-webhook-cert\") pod \"opendatahub-operator-controller-manager-db78fd4c7-bq5pl\" (UID: \"1ac29973-5a2e-4f0a-854e-e85369b8157a\") " pod="opendatahub/opendatahub-operator-controller-manager-db78fd4c7-bq5pl" Jul 12 18:23:00.904510 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:00.904365 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1ac29973-5a2e-4f0a-854e-e85369b8157a-tmp\") pod \"opendatahub-operator-controller-manager-db78fd4c7-bq5pl\" (UID: \"1ac29973-5a2e-4f0a-854e-e85369b8157a\") " pod="opendatahub/opendatahub-operator-controller-manager-db78fd4c7-bq5pl" Jul 12 18:23:00.904605 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:00.904582 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1ac29973-5a2e-4f0a-854e-e85369b8157a-manifests\") pod \"opendatahub-operator-controller-manager-db78fd4c7-bq5pl\" (UID: \"1ac29973-5a2e-4f0a-854e-e85369b8157a\") " pod="opendatahub/opendatahub-operator-controller-manager-db78fd4c7-bq5pl" Jul 12 18:23:00.906600 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:00.906565 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1ac29973-5a2e-4f0a-854e-e85369b8157a-tmp\") pod \"opendatahub-operator-controller-manager-db78fd4c7-bq5pl\" (UID: \"1ac29973-5a2e-4f0a-854e-e85369b8157a\") " pod="opendatahub/opendatahub-operator-controller-manager-db78fd4c7-bq5pl" Jul 12 18:23:00.906785 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:00.906767 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1ac29973-5a2e-4f0a-854e-e85369b8157a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-db78fd4c7-bq5pl\" (UID: \"1ac29973-5a2e-4f0a-854e-e85369b8157a\") " pod="opendatahub/opendatahub-operator-controller-manager-db78fd4c7-bq5pl" Jul 12 18:23:00.907022 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:00.906994 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1ac29973-5a2e-4f0a-854e-e85369b8157a-webhook-cert\") pod \"opendatahub-operator-controller-manager-db78fd4c7-bq5pl\" (UID: \"1ac29973-5a2e-4f0a-854e-e85369b8157a\") " pod="opendatahub/opendatahub-operator-controller-manager-db78fd4c7-bq5pl" Jul 12 18:23:00.912315 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:00.912285 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hctz5\" (UniqueName: \"kubernetes.io/projected/1ac29973-5a2e-4f0a-854e-e85369b8157a-kube-api-access-hctz5\") pod \"opendatahub-operator-controller-manager-db78fd4c7-bq5pl\" (UID: \"1ac29973-5a2e-4f0a-854e-e85369b8157a\") " pod="opendatahub/opendatahub-operator-controller-manager-db78fd4c7-bq5pl" Jul 12 18:23:01.021752 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:01.021661 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-db78fd4c7-bq5pl" Jul 12 18:23:01.165768 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:01.165742 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-db78fd4c7-bq5pl"] Jul 12 18:23:01.167684 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:23:01.167659 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ac29973_5a2e_4f0a_854e_e85369b8157a.slice/crio-1e094c321f65fc0cd5c15e38cfad3bf138c0aa9e44a4140584f9efa23b027882 WatchSource:0}: Error finding container 1e094c321f65fc0cd5c15e38cfad3bf138c0aa9e44a4140584f9efa23b027882: Status 404 returned error can't find the container with id 1e094c321f65fc0cd5c15e38cfad3bf138c0aa9e44a4140584f9efa23b027882 Jul 12 18:23:01.649182 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:01.649138 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-db78fd4c7-bq5pl" event={"ID":"1ac29973-5a2e-4f0a-854e-e85369b8157a","Type":"ContainerStarted","Data":"1e094c321f65fc0cd5c15e38cfad3bf138c0aa9e44a4140584f9efa23b027882"} Jul 12 18:23:04.668064 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:04.668017 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-db78fd4c7-bq5pl" event={"ID":"1ac29973-5a2e-4f0a-854e-e85369b8157a","Type":"ContainerStarted","Data":"1c19d065f6a39d2bb0a370e2ae2529b3373f5c89179804840281e2119769bd67"} Jul 12 18:23:05.674255 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:05.674171 2577 generic.go:356] "Generic (PLEG): container finished" podID="1ac29973-5a2e-4f0a-854e-e85369b8157a" containerID="1c19d065f6a39d2bb0a370e2ae2529b3373f5c89179804840281e2119769bd67" exitCode=0 Jul 12 18:23:05.674255 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:05.674216 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-db78fd4c7-bq5pl" event={"ID":"1ac29973-5a2e-4f0a-854e-e85369b8157a","Type":"ContainerDied","Data":"1c19d065f6a39d2bb0a370e2ae2529b3373f5c89179804840281e2119769bd67"} Jul 12 18:23:06.683128 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:06.683095 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-db78fd4c7-bq5pl" event={"ID":"1ac29973-5a2e-4f0a-854e-e85369b8157a","Type":"ContainerStarted","Data":"fb11985cb593d688c26fb5cccddc95c4a37197fb52dc7e7ae2090d68e6e7901e"} Jul 12 18:23:06.683517 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:06.683190 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-db78fd4c7-bq5pl" Jul 12 18:23:06.708443 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:06.708391 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-db78fd4c7-bq5pl" podStartSLOduration=2.251180584 podStartE2EDuration="6.708375448s" totalImagesPullingTime="3.405008726s" totalInitContainerRuntime="1.052186138s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:23:00 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:23:06.706042307 +0000 UTC m=+359.832947112" watchObservedRunningTime="2026-07-12 18:23:06.708375448 +0000 UTC m=+359.835280252" Jul 12 18:23:17.690766 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:17.690732 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-db78fd4c7-bq5pl" Jul 12 18:23:20.316641 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:20.316586 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-g4dgx"] Jul 12 18:23:20.326196 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:20.326173 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-g4dgx" Jul 12 18:23:20.328835 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:20.328813 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 12 18:23:20.328995 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:20.328972 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-zh74c\"" Jul 12 18:23:20.332857 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:20.332829 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-g4dgx"] Jul 12 18:23:20.468675 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:20.468610 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c035e80f-4c27-48d9-bb9a-59dcb0218fd9-cert\") pod \"odh-model-controller-7998c67dc4-g4dgx\" (UID: \"c035e80f-4c27-48d9-bb9a-59dcb0218fd9\") " pod="opendatahub/odh-model-controller-7998c67dc4-g4dgx" Jul 12 18:23:20.468893 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:20.468729 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4ll5\" (UniqueName: \"kubernetes.io/projected/c035e80f-4c27-48d9-bb9a-59dcb0218fd9-kube-api-access-s4ll5\") pod \"odh-model-controller-7998c67dc4-g4dgx\" (UID: \"c035e80f-4c27-48d9-bb9a-59dcb0218fd9\") " pod="opendatahub/odh-model-controller-7998c67dc4-g4dgx" Jul 12 18:23:20.570164 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:20.570065 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c035e80f-4c27-48d9-bb9a-59dcb0218fd9-cert\") pod \"odh-model-controller-7998c67dc4-g4dgx\" (UID: \"c035e80f-4c27-48d9-bb9a-59dcb0218fd9\") " pod="opendatahub/odh-model-controller-7998c67dc4-g4dgx" Jul 12 18:23:20.570164 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:20.570136 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4ll5\" (UniqueName: \"kubernetes.io/projected/c035e80f-4c27-48d9-bb9a-59dcb0218fd9-kube-api-access-s4ll5\") pod \"odh-model-controller-7998c67dc4-g4dgx\" (UID: \"c035e80f-4c27-48d9-bb9a-59dcb0218fd9\") " pod="opendatahub/odh-model-controller-7998c67dc4-g4dgx" Jul 12 18:23:20.572668 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:20.572617 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c035e80f-4c27-48d9-bb9a-59dcb0218fd9-cert\") pod \"odh-model-controller-7998c67dc4-g4dgx\" (UID: \"c035e80f-4c27-48d9-bb9a-59dcb0218fd9\") " pod="opendatahub/odh-model-controller-7998c67dc4-g4dgx" Jul 12 18:23:20.581041 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:20.581012 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4ll5\" (UniqueName: \"kubernetes.io/projected/c035e80f-4c27-48d9-bb9a-59dcb0218fd9-kube-api-access-s4ll5\") pod \"odh-model-controller-7998c67dc4-g4dgx\" (UID: \"c035e80f-4c27-48d9-bb9a-59dcb0218fd9\") " pod="opendatahub/odh-model-controller-7998c67dc4-g4dgx" Jul 12 18:23:20.637384 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:20.637349 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-g4dgx" Jul 12 18:23:20.774861 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:20.774831 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-g4dgx"] Jul 12 18:23:20.777547 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:23:20.777512 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc035e80f_4c27_48d9_bb9a_59dcb0218fd9.slice/crio-6885e66de1d078b31350a3c4108a558b2197528eafb82559037f0d5de655047d WatchSource:0}: Error finding container 6885e66de1d078b31350a3c4108a558b2197528eafb82559037f0d5de655047d: Status 404 returned error can't find the container with id 6885e66de1d078b31350a3c4108a558b2197528eafb82559037f0d5de655047d Jul 12 18:23:21.755832 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:21.755785 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-g4dgx" event={"ID":"c035e80f-4c27-48d9-bb9a-59dcb0218fd9","Type":"ContainerStarted","Data":"6885e66de1d078b31350a3c4108a558b2197528eafb82559037f0d5de655047d"} Jul 12 18:23:23.768567 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:23.768523 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-g4dgx" event={"ID":"c035e80f-4c27-48d9-bb9a-59dcb0218fd9","Type":"ContainerStarted","Data":"465f1abc3d3d2db5484d201d4e87eea9ce7f76accb961d679339dce8116ce4d6"} Jul 12 18:23:23.769029 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:23.768721 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-g4dgx" Jul 12 18:23:23.785216 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:23.785107 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-g4dgx" podStartSLOduration=1.03905182 podStartE2EDuration="3.785082915s" totalImagesPullingTime="2.746031095s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:23:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:23:23.783965554 +0000 UTC m=+376.910870360" watchObservedRunningTime="2026-07-12 18:23:23.785082915 +0000 UTC m=+376.911987721" Jul 12 18:23:26.473382 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:26.473350 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678qjql2"] Jul 12 18:23:26.476916 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:26.476895 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678qjql2" Jul 12 18:23:26.479710 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:26.479611 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 12 18:23:26.479953 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:26.479732 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-fm2nb\"" Jul 12 18:23:26.479953 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:26.479886 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 12 18:23:26.480306 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:26.480286 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 12 18:23:26.490378 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:26.490353 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678qjql2"] Jul 12 18:23:26.621881 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:26.621834 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05e3c37f-45b4-47dc-9c35-cc81ee6dd97a-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678qjql2\" (UID: \"05e3c37f-45b4-47dc-9c35-cc81ee6dd97a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678qjql2" Jul 12 18:23:26.622067 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:26.621901 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcb45\" (UniqueName: \"kubernetes.io/projected/05e3c37f-45b4-47dc-9c35-cc81ee6dd97a-kube-api-access-zcb45\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678qjql2\" (UID: \"05e3c37f-45b4-47dc-9c35-cc81ee6dd97a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678qjql2" Jul 12 18:23:26.622067 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:26.621945 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05e3c37f-45b4-47dc-9c35-cc81ee6dd97a-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678qjql2\" (UID: \"05e3c37f-45b4-47dc-9c35-cc81ee6dd97a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678qjql2" Jul 12 18:23:26.622067 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:26.621971 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05e3c37f-45b4-47dc-9c35-cc81ee6dd97a-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678qjql2\" (UID: \"05e3c37f-45b4-47dc-9c35-cc81ee6dd97a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678qjql2" Jul 12 18:23:26.622067 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:26.622048 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05e3c37f-45b4-47dc-9c35-cc81ee6dd97a-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678qjql2\" (UID: \"05e3c37f-45b4-47dc-9c35-cc81ee6dd97a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678qjql2" Jul 12 18:23:26.622357 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:26.622110 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05e3c37f-45b4-47dc-9c35-cc81ee6dd97a-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678qjql2\" (UID: \"05e3c37f-45b4-47dc-9c35-cc81ee6dd97a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678qjql2" Jul 12 18:23:26.622357 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:26.622144 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05e3c37f-45b4-47dc-9c35-cc81ee6dd97a-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678qjql2\" (UID: \"05e3c37f-45b4-47dc-9c35-cc81ee6dd97a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678qjql2" Jul 12 18:23:26.622357 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:26.622186 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05e3c37f-45b4-47dc-9c35-cc81ee6dd97a-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678qjql2\" (UID: \"05e3c37f-45b4-47dc-9c35-cc81ee6dd97a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678qjql2" Jul 12 18:23:26.622357 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:26.622217 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05e3c37f-45b4-47dc-9c35-cc81ee6dd97a-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678qjql2\" (UID: \"05e3c37f-45b4-47dc-9c35-cc81ee6dd97a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678qjql2" Jul 12 18:23:26.723216 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:26.723175 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05e3c37f-45b4-47dc-9c35-cc81ee6dd97a-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678qjql2\" (UID: \"05e3c37f-45b4-47dc-9c35-cc81ee6dd97a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678qjql2" Jul 12 18:23:26.723216 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:26.723222 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05e3c37f-45b4-47dc-9c35-cc81ee6dd97a-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678qjql2\" (UID: \"05e3c37f-45b4-47dc-9c35-cc81ee6dd97a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678qjql2" Jul 12 18:23:26.723481 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:26.723342 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05e3c37f-45b4-47dc-9c35-cc81ee6dd97a-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678qjql2\" (UID: \"05e3c37f-45b4-47dc-9c35-cc81ee6dd97a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678qjql2" Jul 12 18:23:26.723481 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:26.723413 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcb45\" (UniqueName: \"kubernetes.io/projected/05e3c37f-45b4-47dc-9c35-cc81ee6dd97a-kube-api-access-zcb45\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678qjql2\" (UID: \"05e3c37f-45b4-47dc-9c35-cc81ee6dd97a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678qjql2" Jul 12 18:23:26.723481 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:26.723460 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05e3c37f-45b4-47dc-9c35-cc81ee6dd97a-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678qjql2\" (UID: \"05e3c37f-45b4-47dc-9c35-cc81ee6dd97a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678qjql2" Jul 12 18:23:26.723686 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:26.723483 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05e3c37f-45b4-47dc-9c35-cc81ee6dd97a-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678qjql2\" (UID: \"05e3c37f-45b4-47dc-9c35-cc81ee6dd97a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678qjql2" Jul 12 18:23:26.723686 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:26.723510 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05e3c37f-45b4-47dc-9c35-cc81ee6dd97a-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678qjql2\" (UID: \"05e3c37f-45b4-47dc-9c35-cc81ee6dd97a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678qjql2" Jul 12 18:23:26.723686 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:26.723554 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05e3c37f-45b4-47dc-9c35-cc81ee6dd97a-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678qjql2\" (UID: \"05e3c37f-45b4-47dc-9c35-cc81ee6dd97a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678qjql2" Jul 12 18:23:26.723686 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:26.723586 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05e3c37f-45b4-47dc-9c35-cc81ee6dd97a-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678qjql2\" (UID: \"05e3c37f-45b4-47dc-9c35-cc81ee6dd97a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678qjql2" Jul 12 18:23:26.723908 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:26.723833 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05e3c37f-45b4-47dc-9c35-cc81ee6dd97a-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678qjql2\" (UID: \"05e3c37f-45b4-47dc-9c35-cc81ee6dd97a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678qjql2" Jul 12 18:23:26.723980 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:26.723955 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05e3c37f-45b4-47dc-9c35-cc81ee6dd97a-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678qjql2\" (UID: \"05e3c37f-45b4-47dc-9c35-cc81ee6dd97a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678qjql2" Jul 12 18:23:26.724040 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:26.724003 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05e3c37f-45b4-47dc-9c35-cc81ee6dd97a-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678qjql2\" (UID: \"05e3c37f-45b4-47dc-9c35-cc81ee6dd97a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678qjql2" Jul 12 18:23:26.724040 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:26.724003 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05e3c37f-45b4-47dc-9c35-cc81ee6dd97a-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678qjql2\" (UID: \"05e3c37f-45b4-47dc-9c35-cc81ee6dd97a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678qjql2" Jul 12 18:23:26.724140 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:26.724072 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05e3c37f-45b4-47dc-9c35-cc81ee6dd97a-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678qjql2\" (UID: \"05e3c37f-45b4-47dc-9c35-cc81ee6dd97a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678qjql2" Jul 12 18:23:26.725777 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:26.725750 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05e3c37f-45b4-47dc-9c35-cc81ee6dd97a-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678qjql2\" (UID: \"05e3c37f-45b4-47dc-9c35-cc81ee6dd97a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678qjql2" Jul 12 18:23:26.725881 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:26.725829 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05e3c37f-45b4-47dc-9c35-cc81ee6dd97a-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678qjql2\" (UID: \"05e3c37f-45b4-47dc-9c35-cc81ee6dd97a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678qjql2" Jul 12 18:23:26.732414 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:26.732385 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05e3c37f-45b4-47dc-9c35-cc81ee6dd97a-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678qjql2\" (UID: \"05e3c37f-45b4-47dc-9c35-cc81ee6dd97a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678qjql2" Jul 12 18:23:26.733216 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:26.733193 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcb45\" (UniqueName: \"kubernetes.io/projected/05e3c37f-45b4-47dc-9c35-cc81ee6dd97a-kube-api-access-zcb45\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678qjql2\" (UID: \"05e3c37f-45b4-47dc-9c35-cc81ee6dd97a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678qjql2" Jul 12 18:23:26.778653 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:26.778595 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-9757cf8df-rcmmq"] Jul 12 18:23:26.783173 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:26.783152 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-9757cf8df-rcmmq" Jul 12 18:23:26.785734 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:26.785702 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 12 18:23:26.785734 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:26.785722 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 12 18:23:26.785899 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:26.785782 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 12 18:23:26.786018 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:26.786001 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-b44qt\"" Jul 12 18:23:26.789513 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:26.789486 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678qjql2" Jul 12 18:23:26.790284 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:26.790262 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-9757cf8df-rcmmq"] Jul 12 18:23:26.926234 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:26.926206 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/19d37878-abe2-47a1-9194-6400f779fe95-metrics-cert\") pod \"lws-controller-manager-9757cf8df-rcmmq\" (UID: \"19d37878-abe2-47a1-9194-6400f779fe95\") " pod="openshift-lws-operator/lws-controller-manager-9757cf8df-rcmmq" Jul 12 18:23:26.926234 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:26.926246 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/19d37878-abe2-47a1-9194-6400f779fe95-cert\") pod \"lws-controller-manager-9757cf8df-rcmmq\" (UID: \"19d37878-abe2-47a1-9194-6400f779fe95\") " pod="openshift-lws-operator/lws-controller-manager-9757cf8df-rcmmq" Jul 12 18:23:26.926427 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:26.926380 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/19d37878-abe2-47a1-9194-6400f779fe95-manager-config\") pod \"lws-controller-manager-9757cf8df-rcmmq\" (UID: \"19d37878-abe2-47a1-9194-6400f779fe95\") " pod="openshift-lws-operator/lws-controller-manager-9757cf8df-rcmmq" Jul 12 18:23:26.926427 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:26.926415 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ksqz\" (UniqueName: \"kubernetes.io/projected/19d37878-abe2-47a1-9194-6400f779fe95-kube-api-access-7ksqz\") pod \"lws-controller-manager-9757cf8df-rcmmq\" (UID: \"19d37878-abe2-47a1-9194-6400f779fe95\") " pod="openshift-lws-operator/lws-controller-manager-9757cf8df-rcmmq" Jul 12 18:23:26.930056 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:26.930033 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678qjql2"] Jul 12 18:23:26.931883 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:23:26.931850 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05e3c37f_45b4_47dc_9c35_cc81ee6dd97a.slice/crio-ca562bedbc8efc3d77da53e055f5cb1754fbf67f3c721e0efa5adf7edf28e999 WatchSource:0}: Error finding container ca562bedbc8efc3d77da53e055f5cb1754fbf67f3c721e0efa5adf7edf28e999: Status 404 returned error can't find the container with id ca562bedbc8efc3d77da53e055f5cb1754fbf67f3c721e0efa5adf7edf28e999 Jul 12 18:23:27.027690 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:27.027593 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/19d37878-abe2-47a1-9194-6400f779fe95-manager-config\") pod \"lws-controller-manager-9757cf8df-rcmmq\" (UID: \"19d37878-abe2-47a1-9194-6400f779fe95\") " pod="openshift-lws-operator/lws-controller-manager-9757cf8df-rcmmq" Jul 12 18:23:27.027690 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:27.027652 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ksqz\" (UniqueName: \"kubernetes.io/projected/19d37878-abe2-47a1-9194-6400f779fe95-kube-api-access-7ksqz\") pod \"lws-controller-manager-9757cf8df-rcmmq\" (UID: \"19d37878-abe2-47a1-9194-6400f779fe95\") " pod="openshift-lws-operator/lws-controller-manager-9757cf8df-rcmmq" Jul 12 18:23:27.027880 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:27.027726 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/19d37878-abe2-47a1-9194-6400f779fe95-metrics-cert\") pod \"lws-controller-manager-9757cf8df-rcmmq\" (UID: \"19d37878-abe2-47a1-9194-6400f779fe95\") " pod="openshift-lws-operator/lws-controller-manager-9757cf8df-rcmmq" Jul 12 18:23:27.027880 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:27.027781 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/19d37878-abe2-47a1-9194-6400f779fe95-cert\") pod \"lws-controller-manager-9757cf8df-rcmmq\" (UID: \"19d37878-abe2-47a1-9194-6400f779fe95\") " pod="openshift-lws-operator/lws-controller-manager-9757cf8df-rcmmq" Jul 12 18:23:27.028352 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:27.028332 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/19d37878-abe2-47a1-9194-6400f779fe95-manager-config\") pod \"lws-controller-manager-9757cf8df-rcmmq\" (UID: \"19d37878-abe2-47a1-9194-6400f779fe95\") " pod="openshift-lws-operator/lws-controller-manager-9757cf8df-rcmmq" Jul 12 18:23:27.030164 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:27.030146 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/19d37878-abe2-47a1-9194-6400f779fe95-cert\") pod \"lws-controller-manager-9757cf8df-rcmmq\" (UID: \"19d37878-abe2-47a1-9194-6400f779fe95\") " pod="openshift-lws-operator/lws-controller-manager-9757cf8df-rcmmq" Jul 12 18:23:27.030255 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:27.030153 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/19d37878-abe2-47a1-9194-6400f779fe95-metrics-cert\") pod \"lws-controller-manager-9757cf8df-rcmmq\" (UID: \"19d37878-abe2-47a1-9194-6400f779fe95\") " pod="openshift-lws-operator/lws-controller-manager-9757cf8df-rcmmq" Jul 12 18:23:27.036938 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:27.036917 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ksqz\" (UniqueName: \"kubernetes.io/projected/19d37878-abe2-47a1-9194-6400f779fe95-kube-api-access-7ksqz\") pod \"lws-controller-manager-9757cf8df-rcmmq\" (UID: \"19d37878-abe2-47a1-9194-6400f779fe95\") " pod="openshift-lws-operator/lws-controller-manager-9757cf8df-rcmmq" Jul 12 18:23:27.093779 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:27.093737 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-9757cf8df-rcmmq" Jul 12 18:23:27.230155 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:27.230128 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-9757cf8df-rcmmq"] Jul 12 18:23:27.232805 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:23:27.232771 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19d37878_abe2_47a1_9194_6400f779fe95.slice/crio-2851ca8d526ab6064d56264b90f2666fa29822914066ba056476161fc2809b8a WatchSource:0}: Error finding container 2851ca8d526ab6064d56264b90f2666fa29822914066ba056476161fc2809b8a: Status 404 returned error can't find the container with id 2851ca8d526ab6064d56264b90f2666fa29822914066ba056476161fc2809b8a Jul 12 18:23:27.793468 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:27.793398 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678qjql2" event={"ID":"05e3c37f-45b4-47dc-9c35-cc81ee6dd97a","Type":"ContainerStarted","Data":"ca562bedbc8efc3d77da53e055f5cb1754fbf67f3c721e0efa5adf7edf28e999"} Jul 12 18:23:27.795022 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:27.794984 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-9757cf8df-rcmmq" event={"ID":"19d37878-abe2-47a1-9194-6400f779fe95","Type":"ContainerStarted","Data":"2851ca8d526ab6064d56264b90f2666fa29822914066ba056476161fc2809b8a"} Jul 12 18:23:28.276581 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:28.276541 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-dvc6v"] Jul 12 18:23:28.280390 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:28.280367 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-dvc6v" Jul 12 18:23:28.283485 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:28.283454 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 12 18:23:28.283485 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:28.283467 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 12 18:23:28.283707 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:28.283519 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 12 18:23:28.283707 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:28.283489 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-p96xq\"" Jul 12 18:23:28.304705 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:28.304665 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-dvc6v"] Jul 12 18:23:28.441124 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:28.440872 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e4faeb51-6114-46cc-a7df-ff844503afac-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-dvc6v\" (UID: \"e4faeb51-6114-46cc-a7df-ff844503afac\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-dvc6v" Jul 12 18:23:28.441124 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:28.440919 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e4faeb51-6114-46cc-a7df-ff844503afac-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-dvc6v\" (UID: \"e4faeb51-6114-46cc-a7df-ff844503afac\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-dvc6v" Jul 12 18:23:28.441124 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:28.440940 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4faeb51-6114-46cc-a7df-ff844503afac-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-dvc6v\" (UID: \"e4faeb51-6114-46cc-a7df-ff844503afac\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-dvc6v" Jul 12 18:23:28.441124 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:28.441048 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e4faeb51-6114-46cc-a7df-ff844503afac-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-dvc6v\" (UID: \"e4faeb51-6114-46cc-a7df-ff844503afac\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-dvc6v" Jul 12 18:23:28.441124 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:28.441104 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e4faeb51-6114-46cc-a7df-ff844503afac-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-dvc6v\" (UID: \"e4faeb51-6114-46cc-a7df-ff844503afac\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-dvc6v" Jul 12 18:23:28.441124 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:28.441129 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfqld\" (UniqueName: \"kubernetes.io/projected/e4faeb51-6114-46cc-a7df-ff844503afac-kube-api-access-kfqld\") pod \"istiod-openshift-gateway-94bb8fbfd-dvc6v\" (UID: \"e4faeb51-6114-46cc-a7df-ff844503afac\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-dvc6v" Jul 12 18:23:28.441567 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:28.441160 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4faeb51-6114-46cc-a7df-ff844503afac-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-dvc6v\" (UID: \"e4faeb51-6114-46cc-a7df-ff844503afac\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-dvc6v" Jul 12 18:23:28.542938 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:28.542021 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e4faeb51-6114-46cc-a7df-ff844503afac-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-dvc6v\" (UID: \"e4faeb51-6114-46cc-a7df-ff844503afac\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-dvc6v" Jul 12 18:23:28.542938 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:28.542094 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e4faeb51-6114-46cc-a7df-ff844503afac-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-dvc6v\" (UID: \"e4faeb51-6114-46cc-a7df-ff844503afac\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-dvc6v" Jul 12 18:23:28.542938 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:28.542121 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfqld\" (UniqueName: \"kubernetes.io/projected/e4faeb51-6114-46cc-a7df-ff844503afac-kube-api-access-kfqld\") pod \"istiod-openshift-gateway-94bb8fbfd-dvc6v\" (UID: \"e4faeb51-6114-46cc-a7df-ff844503afac\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-dvc6v" Jul 12 18:23:28.542938 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:28.542153 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4faeb51-6114-46cc-a7df-ff844503afac-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-dvc6v\" (UID: \"e4faeb51-6114-46cc-a7df-ff844503afac\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-dvc6v" Jul 12 18:23:28.542938 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:28.542271 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e4faeb51-6114-46cc-a7df-ff844503afac-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-dvc6v\" (UID: \"e4faeb51-6114-46cc-a7df-ff844503afac\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-dvc6v" Jul 12 18:23:28.542938 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:28.542306 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e4faeb51-6114-46cc-a7df-ff844503afac-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-dvc6v\" (UID: \"e4faeb51-6114-46cc-a7df-ff844503afac\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-dvc6v" Jul 12 18:23:28.542938 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:28.542331 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4faeb51-6114-46cc-a7df-ff844503afac-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-dvc6v\" (UID: \"e4faeb51-6114-46cc-a7df-ff844503afac\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-dvc6v" Jul 12 18:23:28.543537 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:28.543454 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e4faeb51-6114-46cc-a7df-ff844503afac-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-dvc6v\" (UID: \"e4faeb51-6114-46cc-a7df-ff844503afac\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-dvc6v" Jul 12 18:23:28.545612 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:28.545564 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e4faeb51-6114-46cc-a7df-ff844503afac-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-dvc6v\" (UID: \"e4faeb51-6114-46cc-a7df-ff844503afac\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-dvc6v" Jul 12 18:23:28.545978 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:28.545932 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4faeb51-6114-46cc-a7df-ff844503afac-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-dvc6v\" (UID: \"e4faeb51-6114-46cc-a7df-ff844503afac\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-dvc6v" Jul 12 18:23:28.546865 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:28.546843 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e4faeb51-6114-46cc-a7df-ff844503afac-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-dvc6v\" (UID: \"e4faeb51-6114-46cc-a7df-ff844503afac\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-dvc6v" Jul 12 18:23:28.552102 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:28.552061 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e4faeb51-6114-46cc-a7df-ff844503afac-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-dvc6v\" (UID: \"e4faeb51-6114-46cc-a7df-ff844503afac\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-dvc6v" Jul 12 18:23:28.555577 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:28.555552 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfqld\" (UniqueName: \"kubernetes.io/projected/e4faeb51-6114-46cc-a7df-ff844503afac-kube-api-access-kfqld\") pod \"istiod-openshift-gateway-94bb8fbfd-dvc6v\" (UID: \"e4faeb51-6114-46cc-a7df-ff844503afac\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-dvc6v" Jul 12 18:23:28.558269 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:28.558248 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4faeb51-6114-46cc-a7df-ff844503afac-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-dvc6v\" (UID: \"e4faeb51-6114-46cc-a7df-ff844503afac\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-dvc6v" Jul 12 18:23:28.594236 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:28.594186 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-dvc6v" Jul 12 18:23:30.029656 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:30.029550 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 12 18:23:30.029656 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:30.029644 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 12 18:23:30.030084 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:30.029687 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 12 18:23:30.192949 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:30.192922 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-dvc6v"] Jul 12 18:23:30.194940 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:23:30.194901 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4faeb51_6114_46cc_a7df_ff844503afac.slice/crio-b000c3ef8cb1834d834802f319241a27087a88ba82eb7bbe3a835a37cf784f97 WatchSource:0}: Error finding container b000c3ef8cb1834d834802f319241a27087a88ba82eb7bbe3a835a37cf784f97: Status 404 returned error can't find the container with id b000c3ef8cb1834d834802f319241a27087a88ba82eb7bbe3a835a37cf784f97 Jul 12 18:23:30.816690 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:30.816524 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678qjql2" event={"ID":"05e3c37f-45b4-47dc-9c35-cc81ee6dd97a","Type":"ContainerStarted","Data":"e529ef819129aec03aa09ad3711649b77c5e57e9a4b2440332e6d729ce60db61"} Jul 12 18:23:30.818448 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:30.818418 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-dvc6v" event={"ID":"e4faeb51-6114-46cc-a7df-ff844503afac","Type":"ContainerStarted","Data":"b000c3ef8cb1834d834802f319241a27087a88ba82eb7bbe3a835a37cf784f97"} Jul 12 18:23:30.826843 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:30.821832 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-9757cf8df-rcmmq" event={"ID":"19d37878-abe2-47a1-9194-6400f779fe95","Type":"ContainerStarted","Data":"3af7737948ff48ba95eb61bff3fadce9cfb2b76700a9496162095eec8bdc0efd"} Jul 12 18:23:30.826843 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:30.822767 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-9757cf8df-rcmmq" Jul 12 18:23:30.856044 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:30.855972 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678qjql2" podStartSLOduration=1.760790948 podStartE2EDuration="4.855949244s" totalImagesPullingTime="3.095158296s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:23:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:23:30.85456226 +0000 UTC m=+383.981467066" watchObservedRunningTime="2026-07-12 18:23:30.855949244 +0000 UTC m=+383.982854052" Jul 12 18:23:30.885072 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:30.885009 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-9757cf8df-rcmmq" podStartSLOduration=2.100492809 podStartE2EDuration="4.884985167s" totalImagesPullingTime="2.784492358s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:23:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:23:30.884276448 +0000 UTC m=+384.011181253" watchObservedRunningTime="2026-07-12 18:23:30.884985167 +0000 UTC m=+384.011889973" Jul 12 18:23:31.790169 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:31.790112 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678qjql2" Jul 12 18:23:31.795486 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:31.795458 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678qjql2" Jul 12 18:23:31.826714 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:31.826689 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678qjql2" Jul 12 18:23:31.827801 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:31.827773 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678qjql2" Jul 12 18:23:32.488963 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:32.488935 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 12 18:23:32.489028 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:32.488999 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 12 18:23:32.832898 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:32.832798 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-dvc6v" event={"ID":"e4faeb51-6114-46cc-a7df-ff844503afac","Type":"ContainerStarted","Data":"ca7f9bc65c9b45b1080df821d3dbab027c1d793c5adac7f15c9fc2be761d1793"} Jul 12 18:23:32.833281 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:32.833037 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-dvc6v" Jul 12 18:23:32.869806 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:32.869709 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-dvc6v" podStartSLOduration=2.578054924 podStartE2EDuration="4.869688259s" totalImagesPullingTime="2.291633335s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:23:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:23:32.868425097 +0000 UTC m=+385.995329904" watchObservedRunningTime="2026-07-12 18:23:32.869688259 +0000 UTC m=+385.996593066" Jul 12 18:23:33.839931 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:33.839902 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-dvc6v" Jul 12 18:23:34.775390 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:34.775353 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-g4dgx" Jul 12 18:23:42.834902 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:23:42.834868 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-9757cf8df-rcmmq" Jul 12 18:24:34.916142 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:34.916061 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vpk6h"] Jul 12 18:24:34.919479 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:34.919462 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vpk6h" Jul 12 18:24:34.921807 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:34.921782 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 12 18:24:34.922083 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:34.922068 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 12 18:24:34.922853 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:34.922829 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-crn2p\"" Jul 12 18:24:34.928025 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:34.928001 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vpk6h"] Jul 12 18:24:35.030812 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:35.030776 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7l6n\" (UniqueName: \"kubernetes.io/projected/124f8ebf-36d0-469a-b3b3-c932743898c9-kube-api-access-j7l6n\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vpk6h\" (UID: \"124f8ebf-36d0-469a-b3b3-c932743898c9\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vpk6h" Jul 12 18:24:35.031001 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:35.030854 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/124f8ebf-36d0-469a-b3b3-c932743898c9-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vpk6h\" (UID: \"124f8ebf-36d0-469a-b3b3-c932743898c9\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vpk6h" Jul 12 18:24:35.031001 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:35.030897 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/124f8ebf-36d0-469a-b3b3-c932743898c9-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vpk6h\" (UID: \"124f8ebf-36d0-469a-b3b3-c932743898c9\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vpk6h" Jul 12 18:24:35.132296 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:35.132260 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7l6n\" (UniqueName: \"kubernetes.io/projected/124f8ebf-36d0-469a-b3b3-c932743898c9-kube-api-access-j7l6n\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vpk6h\" (UID: \"124f8ebf-36d0-469a-b3b3-c932743898c9\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vpk6h" Jul 12 18:24:35.132454 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:35.132323 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/124f8ebf-36d0-469a-b3b3-c932743898c9-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vpk6h\" (UID: \"124f8ebf-36d0-469a-b3b3-c932743898c9\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vpk6h" Jul 12 18:24:35.132454 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:35.132350 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/124f8ebf-36d0-469a-b3b3-c932743898c9-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vpk6h\" (UID: \"124f8ebf-36d0-469a-b3b3-c932743898c9\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vpk6h" Jul 12 18:24:35.132687 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:35.132669 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/124f8ebf-36d0-469a-b3b3-c932743898c9-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vpk6h\" (UID: \"124f8ebf-36d0-469a-b3b3-c932743898c9\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vpk6h" Jul 12 18:24:35.132741 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:35.132696 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/124f8ebf-36d0-469a-b3b3-c932743898c9-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vpk6h\" (UID: \"124f8ebf-36d0-469a-b3b3-c932743898c9\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vpk6h" Jul 12 18:24:35.140598 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:35.140568 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7l6n\" (UniqueName: \"kubernetes.io/projected/124f8ebf-36d0-469a-b3b3-c932743898c9-kube-api-access-j7l6n\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vpk6h\" (UID: \"124f8ebf-36d0-469a-b3b3-c932743898c9\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vpk6h" Jul 12 18:24:35.229806 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:35.229775 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vpk6h" Jul 12 18:24:35.366617 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:35.366595 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vpk6h"] Jul 12 18:24:35.368849 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:24:35.368822 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod124f8ebf_36d0_469a_b3b3_c932743898c9.slice/crio-e60bb4749df2be406c78c7df74f572415871e0b8cbed2fa84e2bceef383aabf5 WatchSource:0}: Error finding container e60bb4749df2be406c78c7df74f572415871e0b8cbed2fa84e2bceef383aabf5: Status 404 returned error can't find the container with id e60bb4749df2be406c78c7df74f572415871e0b8cbed2fa84e2bceef383aabf5 Jul 12 18:24:35.516204 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:35.516132 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gp9dz"] Jul 12 18:24:35.519813 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:35.519795 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gp9dz" Jul 12 18:24:35.527489 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:35.527460 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gp9dz"] Jul 12 18:24:35.636416 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:35.636381 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgdps\" (UniqueName: \"kubernetes.io/projected/f6d5bf1a-f136-45cf-9c46-1d4cad723409-kube-api-access-qgdps\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gp9dz\" (UID: \"f6d5bf1a-f136-45cf-9c46-1d4cad723409\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gp9dz" Jul 12 18:24:35.636606 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:35.636445 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f6d5bf1a-f136-45cf-9c46-1d4cad723409-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gp9dz\" (UID: \"f6d5bf1a-f136-45cf-9c46-1d4cad723409\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gp9dz" Jul 12 18:24:35.636606 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:35.636527 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f6d5bf1a-f136-45cf-9c46-1d4cad723409-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gp9dz\" (UID: \"f6d5bf1a-f136-45cf-9c46-1d4cad723409\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gp9dz" Jul 12 18:24:35.737336 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:35.737305 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgdps\" (UniqueName: \"kubernetes.io/projected/f6d5bf1a-f136-45cf-9c46-1d4cad723409-kube-api-access-qgdps\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gp9dz\" (UID: \"f6d5bf1a-f136-45cf-9c46-1d4cad723409\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gp9dz" Jul 12 18:24:35.737529 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:35.737361 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f6d5bf1a-f136-45cf-9c46-1d4cad723409-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gp9dz\" (UID: \"f6d5bf1a-f136-45cf-9c46-1d4cad723409\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gp9dz" Jul 12 18:24:35.737529 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:35.737423 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f6d5bf1a-f136-45cf-9c46-1d4cad723409-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gp9dz\" (UID: \"f6d5bf1a-f136-45cf-9c46-1d4cad723409\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gp9dz" Jul 12 18:24:35.737788 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:35.737768 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f6d5bf1a-f136-45cf-9c46-1d4cad723409-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gp9dz\" (UID: \"f6d5bf1a-f136-45cf-9c46-1d4cad723409\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gp9dz" Jul 12 18:24:35.737871 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:35.737816 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f6d5bf1a-f136-45cf-9c46-1d4cad723409-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gp9dz\" (UID: \"f6d5bf1a-f136-45cf-9c46-1d4cad723409\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gp9dz" Jul 12 18:24:35.745459 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:35.745431 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgdps\" (UniqueName: \"kubernetes.io/projected/f6d5bf1a-f136-45cf-9c46-1d4cad723409-kube-api-access-qgdps\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gp9dz\" (UID: \"f6d5bf1a-f136-45cf-9c46-1d4cad723409\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gp9dz" Jul 12 18:24:35.841788 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:35.841694 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gp9dz" Jul 12 18:24:35.974147 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:35.974120 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gp9dz"] Jul 12 18:24:35.976280 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:24:35.976248 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6d5bf1a_f136_45cf_9c46_1d4cad723409.slice/crio-10c30ed2bf274b26caa5e3dce0aa1a1272e368f8732a208c6f05daa4fab93d40 WatchSource:0}: Error finding container 10c30ed2bf274b26caa5e3dce0aa1a1272e368f8732a208c6f05daa4fab93d40: Status 404 returned error can't find the container with id 10c30ed2bf274b26caa5e3dce0aa1a1272e368f8732a208c6f05daa4fab93d40 Jul 12 18:24:36.114145 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:36.114073 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f4r57"] Jul 12 18:24:36.117750 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:36.117728 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f4r57" Jul 12 18:24:36.121419 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:36.121394 2577 generic.go:356] "Generic (PLEG): container finished" podID="f6d5bf1a-f136-45cf-9c46-1d4cad723409" containerID="dfcfdc7de160a25583aee6833dad9afa3a05572bb6db7979edeaebc1a6f96fd0" exitCode=0 Jul 12 18:24:36.121538 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:36.121481 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gp9dz" event={"ID":"f6d5bf1a-f136-45cf-9c46-1d4cad723409","Type":"ContainerDied","Data":"dfcfdc7de160a25583aee6833dad9afa3a05572bb6db7979edeaebc1a6f96fd0"} Jul 12 18:24:36.121538 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:36.121525 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gp9dz" event={"ID":"f6d5bf1a-f136-45cf-9c46-1d4cad723409","Type":"ContainerStarted","Data":"10c30ed2bf274b26caa5e3dce0aa1a1272e368f8732a208c6f05daa4fab93d40"} Jul 12 18:24:36.124087 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:36.123115 2577 generic.go:356] "Generic (PLEG): container finished" podID="124f8ebf-36d0-469a-b3b3-c932743898c9" containerID="c31f956b25f34d1c2a7ead3cf681ad0730e0205611912cc4733a1208cda992b9" exitCode=0 Jul 12 18:24:36.124087 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:36.123165 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vpk6h" event={"ID":"124f8ebf-36d0-469a-b3b3-c932743898c9","Type":"ContainerDied","Data":"c31f956b25f34d1c2a7ead3cf681ad0730e0205611912cc4733a1208cda992b9"} Jul 12 18:24:36.124087 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:36.123199 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vpk6h" event={"ID":"124f8ebf-36d0-469a-b3b3-c932743898c9","Type":"ContainerStarted","Data":"e60bb4749df2be406c78c7df74f572415871e0b8cbed2fa84e2bceef383aabf5"} Jul 12 18:24:36.126277 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:36.126253 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f4r57"] Jul 12 18:24:36.242307 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:36.242274 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c3e07319-173b-4c5c-85ef-3af760954600-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f4r57\" (UID: \"c3e07319-173b-4c5c-85ef-3af760954600\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f4r57" Jul 12 18:24:36.242307 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:36.242307 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-np2p6\" (UniqueName: \"kubernetes.io/projected/c3e07319-173b-4c5c-85ef-3af760954600-kube-api-access-np2p6\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f4r57\" (UID: \"c3e07319-173b-4c5c-85ef-3af760954600\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f4r57" Jul 12 18:24:36.242541 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:36.242380 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c3e07319-173b-4c5c-85ef-3af760954600-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f4r57\" (UID: \"c3e07319-173b-4c5c-85ef-3af760954600\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f4r57" Jul 12 18:24:36.343280 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:36.343235 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c3e07319-173b-4c5c-85ef-3af760954600-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f4r57\" (UID: \"c3e07319-173b-4c5c-85ef-3af760954600\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f4r57" Jul 12 18:24:36.343280 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:36.343285 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-np2p6\" (UniqueName: \"kubernetes.io/projected/c3e07319-173b-4c5c-85ef-3af760954600-kube-api-access-np2p6\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f4r57\" (UID: \"c3e07319-173b-4c5c-85ef-3af760954600\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f4r57" Jul 12 18:24:36.343548 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:36.343407 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c3e07319-173b-4c5c-85ef-3af760954600-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f4r57\" (UID: \"c3e07319-173b-4c5c-85ef-3af760954600\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f4r57" Jul 12 18:24:36.343722 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:36.343703 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c3e07319-173b-4c5c-85ef-3af760954600-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f4r57\" (UID: \"c3e07319-173b-4c5c-85ef-3af760954600\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f4r57" Jul 12 18:24:36.343810 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:36.343743 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c3e07319-173b-4c5c-85ef-3af760954600-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f4r57\" (UID: \"c3e07319-173b-4c5c-85ef-3af760954600\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f4r57" Jul 12 18:24:36.352056 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:36.352031 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-np2p6\" (UniqueName: \"kubernetes.io/projected/c3e07319-173b-4c5c-85ef-3af760954600-kube-api-access-np2p6\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f4r57\" (UID: \"c3e07319-173b-4c5c-85ef-3af760954600\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f4r57" Jul 12 18:24:36.469302 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:36.469271 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f4r57" Jul 12 18:24:36.633850 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:36.633784 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f4r57"] Jul 12 18:24:36.639456 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:24:36.639424 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3e07319_173b_4c5c_85ef_3af760954600.slice/crio-1a8f7d0529e835048794c42c72cfc4fef0d78ec38ebc264f50c8292ebb993c4f WatchSource:0}: Error finding container 1a8f7d0529e835048794c42c72cfc4fef0d78ec38ebc264f50c8292ebb993c4f: Status 404 returned error can't find the container with id 1a8f7d0529e835048794c42c72cfc4fef0d78ec38ebc264f50c8292ebb993c4f Jul 12 18:24:36.716535 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:36.716501 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fnqbh"] Jul 12 18:24:36.720414 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:36.720348 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fnqbh" Jul 12 18:24:36.729720 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:36.729689 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fnqbh"] Jul 12 18:24:36.851678 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:36.851640 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5b65d5ed-ac9a-48bc-8aef-3c608afe4f79-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fnqbh\" (UID: \"5b65d5ed-ac9a-48bc-8aef-3c608afe4f79\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fnqbh" Jul 12 18:24:36.851836 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:36.851720 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jcvn\" (UniqueName: \"kubernetes.io/projected/5b65d5ed-ac9a-48bc-8aef-3c608afe4f79-kube-api-access-8jcvn\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fnqbh\" (UID: \"5b65d5ed-ac9a-48bc-8aef-3c608afe4f79\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fnqbh" Jul 12 18:24:36.851836 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:36.851770 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5b65d5ed-ac9a-48bc-8aef-3c608afe4f79-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fnqbh\" (UID: \"5b65d5ed-ac9a-48bc-8aef-3c608afe4f79\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fnqbh" Jul 12 18:24:36.952978 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:36.952953 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jcvn\" (UniqueName: \"kubernetes.io/projected/5b65d5ed-ac9a-48bc-8aef-3c608afe4f79-kube-api-access-8jcvn\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fnqbh\" (UID: \"5b65d5ed-ac9a-48bc-8aef-3c608afe4f79\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fnqbh" Jul 12 18:24:36.953113 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:36.952994 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5b65d5ed-ac9a-48bc-8aef-3c608afe4f79-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fnqbh\" (UID: \"5b65d5ed-ac9a-48bc-8aef-3c608afe4f79\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fnqbh" Jul 12 18:24:36.953113 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:36.953106 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5b65d5ed-ac9a-48bc-8aef-3c608afe4f79-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fnqbh\" (UID: \"5b65d5ed-ac9a-48bc-8aef-3c608afe4f79\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fnqbh" Jul 12 18:24:36.953381 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:36.953362 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5b65d5ed-ac9a-48bc-8aef-3c608afe4f79-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fnqbh\" (UID: \"5b65d5ed-ac9a-48bc-8aef-3c608afe4f79\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fnqbh" Jul 12 18:24:36.953425 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:36.953409 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5b65d5ed-ac9a-48bc-8aef-3c608afe4f79-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fnqbh\" (UID: \"5b65d5ed-ac9a-48bc-8aef-3c608afe4f79\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fnqbh" Jul 12 18:24:36.964934 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:36.964905 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jcvn\" (UniqueName: \"kubernetes.io/projected/5b65d5ed-ac9a-48bc-8aef-3c608afe4f79-kube-api-access-8jcvn\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fnqbh\" (UID: \"5b65d5ed-ac9a-48bc-8aef-3c608afe4f79\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fnqbh" Jul 12 18:24:37.031713 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:37.031684 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fnqbh" Jul 12 18:24:37.130481 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:37.130446 2577 generic.go:356] "Generic (PLEG): container finished" podID="124f8ebf-36d0-469a-b3b3-c932743898c9" containerID="2ee42cf4e8187adab0a6db906d58148b6f04aba582300781c5d3615a129016a1" exitCode=0 Jul 12 18:24:37.130660 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:37.130530 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vpk6h" event={"ID":"124f8ebf-36d0-469a-b3b3-c932743898c9","Type":"ContainerDied","Data":"2ee42cf4e8187adab0a6db906d58148b6f04aba582300781c5d3615a129016a1"} Jul 12 18:24:37.132465 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:37.132335 2577 generic.go:356] "Generic (PLEG): container finished" podID="c3e07319-173b-4c5c-85ef-3af760954600" containerID="469f0037243ff1fa1612d2f03e493b1206b5fd95c85f40dc08faa13ddef2ae4a" exitCode=0 Jul 12 18:24:37.132687 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:37.132664 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f4r57" event={"ID":"c3e07319-173b-4c5c-85ef-3af760954600","Type":"ContainerDied","Data":"469f0037243ff1fa1612d2f03e493b1206b5fd95c85f40dc08faa13ddef2ae4a"} Jul 12 18:24:37.132787 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:37.132714 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f4r57" event={"ID":"c3e07319-173b-4c5c-85ef-3af760954600","Type":"ContainerStarted","Data":"1a8f7d0529e835048794c42c72cfc4fef0d78ec38ebc264f50c8292ebb993c4f"} Jul 12 18:24:37.135716 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:37.135460 2577 generic.go:356] "Generic (PLEG): container finished" podID="f6d5bf1a-f136-45cf-9c46-1d4cad723409" containerID="76896a4e8084bee74639097b5b10499086c4a4e842ed19d080a5e7505f7cfd69" exitCode=0 Jul 12 18:24:37.135716 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:37.135493 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gp9dz" event={"ID":"f6d5bf1a-f136-45cf-9c46-1d4cad723409","Type":"ContainerDied","Data":"76896a4e8084bee74639097b5b10499086c4a4e842ed19d080a5e7505f7cfd69"} Jul 12 18:24:37.165755 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:37.165731 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fnqbh"] Jul 12 18:24:37.169728 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:24:37.169700 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b65d5ed_ac9a_48bc_8aef_3c608afe4f79.slice/crio-0951a054e02933df9269e8a9638674f8e9fdeb90274a7be23c6039d8e99e0529 WatchSource:0}: Error finding container 0951a054e02933df9269e8a9638674f8e9fdeb90274a7be23c6039d8e99e0529: Status 404 returned error can't find the container with id 0951a054e02933df9269e8a9638674f8e9fdeb90274a7be23c6039d8e99e0529 Jul 12 18:24:38.142740 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:38.142708 2577 generic.go:356] "Generic (PLEG): container finished" podID="5b65d5ed-ac9a-48bc-8aef-3c608afe4f79" containerID="a119976d1a16cc46456cf5777b23c346b960e28b24a07371428c6e02829c5673" exitCode=0 Jul 12 18:24:38.143144 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:38.142783 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fnqbh" event={"ID":"5b65d5ed-ac9a-48bc-8aef-3c608afe4f79","Type":"ContainerDied","Data":"a119976d1a16cc46456cf5777b23c346b960e28b24a07371428c6e02829c5673"} Jul 12 18:24:38.143144 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:38.142830 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fnqbh" event={"ID":"5b65d5ed-ac9a-48bc-8aef-3c608afe4f79","Type":"ContainerStarted","Data":"0951a054e02933df9269e8a9638674f8e9fdeb90274a7be23c6039d8e99e0529"} Jul 12 18:24:38.145093 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:38.145040 2577 generic.go:356] "Generic (PLEG): container finished" podID="f6d5bf1a-f136-45cf-9c46-1d4cad723409" containerID="854451d021c7fb133e434c0a3983bddc3aaaf6353e9715581f863f8150ed4a1b" exitCode=0 Jul 12 18:24:38.145154 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:38.145118 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gp9dz" event={"ID":"f6d5bf1a-f136-45cf-9c46-1d4cad723409","Type":"ContainerDied","Data":"854451d021c7fb133e434c0a3983bddc3aaaf6353e9715581f863f8150ed4a1b"} Jul 12 18:24:38.147105 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:38.147083 2577 generic.go:356] "Generic (PLEG): container finished" podID="124f8ebf-36d0-469a-b3b3-c932743898c9" containerID="7f6a577e12b10c53d41490025811bf4ac2a3e29d3af9a3c8067e9ea972e2da60" exitCode=0 Jul 12 18:24:38.147210 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:38.147127 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vpk6h" event={"ID":"124f8ebf-36d0-469a-b3b3-c932743898c9","Type":"ContainerDied","Data":"7f6a577e12b10c53d41490025811bf4ac2a3e29d3af9a3c8067e9ea972e2da60"} Jul 12 18:24:38.148772 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:38.148754 2577 generic.go:356] "Generic (PLEG): container finished" podID="c3e07319-173b-4c5c-85ef-3af760954600" containerID="1edb967308743428ead5ae64a6e4f6a775d4812b1d09296d2efa44c98a5709bc" exitCode=0 Jul 12 18:24:38.148864 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:38.148780 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f4r57" event={"ID":"c3e07319-173b-4c5c-85ef-3af760954600","Type":"ContainerDied","Data":"1edb967308743428ead5ae64a6e4f6a775d4812b1d09296d2efa44c98a5709bc"} Jul 12 18:24:39.154936 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:39.154853 2577 generic.go:356] "Generic (PLEG): container finished" podID="5b65d5ed-ac9a-48bc-8aef-3c608afe4f79" containerID="19e2490e4ff8183bdd5dcc0bf6d9af38fe4bbcb6d1048863d2edc4d74caf7fec" exitCode=0 Jul 12 18:24:39.155363 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:39.154936 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fnqbh" event={"ID":"5b65d5ed-ac9a-48bc-8aef-3c608afe4f79","Type":"ContainerDied","Data":"19e2490e4ff8183bdd5dcc0bf6d9af38fe4bbcb6d1048863d2edc4d74caf7fec"} Jul 12 18:24:39.157163 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:39.157139 2577 generic.go:356] "Generic (PLEG): container finished" podID="c3e07319-173b-4c5c-85ef-3af760954600" containerID="8f4b02250b63e684affd9e30ee3a6f5b01ea177ccdd712ca08d4539c9adcaf25" exitCode=0 Jul 12 18:24:39.157260 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:39.157203 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f4r57" event={"ID":"c3e07319-173b-4c5c-85ef-3af760954600","Type":"ContainerDied","Data":"8f4b02250b63e684affd9e30ee3a6f5b01ea177ccdd712ca08d4539c9adcaf25"} Jul 12 18:24:39.325219 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:39.325182 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gp9dz" Jul 12 18:24:39.328336 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:39.328305 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vpk6h" Jul 12 18:24:39.375046 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:39.375017 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/124f8ebf-36d0-469a-b3b3-c932743898c9-util\") pod \"124f8ebf-36d0-469a-b3b3-c932743898c9\" (UID: \"124f8ebf-36d0-469a-b3b3-c932743898c9\") " Jul 12 18:24:39.375200 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:39.375054 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f6d5bf1a-f136-45cf-9c46-1d4cad723409-util\") pod \"f6d5bf1a-f136-45cf-9c46-1d4cad723409\" (UID: \"f6d5bf1a-f136-45cf-9c46-1d4cad723409\") " Jul 12 18:24:39.375200 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:39.375102 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qgdps\" (UniqueName: \"kubernetes.io/projected/f6d5bf1a-f136-45cf-9c46-1d4cad723409-kube-api-access-qgdps\") pod \"f6d5bf1a-f136-45cf-9c46-1d4cad723409\" (UID: \"f6d5bf1a-f136-45cf-9c46-1d4cad723409\") " Jul 12 18:24:39.375200 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:39.375153 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f6d5bf1a-f136-45cf-9c46-1d4cad723409-bundle\") pod \"f6d5bf1a-f136-45cf-9c46-1d4cad723409\" (UID: \"f6d5bf1a-f136-45cf-9c46-1d4cad723409\") " Jul 12 18:24:39.375392 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:39.375217 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/124f8ebf-36d0-469a-b3b3-c932743898c9-bundle\") pod \"124f8ebf-36d0-469a-b3b3-c932743898c9\" (UID: \"124f8ebf-36d0-469a-b3b3-c932743898c9\") " Jul 12 18:24:39.375392 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:39.375259 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j7l6n\" (UniqueName: \"kubernetes.io/projected/124f8ebf-36d0-469a-b3b3-c932743898c9-kube-api-access-j7l6n\") pod \"124f8ebf-36d0-469a-b3b3-c932743898c9\" (UID: \"124f8ebf-36d0-469a-b3b3-c932743898c9\") " Jul 12 18:24:39.375901 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:39.375859 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/124f8ebf-36d0-469a-b3b3-c932743898c9-bundle" (OuterVolumeSpecName: "bundle") pod "124f8ebf-36d0-469a-b3b3-c932743898c9" (UID: "124f8ebf-36d0-469a-b3b3-c932743898c9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:24:39.376012 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:39.375923 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6d5bf1a-f136-45cf-9c46-1d4cad723409-bundle" (OuterVolumeSpecName: "bundle") pod "f6d5bf1a-f136-45cf-9c46-1d4cad723409" (UID: "f6d5bf1a-f136-45cf-9c46-1d4cad723409"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:24:39.377411 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:39.377386 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6d5bf1a-f136-45cf-9c46-1d4cad723409-kube-api-access-qgdps" (OuterVolumeSpecName: "kube-api-access-qgdps") pod "f6d5bf1a-f136-45cf-9c46-1d4cad723409" (UID: "f6d5bf1a-f136-45cf-9c46-1d4cad723409"). InnerVolumeSpecName "kube-api-access-qgdps". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:24:39.377966 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:39.377942 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/124f8ebf-36d0-469a-b3b3-c932743898c9-kube-api-access-j7l6n" (OuterVolumeSpecName: "kube-api-access-j7l6n") pod "124f8ebf-36d0-469a-b3b3-c932743898c9" (UID: "124f8ebf-36d0-469a-b3b3-c932743898c9"). InnerVolumeSpecName "kube-api-access-j7l6n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:24:39.381269 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:39.381244 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6d5bf1a-f136-45cf-9c46-1d4cad723409-util" (OuterVolumeSpecName: "util") pod "f6d5bf1a-f136-45cf-9c46-1d4cad723409" (UID: "f6d5bf1a-f136-45cf-9c46-1d4cad723409"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:24:39.384783 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:39.384763 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/124f8ebf-36d0-469a-b3b3-c932743898c9-util" (OuterVolumeSpecName: "util") pod "124f8ebf-36d0-469a-b3b3-c932743898c9" (UID: "124f8ebf-36d0-469a-b3b3-c932743898c9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:24:39.476701 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:39.476672 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/124f8ebf-36d0-469a-b3b3-c932743898c9-util\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:24:39.476701 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:39.476698 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f6d5bf1a-f136-45cf-9c46-1d4cad723409-util\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:24:39.476701 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:39.476707 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qgdps\" (UniqueName: \"kubernetes.io/projected/f6d5bf1a-f136-45cf-9c46-1d4cad723409-kube-api-access-qgdps\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:24:39.476919 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:39.476719 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f6d5bf1a-f136-45cf-9c46-1d4cad723409-bundle\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:24:39.476919 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:39.476729 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/124f8ebf-36d0-469a-b3b3-c932743898c9-bundle\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:24:39.476919 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:39.476736 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j7l6n\" (UniqueName: \"kubernetes.io/projected/124f8ebf-36d0-469a-b3b3-c932743898c9-kube-api-access-j7l6n\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:24:40.164241 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:40.164207 2577 generic.go:356] "Generic (PLEG): container finished" podID="5b65d5ed-ac9a-48bc-8aef-3c608afe4f79" containerID="39d3efa3a7201195f88c940fc743c07bbd0f25efb7ebed54ff7ff97bf8e28ae2" exitCode=0 Jul 12 18:24:40.164646 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:40.164288 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fnqbh" event={"ID":"5b65d5ed-ac9a-48bc-8aef-3c608afe4f79","Type":"ContainerDied","Data":"39d3efa3a7201195f88c940fc743c07bbd0f25efb7ebed54ff7ff97bf8e28ae2"} Jul 12 18:24:40.166118 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:40.166095 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gp9dz" event={"ID":"f6d5bf1a-f136-45cf-9c46-1d4cad723409","Type":"ContainerDied","Data":"10c30ed2bf274b26caa5e3dce0aa1a1272e368f8732a208c6f05daa4fab93d40"} Jul 12 18:24:40.166245 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:40.166134 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="10c30ed2bf274b26caa5e3dce0aa1a1272e368f8732a208c6f05daa4fab93d40" Jul 12 18:24:40.166245 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:40.166100 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gp9dz" Jul 12 18:24:40.168007 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:40.167985 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vpk6h" Jul 12 18:24:40.168125 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:40.168016 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vpk6h" event={"ID":"124f8ebf-36d0-469a-b3b3-c932743898c9","Type":"ContainerDied","Data":"e60bb4749df2be406c78c7df74f572415871e0b8cbed2fa84e2bceef383aabf5"} Jul 12 18:24:40.168125 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:40.168050 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e60bb4749df2be406c78c7df74f572415871e0b8cbed2fa84e2bceef383aabf5" Jul 12 18:24:40.300560 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:40.300538 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f4r57" Jul 12 18:24:40.384410 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:40.384378 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c3e07319-173b-4c5c-85ef-3af760954600-bundle\") pod \"c3e07319-173b-4c5c-85ef-3af760954600\" (UID: \"c3e07319-173b-4c5c-85ef-3af760954600\") " Jul 12 18:24:40.384607 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:40.384426 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-np2p6\" (UniqueName: \"kubernetes.io/projected/c3e07319-173b-4c5c-85ef-3af760954600-kube-api-access-np2p6\") pod \"c3e07319-173b-4c5c-85ef-3af760954600\" (UID: \"c3e07319-173b-4c5c-85ef-3af760954600\") " Jul 12 18:24:40.384607 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:40.384467 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c3e07319-173b-4c5c-85ef-3af760954600-util\") pod \"c3e07319-173b-4c5c-85ef-3af760954600\" (UID: \"c3e07319-173b-4c5c-85ef-3af760954600\") " Jul 12 18:24:40.384899 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:40.384875 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3e07319-173b-4c5c-85ef-3af760954600-bundle" (OuterVolumeSpecName: "bundle") pod "c3e07319-173b-4c5c-85ef-3af760954600" (UID: "c3e07319-173b-4c5c-85ef-3af760954600"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:24:40.386522 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:40.386490 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3e07319-173b-4c5c-85ef-3af760954600-kube-api-access-np2p6" (OuterVolumeSpecName: "kube-api-access-np2p6") pod "c3e07319-173b-4c5c-85ef-3af760954600" (UID: "c3e07319-173b-4c5c-85ef-3af760954600"). InnerVolumeSpecName "kube-api-access-np2p6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:24:40.390289 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:40.390252 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3e07319-173b-4c5c-85ef-3af760954600-util" (OuterVolumeSpecName: "util") pod "c3e07319-173b-4c5c-85ef-3af760954600" (UID: "c3e07319-173b-4c5c-85ef-3af760954600"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:24:40.486024 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:40.485980 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c3e07319-173b-4c5c-85ef-3af760954600-util\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:24:40.486024 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:40.486021 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c3e07319-173b-4c5c-85ef-3af760954600-bundle\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:24:40.486024 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:40.486032 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-np2p6\" (UniqueName: \"kubernetes.io/projected/c3e07319-173b-4c5c-85ef-3af760954600-kube-api-access-np2p6\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:24:41.181210 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:41.181181 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f4r57" Jul 12 18:24:41.181210 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:41.181199 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f4r57" event={"ID":"c3e07319-173b-4c5c-85ef-3af760954600","Type":"ContainerDied","Data":"1a8f7d0529e835048794c42c72cfc4fef0d78ec38ebc264f50c8292ebb993c4f"} Jul 12 18:24:41.181646 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:41.181266 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1a8f7d0529e835048794c42c72cfc4fef0d78ec38ebc264f50c8292ebb993c4f" Jul 12 18:24:41.303822 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:41.303800 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fnqbh" Jul 12 18:24:41.393499 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:41.393410 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5b65d5ed-ac9a-48bc-8aef-3c608afe4f79-bundle\") pod \"5b65d5ed-ac9a-48bc-8aef-3c608afe4f79\" (UID: \"5b65d5ed-ac9a-48bc-8aef-3c608afe4f79\") " Jul 12 18:24:41.393499 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:41.393498 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5b65d5ed-ac9a-48bc-8aef-3c608afe4f79-util\") pod \"5b65d5ed-ac9a-48bc-8aef-3c608afe4f79\" (UID: \"5b65d5ed-ac9a-48bc-8aef-3c608afe4f79\") " Jul 12 18:24:41.393716 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:41.393534 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8jcvn\" (UniqueName: \"kubernetes.io/projected/5b65d5ed-ac9a-48bc-8aef-3c608afe4f79-kube-api-access-8jcvn\") pod \"5b65d5ed-ac9a-48bc-8aef-3c608afe4f79\" (UID: \"5b65d5ed-ac9a-48bc-8aef-3c608afe4f79\") " Jul 12 18:24:41.394047 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:41.394019 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b65d5ed-ac9a-48bc-8aef-3c608afe4f79-bundle" (OuterVolumeSpecName: "bundle") pod "5b65d5ed-ac9a-48bc-8aef-3c608afe4f79" (UID: "5b65d5ed-ac9a-48bc-8aef-3c608afe4f79"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:24:41.395573 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:41.395553 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b65d5ed-ac9a-48bc-8aef-3c608afe4f79-kube-api-access-8jcvn" (OuterVolumeSpecName: "kube-api-access-8jcvn") pod "5b65d5ed-ac9a-48bc-8aef-3c608afe4f79" (UID: "5b65d5ed-ac9a-48bc-8aef-3c608afe4f79"). InnerVolumeSpecName "kube-api-access-8jcvn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:24:41.399202 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:41.399165 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b65d5ed-ac9a-48bc-8aef-3c608afe4f79-util" (OuterVolumeSpecName: "util") pod "5b65d5ed-ac9a-48bc-8aef-3c608afe4f79" (UID: "5b65d5ed-ac9a-48bc-8aef-3c608afe4f79"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:24:41.494449 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:41.494408 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5b65d5ed-ac9a-48bc-8aef-3c608afe4f79-util\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:24:41.494449 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:41.494441 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8jcvn\" (UniqueName: \"kubernetes.io/projected/5b65d5ed-ac9a-48bc-8aef-3c608afe4f79-kube-api-access-8jcvn\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:24:41.494449 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:41.494452 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5b65d5ed-ac9a-48bc-8aef-3c608afe4f79-bundle\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:24:42.188264 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:42.188230 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fnqbh" Jul 12 18:24:42.188264 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:42.188239 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fnqbh" event={"ID":"5b65d5ed-ac9a-48bc-8aef-3c608afe4f79","Type":"ContainerDied","Data":"0951a054e02933df9269e8a9638674f8e9fdeb90274a7be23c6039d8e99e0529"} Jul 12 18:24:42.188721 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:42.188285 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0951a054e02933df9269e8a9638674f8e9fdeb90274a7be23c6039d8e99e0529" Jul 12 18:24:58.717928 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:58.717895 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-747d58bd59-b7nw9"] Jul 12 18:24:58.718276 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:58.718236 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6d5bf1a-f136-45cf-9c46-1d4cad723409" containerName="util" Jul 12 18:24:58.718276 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:58.718248 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6d5bf1a-f136-45cf-9c46-1d4cad723409" containerName="util" Jul 12 18:24:58.718276 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:58.718255 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3e07319-173b-4c5c-85ef-3af760954600" containerName="util" Jul 12 18:24:58.718276 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:58.718261 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3e07319-173b-4c5c-85ef-3af760954600" containerName="util" Jul 12 18:24:58.718276 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:58.718268 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="124f8ebf-36d0-469a-b3b3-c932743898c9" containerName="extract" Jul 12 18:24:58.718276 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:58.718276 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="124f8ebf-36d0-469a-b3b3-c932743898c9" containerName="extract" Jul 12 18:24:58.718471 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:58.718293 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b65d5ed-ac9a-48bc-8aef-3c608afe4f79" containerName="util" Jul 12 18:24:58.718471 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:58.718300 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b65d5ed-ac9a-48bc-8aef-3c608afe4f79" containerName="util" Jul 12 18:24:58.718471 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:58.718306 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="124f8ebf-36d0-469a-b3b3-c932743898c9" containerName="pull" Jul 12 18:24:58.718471 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:58.718311 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="124f8ebf-36d0-469a-b3b3-c932743898c9" containerName="pull" Jul 12 18:24:58.718471 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:58.718318 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="124f8ebf-36d0-469a-b3b3-c932743898c9" containerName="util" Jul 12 18:24:58.718471 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:58.718324 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="124f8ebf-36d0-469a-b3b3-c932743898c9" containerName="util" Jul 12 18:24:58.718471 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:58.718330 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6d5bf1a-f136-45cf-9c46-1d4cad723409" containerName="extract" Jul 12 18:24:58.718471 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:58.718335 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6d5bf1a-f136-45cf-9c46-1d4cad723409" containerName="extract" Jul 12 18:24:58.718471 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:58.718345 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6d5bf1a-f136-45cf-9c46-1d4cad723409" containerName="pull" Jul 12 18:24:58.718471 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:58.718350 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6d5bf1a-f136-45cf-9c46-1d4cad723409" containerName="pull" Jul 12 18:24:58.718471 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:58.718358 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3e07319-173b-4c5c-85ef-3af760954600" containerName="extract" Jul 12 18:24:58.718471 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:58.718366 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3e07319-173b-4c5c-85ef-3af760954600" containerName="extract" Jul 12 18:24:58.718471 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:58.718375 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b65d5ed-ac9a-48bc-8aef-3c608afe4f79" containerName="extract" Jul 12 18:24:58.718471 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:58.718379 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b65d5ed-ac9a-48bc-8aef-3c608afe4f79" containerName="extract" Jul 12 18:24:58.718471 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:58.718386 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3e07319-173b-4c5c-85ef-3af760954600" containerName="pull" Jul 12 18:24:58.718471 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:58.718391 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3e07319-173b-4c5c-85ef-3af760954600" containerName="pull" Jul 12 18:24:58.718471 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:58.718398 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b65d5ed-ac9a-48bc-8aef-3c608afe4f79" containerName="pull" Jul 12 18:24:58.718471 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:58.718403 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b65d5ed-ac9a-48bc-8aef-3c608afe4f79" containerName="pull" Jul 12 18:24:58.718471 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:58.718458 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b65d5ed-ac9a-48bc-8aef-3c608afe4f79" containerName="extract" Jul 12 18:24:58.718471 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:58.718467 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6d5bf1a-f136-45cf-9c46-1d4cad723409" containerName="extract" Jul 12 18:24:58.718471 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:58.718474 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="124f8ebf-36d0-469a-b3b3-c932743898c9" containerName="extract" Jul 12 18:24:58.718471 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:58.718479 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3e07319-173b-4c5c-85ef-3af760954600" containerName="extract" Jul 12 18:24:58.721849 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:58.721830 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-747d58bd59-b7nw9" Jul 12 18:24:58.731916 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:58.731887 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-747d58bd59-b7nw9"] Jul 12 18:24:58.853221 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:58.853184 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/96ab96da-ce89-4d82-91ef-46bebade2984-console-oauth-config\") pod \"console-747d58bd59-b7nw9\" (UID: \"96ab96da-ce89-4d82-91ef-46bebade2984\") " pod="openshift-console/console-747d58bd59-b7nw9" Jul 12 18:24:58.853421 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:58.853245 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/96ab96da-ce89-4d82-91ef-46bebade2984-console-config\") pod \"console-747d58bd59-b7nw9\" (UID: \"96ab96da-ce89-4d82-91ef-46bebade2984\") " pod="openshift-console/console-747d58bd59-b7nw9" Jul 12 18:24:58.853421 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:58.853346 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mklg\" (UniqueName: \"kubernetes.io/projected/96ab96da-ce89-4d82-91ef-46bebade2984-kube-api-access-7mklg\") pod \"console-747d58bd59-b7nw9\" (UID: \"96ab96da-ce89-4d82-91ef-46bebade2984\") " pod="openshift-console/console-747d58bd59-b7nw9" Jul 12 18:24:58.853421 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:58.853404 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/96ab96da-ce89-4d82-91ef-46bebade2984-console-serving-cert\") pod \"console-747d58bd59-b7nw9\" (UID: \"96ab96da-ce89-4d82-91ef-46bebade2984\") " pod="openshift-console/console-747d58bd59-b7nw9" Jul 12 18:24:58.853571 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:58.853427 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/96ab96da-ce89-4d82-91ef-46bebade2984-service-ca\") pod \"console-747d58bd59-b7nw9\" (UID: \"96ab96da-ce89-4d82-91ef-46bebade2984\") " pod="openshift-console/console-747d58bd59-b7nw9" Jul 12 18:24:58.853571 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:58.853459 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/96ab96da-ce89-4d82-91ef-46bebade2984-trusted-ca-bundle\") pod \"console-747d58bd59-b7nw9\" (UID: \"96ab96da-ce89-4d82-91ef-46bebade2984\") " pod="openshift-console/console-747d58bd59-b7nw9" Jul 12 18:24:58.853571 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:58.853482 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/96ab96da-ce89-4d82-91ef-46bebade2984-oauth-serving-cert\") pod \"console-747d58bd59-b7nw9\" (UID: \"96ab96da-ce89-4d82-91ef-46bebade2984\") " pod="openshift-console/console-747d58bd59-b7nw9" Jul 12 18:24:58.954907 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:58.954868 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/96ab96da-ce89-4d82-91ef-46bebade2984-console-serving-cert\") pod \"console-747d58bd59-b7nw9\" (UID: \"96ab96da-ce89-4d82-91ef-46bebade2984\") " pod="openshift-console/console-747d58bd59-b7nw9" Jul 12 18:24:58.954907 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:58.954908 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/96ab96da-ce89-4d82-91ef-46bebade2984-service-ca\") pod \"console-747d58bd59-b7nw9\" (UID: \"96ab96da-ce89-4d82-91ef-46bebade2984\") " pod="openshift-console/console-747d58bd59-b7nw9" Jul 12 18:24:58.955160 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:58.954939 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/96ab96da-ce89-4d82-91ef-46bebade2984-trusted-ca-bundle\") pod \"console-747d58bd59-b7nw9\" (UID: \"96ab96da-ce89-4d82-91ef-46bebade2984\") " pod="openshift-console/console-747d58bd59-b7nw9" Jul 12 18:24:58.955160 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:58.954954 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/96ab96da-ce89-4d82-91ef-46bebade2984-oauth-serving-cert\") pod \"console-747d58bd59-b7nw9\" (UID: \"96ab96da-ce89-4d82-91ef-46bebade2984\") " pod="openshift-console/console-747d58bd59-b7nw9" Jul 12 18:24:58.955160 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:58.955007 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/96ab96da-ce89-4d82-91ef-46bebade2984-console-oauth-config\") pod \"console-747d58bd59-b7nw9\" (UID: \"96ab96da-ce89-4d82-91ef-46bebade2984\") " pod="openshift-console/console-747d58bd59-b7nw9" Jul 12 18:24:58.955160 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:58.955081 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/96ab96da-ce89-4d82-91ef-46bebade2984-console-config\") pod \"console-747d58bd59-b7nw9\" (UID: \"96ab96da-ce89-4d82-91ef-46bebade2984\") " pod="openshift-console/console-747d58bd59-b7nw9" Jul 12 18:24:58.955160 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:58.955152 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mklg\" (UniqueName: \"kubernetes.io/projected/96ab96da-ce89-4d82-91ef-46bebade2984-kube-api-access-7mklg\") pod \"console-747d58bd59-b7nw9\" (UID: \"96ab96da-ce89-4d82-91ef-46bebade2984\") " pod="openshift-console/console-747d58bd59-b7nw9" Jul 12 18:24:58.955781 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:58.955754 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/96ab96da-ce89-4d82-91ef-46bebade2984-oauth-serving-cert\") pod \"console-747d58bd59-b7nw9\" (UID: \"96ab96da-ce89-4d82-91ef-46bebade2984\") " pod="openshift-console/console-747d58bd59-b7nw9" Jul 12 18:24:58.955918 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:58.955843 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/96ab96da-ce89-4d82-91ef-46bebade2984-console-config\") pod \"console-747d58bd59-b7nw9\" (UID: \"96ab96da-ce89-4d82-91ef-46bebade2984\") " pod="openshift-console/console-747d58bd59-b7nw9" Jul 12 18:24:58.955997 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:58.955945 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/96ab96da-ce89-4d82-91ef-46bebade2984-service-ca\") pod \"console-747d58bd59-b7nw9\" (UID: \"96ab96da-ce89-4d82-91ef-46bebade2984\") " pod="openshift-console/console-747d58bd59-b7nw9" Jul 12 18:24:58.955997 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:58.955945 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/96ab96da-ce89-4d82-91ef-46bebade2984-trusted-ca-bundle\") pod \"console-747d58bd59-b7nw9\" (UID: \"96ab96da-ce89-4d82-91ef-46bebade2984\") " pod="openshift-console/console-747d58bd59-b7nw9" Jul 12 18:24:58.957416 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:58.957395 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/96ab96da-ce89-4d82-91ef-46bebade2984-console-oauth-config\") pod \"console-747d58bd59-b7nw9\" (UID: \"96ab96da-ce89-4d82-91ef-46bebade2984\") " pod="openshift-console/console-747d58bd59-b7nw9" Jul 12 18:24:58.957497 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:58.957481 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/96ab96da-ce89-4d82-91ef-46bebade2984-console-serving-cert\") pod \"console-747d58bd59-b7nw9\" (UID: \"96ab96da-ce89-4d82-91ef-46bebade2984\") " pod="openshift-console/console-747d58bd59-b7nw9" Jul 12 18:24:58.966393 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:58.966374 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mklg\" (UniqueName: \"kubernetes.io/projected/96ab96da-ce89-4d82-91ef-46bebade2984-kube-api-access-7mklg\") pod \"console-747d58bd59-b7nw9\" (UID: \"96ab96da-ce89-4d82-91ef-46bebade2984\") " pod="openshift-console/console-747d58bd59-b7nw9" Jul 12 18:24:59.031958 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:59.031865 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-747d58bd59-b7nw9" Jul 12 18:24:59.180880 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:59.180734 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-747d58bd59-b7nw9"] Jul 12 18:24:59.183158 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:24:59.183130 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96ab96da_ce89_4d82_91ef_46bebade2984.slice/crio-93907de9fcd7d6f316ad5dc3f6d0e427ea1a6a5219ca1864d8ea7394e6c3bb75 WatchSource:0}: Error finding container 93907de9fcd7d6f316ad5dc3f6d0e427ea1a6a5219ca1864d8ea7394e6c3bb75: Status 404 returned error can't find the container with id 93907de9fcd7d6f316ad5dc3f6d0e427ea1a6a5219ca1864d8ea7394e6c3bb75 Jul 12 18:24:59.273120 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:59.273086 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-747d58bd59-b7nw9" event={"ID":"96ab96da-ce89-4d82-91ef-46bebade2984","Type":"ContainerStarted","Data":"02d76f2aa244ee522178bf1dabd78276728fdb95dc33892cf8f0895c348e153c"} Jul 12 18:24:59.273267 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:59.273142 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-747d58bd59-b7nw9" event={"ID":"96ab96da-ce89-4d82-91ef-46bebade2984","Type":"ContainerStarted","Data":"93907de9fcd7d6f316ad5dc3f6d0e427ea1a6a5219ca1864d8ea7394e6c3bb75"} Jul 12 18:24:59.296113 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:24:59.296002 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-747d58bd59-b7nw9" podStartSLOduration=1.295986772 podStartE2EDuration="1.295986772s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:24:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:24:59.294919854 +0000 UTC m=+472.421824658" watchObservedRunningTime="2026-07-12 18:24:59.295986772 +0000 UTC m=+472.422891578" Jul 12 18:25:09.032495 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:09.032462 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-747d58bd59-b7nw9" Jul 12 18:25:09.032943 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:09.032520 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-747d58bd59-b7nw9" Jul 12 18:25:09.037343 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:09.037323 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-747d58bd59-b7nw9" Jul 12 18:25:09.329523 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:09.329438 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-747d58bd59-b7nw9" Jul 12 18:25:09.388147 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:09.388113 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-686b8f579b-j6tn5"] Jul 12 18:25:34.408954 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:34.408889 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-686b8f579b-j6tn5" podUID="cd14dd7d-4812-4fb7-9a42-067285985d2b" containerName="console" containerID="cri-o://deecdda4012a9aa371f32c8ab030bfee758061b2c283100296cf591fc5f6d0d2" gracePeriod=15 Jul 12 18:25:34.660848 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:34.660795 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-686b8f579b-j6tn5" Jul 12 18:25:34.773520 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:34.773481 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cd14dd7d-4812-4fb7-9a42-067285985d2b-console-config\") pod \"cd14dd7d-4812-4fb7-9a42-067285985d2b\" (UID: \"cd14dd7d-4812-4fb7-9a42-067285985d2b\") " Jul 12 18:25:34.773775 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:34.773532 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cd14dd7d-4812-4fb7-9a42-067285985d2b-console-serving-cert\") pod \"cd14dd7d-4812-4fb7-9a42-067285985d2b\" (UID: \"cd14dd7d-4812-4fb7-9a42-067285985d2b\") " Jul 12 18:25:34.773775 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:34.773566 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cd14dd7d-4812-4fb7-9a42-067285985d2b-console-oauth-config\") pod \"cd14dd7d-4812-4fb7-9a42-067285985d2b\" (UID: \"cd14dd7d-4812-4fb7-9a42-067285985d2b\") " Jul 12 18:25:34.773775 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:34.773753 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cd14dd7d-4812-4fb7-9a42-067285985d2b-service-ca\") pod \"cd14dd7d-4812-4fb7-9a42-067285985d2b\" (UID: \"cd14dd7d-4812-4fb7-9a42-067285985d2b\") " Jul 12 18:25:34.773954 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:34.773843 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd14dd7d-4812-4fb7-9a42-067285985d2b-trusted-ca-bundle\") pod \"cd14dd7d-4812-4fb7-9a42-067285985d2b\" (UID: \"cd14dd7d-4812-4fb7-9a42-067285985d2b\") " Jul 12 18:25:34.773954 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:34.773874 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cd14dd7d-4812-4fb7-9a42-067285985d2b-oauth-serving-cert\") pod \"cd14dd7d-4812-4fb7-9a42-067285985d2b\" (UID: \"cd14dd7d-4812-4fb7-9a42-067285985d2b\") " Jul 12 18:25:34.774074 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:34.774052 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m2llb\" (UniqueName: \"kubernetes.io/projected/cd14dd7d-4812-4fb7-9a42-067285985d2b-kube-api-access-m2llb\") pod \"cd14dd7d-4812-4fb7-9a42-067285985d2b\" (UID: \"cd14dd7d-4812-4fb7-9a42-067285985d2b\") " Jul 12 18:25:34.774176 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:34.774150 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd14dd7d-4812-4fb7-9a42-067285985d2b-console-config" (OuterVolumeSpecName: "console-config") pod "cd14dd7d-4812-4fb7-9a42-067285985d2b" (UID: "cd14dd7d-4812-4fb7-9a42-067285985d2b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:25:34.774239 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:34.774219 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd14dd7d-4812-4fb7-9a42-067285985d2b-service-ca" (OuterVolumeSpecName: "service-ca") pod "cd14dd7d-4812-4fb7-9a42-067285985d2b" (UID: "cd14dd7d-4812-4fb7-9a42-067285985d2b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:25:34.774239 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:34.774227 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd14dd7d-4812-4fb7-9a42-067285985d2b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "cd14dd7d-4812-4fb7-9a42-067285985d2b" (UID: "cd14dd7d-4812-4fb7-9a42-067285985d2b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:25:34.774364 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:34.774250 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd14dd7d-4812-4fb7-9a42-067285985d2b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "cd14dd7d-4812-4fb7-9a42-067285985d2b" (UID: "cd14dd7d-4812-4fb7-9a42-067285985d2b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:25:34.774462 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:34.774442 2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cd14dd7d-4812-4fb7-9a42-067285985d2b-service-ca\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:25:34.774530 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:34.774470 2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cd14dd7d-4812-4fb7-9a42-067285985d2b-trusted-ca-bundle\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:25:34.774530 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:34.774484 2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cd14dd7d-4812-4fb7-9a42-067285985d2b-oauth-serving-cert\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:25:34.774530 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:34.774495 2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cd14dd7d-4812-4fb7-9a42-067285985d2b-console-config\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:25:34.775971 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:34.775938 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd14dd7d-4812-4fb7-9a42-067285985d2b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "cd14dd7d-4812-4fb7-9a42-067285985d2b" (UID: "cd14dd7d-4812-4fb7-9a42-067285985d2b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 18:25:34.775971 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:34.775956 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd14dd7d-4812-4fb7-9a42-067285985d2b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "cd14dd7d-4812-4fb7-9a42-067285985d2b" (UID: "cd14dd7d-4812-4fb7-9a42-067285985d2b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 18:25:34.776138 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:34.776120 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd14dd7d-4812-4fb7-9a42-067285985d2b-kube-api-access-m2llb" (OuterVolumeSpecName: "kube-api-access-m2llb") pod "cd14dd7d-4812-4fb7-9a42-067285985d2b" (UID: "cd14dd7d-4812-4fb7-9a42-067285985d2b"). InnerVolumeSpecName "kube-api-access-m2llb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:25:34.875525 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:34.875490 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m2llb\" (UniqueName: \"kubernetes.io/projected/cd14dd7d-4812-4fb7-9a42-067285985d2b-kube-api-access-m2llb\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:25:34.875525 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:34.875522 2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cd14dd7d-4812-4fb7-9a42-067285985d2b-console-serving-cert\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:25:34.875525 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:34.875537 2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cd14dd7d-4812-4fb7-9a42-067285985d2b-console-oauth-config\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:25:35.458181 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:35.458143 2577 generic.go:356] "Generic (PLEG): container finished" podID="cd14dd7d-4812-4fb7-9a42-067285985d2b" containerID="deecdda4012a9aa371f32c8ab030bfee758061b2c283100296cf591fc5f6d0d2" exitCode=0 Jul 12 18:25:35.458615 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:35.458189 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-686b8f579b-j6tn5" Jul 12 18:25:35.458615 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:35.458197 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-686b8f579b-j6tn5" event={"ID":"cd14dd7d-4812-4fb7-9a42-067285985d2b","Type":"ContainerDied","Data":"deecdda4012a9aa371f32c8ab030bfee758061b2c283100296cf591fc5f6d0d2"} Jul 12 18:25:35.458615 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:35.458245 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-686b8f579b-j6tn5" event={"ID":"cd14dd7d-4812-4fb7-9a42-067285985d2b","Type":"ContainerDied","Data":"1c29013701373a8575949c306a7e95f318dd8be66b6044369f8b1d51dc8708ce"} Jul 12 18:25:35.458615 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:35.458272 2577 scope.go:117] "RemoveContainer" containerID="deecdda4012a9aa371f32c8ab030bfee758061b2c283100296cf591fc5f6d0d2" Jul 12 18:25:35.466986 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:35.466965 2577 scope.go:117] "RemoveContainer" containerID="deecdda4012a9aa371f32c8ab030bfee758061b2c283100296cf591fc5f6d0d2" Jul 12 18:25:35.467262 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:25:35.467240 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"deecdda4012a9aa371f32c8ab030bfee758061b2c283100296cf591fc5f6d0d2\": container with ID starting with deecdda4012a9aa371f32c8ab030bfee758061b2c283100296cf591fc5f6d0d2 not found: ID does not exist" containerID="deecdda4012a9aa371f32c8ab030bfee758061b2c283100296cf591fc5f6d0d2" Jul 12 18:25:35.467316 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:35.467273 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"deecdda4012a9aa371f32c8ab030bfee758061b2c283100296cf591fc5f6d0d2"} err="failed to get container status \"deecdda4012a9aa371f32c8ab030bfee758061b2c283100296cf591fc5f6d0d2\": rpc error: code = NotFound desc = could not find container \"deecdda4012a9aa371f32c8ab030bfee758061b2c283100296cf591fc5f6d0d2\": container with ID starting with deecdda4012a9aa371f32c8ab030bfee758061b2c283100296cf591fc5f6d0d2 not found: ID does not exist" Jul 12 18:25:35.475287 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:35.475266 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-686b8f579b-j6tn5"] Jul 12 18:25:35.478996 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:35.478974 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-686b8f579b-j6tn5"] Jul 12 18:25:37.076213 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:37.076181 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd14dd7d-4812-4fb7-9a42-067285985d2b" path="/var/lib/kubelet/pods/cd14dd7d-4812-4fb7-9a42-067285985d2b/volumes" Jul 12 18:25:58.227242 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:58.227207 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:25:58.227825 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:58.227523 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd14dd7d-4812-4fb7-9a42-067285985d2b" containerName="console" Jul 12 18:25:58.227825 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:58.227537 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd14dd7d-4812-4fb7-9a42-067285985d2b" containerName="console" Jul 12 18:25:58.227825 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:58.227610 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd14dd7d-4812-4fb7-9a42-067285985d2b" containerName="console" Jul 12 18:25:58.236323 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:58.236294 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-4httc" Jul 12 18:25:58.238886 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:58.238866 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 12 18:25:58.240754 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:58.240725 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-crn2p\"" Jul 12 18:25:58.241426 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:58.241405 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 12 18:25:58.241602 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:58.241439 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 12 18:25:58.244271 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:58.244248 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:25:58.282881 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:58.282852 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:25:58.377880 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:58.377847 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/47a4888e-37d0-47e2-84c6-d8b8b02db23a-config-file\") pod \"limitador-limitador-54d6d988d5-4httc\" (UID: \"47a4888e-37d0-47e2-84c6-d8b8b02db23a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-4httc" Jul 12 18:25:58.378062 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:58.377921 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6g5x5\" (UniqueName: \"kubernetes.io/projected/47a4888e-37d0-47e2-84c6-d8b8b02db23a-kube-api-access-6g5x5\") pod \"limitador-limitador-54d6d988d5-4httc\" (UID: \"47a4888e-37d0-47e2-84c6-d8b8b02db23a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-4httc" Jul 12 18:25:58.478618 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:58.478528 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6g5x5\" (UniqueName: \"kubernetes.io/projected/47a4888e-37d0-47e2-84c6-d8b8b02db23a-kube-api-access-6g5x5\") pod \"limitador-limitador-54d6d988d5-4httc\" (UID: \"47a4888e-37d0-47e2-84c6-d8b8b02db23a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-4httc" Jul 12 18:25:58.478618 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:58.478591 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/47a4888e-37d0-47e2-84c6-d8b8b02db23a-config-file\") pod \"limitador-limitador-54d6d988d5-4httc\" (UID: \"47a4888e-37d0-47e2-84c6-d8b8b02db23a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-4httc" Jul 12 18:25:58.479181 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:58.479163 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/47a4888e-37d0-47e2-84c6-d8b8b02db23a-config-file\") pod \"limitador-limitador-54d6d988d5-4httc\" (UID: \"47a4888e-37d0-47e2-84c6-d8b8b02db23a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-4httc" Jul 12 18:25:58.486343 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:58.486320 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6g5x5\" (UniqueName: \"kubernetes.io/projected/47a4888e-37d0-47e2-84c6-d8b8b02db23a-kube-api-access-6g5x5\") pod \"limitador-limitador-54d6d988d5-4httc\" (UID: \"47a4888e-37d0-47e2-84c6-d8b8b02db23a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-4httc" Jul 12 18:25:58.548308 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:58.548276 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-4httc" Jul 12 18:25:58.679621 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:58.679589 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:25:58.681982 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:25:58.681957 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47a4888e_37d0_47e2_84c6_d8b8b02db23a.slice/crio-db28a26d0c5bae8aec9e04e4b55686d3f26735eb63c4431efb5c8ae33390c5f3 WatchSource:0}: Error finding container db28a26d0c5bae8aec9e04e4b55686d3f26735eb63c4431efb5c8ae33390c5f3: Status 404 returned error can't find the container with id db28a26d0c5bae8aec9e04e4b55686d3f26735eb63c4431efb5c8ae33390c5f3 Jul 12 18:25:59.589448 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:25:59.589360 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-4httc" event={"ID":"47a4888e-37d0-47e2-84c6-d8b8b02db23a","Type":"ContainerStarted","Data":"db28a26d0c5bae8aec9e04e4b55686d3f26735eb63c4431efb5c8ae33390c5f3"} Jul 12 18:26:01.603049 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:26:01.602966 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-4httc" event={"ID":"47a4888e-37d0-47e2-84c6-d8b8b02db23a","Type":"ContainerStarted","Data":"4a42de48bb6a8e616d9595d6f754bd9fbfa429a64ba5478bbb2c33f1d9793924"} Jul 12 18:26:01.603399 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:26:01.603101 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-4httc" Jul 12 18:26:01.621746 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:26:01.621690 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-4httc" podStartSLOduration=1.077314254 podStartE2EDuration="3.621674542s" totalImagesPullingTime="2.544360288s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:25:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:26:01.619587379 +0000 UTC m=+534.746492183" watchObservedRunningTime="2026-07-12 18:26:01.621674542 +0000 UTC m=+534.748579348" Jul 12 18:26:12.609937 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:26:12.609903 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-4httc" Jul 12 18:27:09.855680 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:27:09.855557 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-spzqd_ed115460-b220-476d-b9d5-65f95ec3d059/ovn-acl-logging/0.log" Jul 12 18:27:09.859513 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:27:09.857256 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-spzqd_ed115460-b220-476d-b9d5-65f95ec3d059/ovn-acl-logging/0.log" Jul 12 18:27:11.133593 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:27:11.133561 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:27:29.158875 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:27:29.158790 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:27:41.036413 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:27:41.036379 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:27:55.026399 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:27:55.026364 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:28:07.493840 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:28:07.493805 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:28:13.383681 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:28:13.383649 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-lwcn2"] Jul 12 18:28:13.387033 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:28:13.387013 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lwcn2" Jul 12 18:28:13.390755 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:28:13.390732 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 12 18:28:13.390755 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:28:13.390744 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 12 18:28:13.390935 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:28:13.390834 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-zf69s\"" Jul 12 18:28:13.390935 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:28:13.390865 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 12 18:28:13.397042 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:28:13.397020 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-lwcn2"] Jul 12 18:28:13.437398 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:28:13.437364 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/732b3a95-f015-4cda-b6c6-56cf546be766-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lwcn2\" (UID: \"732b3a95-f015-4cda-b6c6-56cf546be766\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lwcn2" Jul 12 18:28:13.437398 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:28:13.437405 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/732b3a95-f015-4cda-b6c6-56cf546be766-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lwcn2\" (UID: \"732b3a95-f015-4cda-b6c6-56cf546be766\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lwcn2" Jul 12 18:28:13.437619 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:28:13.437490 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/732b3a95-f015-4cda-b6c6-56cf546be766-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lwcn2\" (UID: \"732b3a95-f015-4cda-b6c6-56cf546be766\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lwcn2" Jul 12 18:28:13.437619 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:28:13.437518 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/732b3a95-f015-4cda-b6c6-56cf546be766-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lwcn2\" (UID: \"732b3a95-f015-4cda-b6c6-56cf546be766\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lwcn2" Jul 12 18:28:13.437619 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:28:13.437617 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/732b3a95-f015-4cda-b6c6-56cf546be766-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lwcn2\" (UID: \"732b3a95-f015-4cda-b6c6-56cf546be766\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lwcn2" Jul 12 18:28:13.437770 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:28:13.437659 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwstx\" (UniqueName: \"kubernetes.io/projected/732b3a95-f015-4cda-b6c6-56cf546be766-kube-api-access-bwstx\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lwcn2\" (UID: \"732b3a95-f015-4cda-b6c6-56cf546be766\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lwcn2" Jul 12 18:28:13.538770 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:28:13.538732 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/732b3a95-f015-4cda-b6c6-56cf546be766-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lwcn2\" (UID: \"732b3a95-f015-4cda-b6c6-56cf546be766\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lwcn2" Jul 12 18:28:13.538967 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:28:13.538813 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/732b3a95-f015-4cda-b6c6-56cf546be766-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lwcn2\" (UID: \"732b3a95-f015-4cda-b6c6-56cf546be766\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lwcn2" Jul 12 18:28:13.538967 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:28:13.538847 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/732b3a95-f015-4cda-b6c6-56cf546be766-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lwcn2\" (UID: \"732b3a95-f015-4cda-b6c6-56cf546be766\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lwcn2" Jul 12 18:28:13.538967 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:28:13.538886 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/732b3a95-f015-4cda-b6c6-56cf546be766-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lwcn2\" (UID: \"732b3a95-f015-4cda-b6c6-56cf546be766\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lwcn2" Jul 12 18:28:13.538967 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:28:13.538905 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwstx\" (UniqueName: \"kubernetes.io/projected/732b3a95-f015-4cda-b6c6-56cf546be766-kube-api-access-bwstx\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lwcn2\" (UID: \"732b3a95-f015-4cda-b6c6-56cf546be766\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lwcn2" Jul 12 18:28:13.538967 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:28:13.538948 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/732b3a95-f015-4cda-b6c6-56cf546be766-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lwcn2\" (UID: \"732b3a95-f015-4cda-b6c6-56cf546be766\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lwcn2" Jul 12 18:28:13.539361 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:28:13.539322 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/732b3a95-f015-4cda-b6c6-56cf546be766-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lwcn2\" (UID: \"732b3a95-f015-4cda-b6c6-56cf546be766\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lwcn2" Jul 12 18:28:13.539468 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:28:13.539334 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/732b3a95-f015-4cda-b6c6-56cf546be766-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lwcn2\" (UID: \"732b3a95-f015-4cda-b6c6-56cf546be766\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lwcn2" Jul 12 18:28:13.539468 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:28:13.539383 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/732b3a95-f015-4cda-b6c6-56cf546be766-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lwcn2\" (UID: \"732b3a95-f015-4cda-b6c6-56cf546be766\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lwcn2" Jul 12 18:28:13.541209 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:28:13.541182 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/732b3a95-f015-4cda-b6c6-56cf546be766-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lwcn2\" (UID: \"732b3a95-f015-4cda-b6c6-56cf546be766\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lwcn2" Jul 12 18:28:13.541311 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:28:13.541254 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/732b3a95-f015-4cda-b6c6-56cf546be766-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lwcn2\" (UID: \"732b3a95-f015-4cda-b6c6-56cf546be766\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lwcn2" Jul 12 18:28:13.547230 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:28:13.547211 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwstx\" (UniqueName: \"kubernetes.io/projected/732b3a95-f015-4cda-b6c6-56cf546be766-kube-api-access-bwstx\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-lwcn2\" (UID: \"732b3a95-f015-4cda-b6c6-56cf546be766\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lwcn2" Jul 12 18:28:13.698397 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:28:13.698365 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lwcn2" Jul 12 18:28:13.835218 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:28:13.835185 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-lwcn2"] Jul 12 18:28:13.841455 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:28:13.841420 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod732b3a95_f015_4cda_b6c6_56cf546be766.slice/crio-8e16b88cbe09b32e22a0b595b5f4671d8c1bee3266891ceb42483dfa2f3bfaa4 WatchSource:0}: Error finding container 8e16b88cbe09b32e22a0b595b5f4671d8c1bee3266891ceb42483dfa2f3bfaa4: Status 404 returned error can't find the container with id 8e16b88cbe09b32e22a0b595b5f4671d8c1bee3266891ceb42483dfa2f3bfaa4 Jul 12 18:28:13.842817 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:28:13.842797 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 18:28:14.265511 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:28:14.265470 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lwcn2" event={"ID":"732b3a95-f015-4cda-b6c6-56cf546be766","Type":"ContainerStarted","Data":"8e16b88cbe09b32e22a0b595b5f4671d8c1bee3266891ceb42483dfa2f3bfaa4"} Jul 12 18:28:15.274404 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:28:15.274369 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lwcn2" event={"ID":"732b3a95-f015-4cda-b6c6-56cf546be766","Type":"ContainerStarted","Data":"1019b70286807f7da351fa9f8437a947ae66fa54b831fb758549abb24c54fef1"} Jul 12 18:28:15.296336 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:28:15.296288 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lwcn2" podStartSLOduration=1.311392788 podStartE2EDuration="2.296270316s" totalImagesPullingTime="984.877528ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:28:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:28:15.294217916 +0000 UTC m=+668.421122723" watchObservedRunningTime="2026-07-12 18:28:15.296270316 +0000 UTC m=+668.423175120" Jul 12 18:28:16.084714 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:28:16.084677 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:28:23.084989 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:28:23.084954 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:28:23.699647 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:28:23.699588 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lwcn2" Jul 12 18:28:23.699905 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:28:23.699676 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lwcn2" Jul 12 18:28:23.712109 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:28:23.712078 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lwcn2" Jul 12 18:28:24.337357 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:28:24.337331 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-lwcn2" Jul 12 18:29:54.189986 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:29:54.189949 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:30:04.483908 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:30:04.483874 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:30:14.589337 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:30:14.589295 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:30:23.784357 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:30:23.784320 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:30:33.879942 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:30:33.879866 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:31:01.582162 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:31:01.582132 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:31:16.178398 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:31:16.178364 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:31:45.781529 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:31:45.781493 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:31:55.477187 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:31:55.477148 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:32:04.483484 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:32:04.483451 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:32:09.888080 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:32:09.887973 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-spzqd_ed115460-b220-476d-b9d5-65f95ec3d059/ovn-acl-logging/0.log" Jul 12 18:32:09.895874 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:32:09.893598 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-spzqd_ed115460-b220-476d-b9d5-65f95ec3d059/ovn-acl-logging/0.log" Jul 12 18:32:16.077599 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:32:16.077564 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:32:27.787435 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:32:27.787400 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:32:37.591821 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:32:37.591785 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:32:47.976154 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:32:47.976117 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:32:58.075775 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:32:58.075739 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:33:19.594793 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:33:19.594755 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:33:31.792346 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:33:31.792250 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:33:54.891509 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:33:54.891469 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:34:05.585055 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:34:05.585017 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:34:20.193267 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:34:20.193232 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:34:29.828584 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:34:29.828551 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:34:40.988612 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:34:40.988574 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:34:50.592124 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:34:50.592085 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:35:00.497247 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:35:00.497161 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:36:46.987789 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:36:46.987697 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:36:59.786903 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:36:59.786866 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:37:09.179710 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:37:09.179675 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:37:09.925423 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:37:09.925315 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-spzqd_ed115460-b220-476d-b9d5-65f95ec3d059/ovn-acl-logging/0.log" Jul 12 18:37:09.930042 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:37:09.930019 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-spzqd_ed115460-b220-476d-b9d5-65f95ec3d059/ovn-acl-logging/0.log" Jul 12 18:37:19.488056 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:37:19.488015 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:37:31.986652 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:37:31.986605 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:37:42.002044 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:37:42.002012 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:37:50.188521 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:37:50.188487 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:38:01.292891 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:38:01.292851 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:38:12.387164 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:38:12.387133 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:38:20.984085 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:38:20.984045 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:38:31.886770 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:38:31.886734 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:38:56.579019 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:38:56.578975 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:39:03.882237 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:39:03.882196 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:39:13.980001 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:39:13.979961 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:39:24.082598 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:39:24.082560 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:39:33.088109 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:39:33.088031 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:39:45.478278 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:39:45.478242 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:39:56.580677 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:39:56.580640 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:40:06.084234 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:40:06.084202 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:40:13.880396 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:40:13.880355 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:40:24.983445 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:40:24.983414 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:40:34.783115 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:40:34.783075 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:40:48.185861 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:40:48.182785 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:40:57.580872 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:40:57.580838 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:41:07.490067 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:41:07.490029 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:41:20.083643 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:41:20.083589 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:41:32.084484 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:41:32.084448 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:41:39.685371 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:41:39.685334 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:41:52.785586 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:41:52.785556 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:42:04.588170 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:42:04.588135 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:42:09.958032 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:42:09.958000 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-spzqd_ed115460-b220-476d-b9d5-65f95ec3d059/ovn-acl-logging/0.log" Jul 12 18:42:09.964982 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:42:09.964959 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-spzqd_ed115460-b220-476d-b9d5-65f95ec3d059/ovn-acl-logging/0.log" Jul 12 18:42:17.893178 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:42:17.893143 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:42:31.476989 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:42:31.476914 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:42:40.283664 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:42:40.283619 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:42:50.092696 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:42:50.092659 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:43:02.428455 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:02.428422 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz"] Jul 12 18:43:02.432059 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:02.432037 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz" Jul 12 18:43:02.435900 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:02.435601 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-b7e51ce2-openshift-default-dockercfg-c49t5\"" Jul 12 18:43:02.448863 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:02.448829 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz"] Jul 12 18:43:02.534021 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:02.533934 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c-workload-certs\") pod \"e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz\" (UID: \"045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c\") " pod="openshift-ingress/e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz" Jul 12 18:43:02.534216 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:02.534105 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c-istio-envoy\") pod \"e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz\" (UID: \"045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c\") " pod="openshift-ingress/e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz" Jul 12 18:43:02.534216 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:02.534166 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c-istio-token\") pod \"e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz\" (UID: \"045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c\") " pod="openshift-ingress/e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz" Jul 12 18:43:02.534341 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:02.534217 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c-istio-podinfo\") pod \"e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz\" (UID: \"045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c\") " pod="openshift-ingress/e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz" Jul 12 18:43:02.534400 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:02.534361 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c-credential-socket\") pod \"e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz\" (UID: \"045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c\") " pod="openshift-ingress/e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz" Jul 12 18:43:02.534459 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:02.534402 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c-istio-data\") pod \"e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz\" (UID: \"045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c\") " pod="openshift-ingress/e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz" Jul 12 18:43:02.534459 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:02.534444 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pv8g\" (UniqueName: \"kubernetes.io/projected/045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c-kube-api-access-7pv8g\") pod \"e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz\" (UID: \"045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c\") " pod="openshift-ingress/e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz" Jul 12 18:43:02.534578 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:02.534546 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c-istiod-ca-cert\") pod \"e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz\" (UID: \"045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c\") " pod="openshift-ingress/e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz" Jul 12 18:43:02.534669 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:02.534602 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c-workload-socket\") pod \"e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz\" (UID: \"045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c\") " pod="openshift-ingress/e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz" Jul 12 18:43:02.635758 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:02.635723 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c-workload-certs\") pod \"e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz\" (UID: \"045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c\") " pod="openshift-ingress/e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz" Jul 12 18:43:02.635758 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:02.635757 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c-istio-envoy\") pod \"e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz\" (UID: \"045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c\") " pod="openshift-ingress/e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz" Jul 12 18:43:02.635984 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:02.635802 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c-istio-token\") pod \"e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz\" (UID: \"045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c\") " pod="openshift-ingress/e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz" Jul 12 18:43:02.635984 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:02.635840 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c-istio-podinfo\") pod \"e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz\" (UID: \"045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c\") " pod="openshift-ingress/e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz" Jul 12 18:43:02.635984 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:02.635909 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c-credential-socket\") pod \"e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz\" (UID: \"045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c\") " pod="openshift-ingress/e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz" Jul 12 18:43:02.635984 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:02.635932 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c-istio-data\") pod \"e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz\" (UID: \"045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c\") " pod="openshift-ingress/e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz" Jul 12 18:43:02.635984 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:02.635973 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pv8g\" (UniqueName: \"kubernetes.io/projected/045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c-kube-api-access-7pv8g\") pod \"e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz\" (UID: \"045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c\") " pod="openshift-ingress/e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz" Jul 12 18:43:02.636803 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:02.636002 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c-istiod-ca-cert\") pod \"e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz\" (UID: \"045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c\") " pod="openshift-ingress/e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz" Jul 12 18:43:02.636803 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:02.636034 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c-workload-socket\") pod \"e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz\" (UID: \"045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c\") " pod="openshift-ingress/e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz" Jul 12 18:43:02.636803 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:02.636198 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c-workload-certs\") pod \"e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz\" (UID: \"045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c\") " pod="openshift-ingress/e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz" Jul 12 18:43:02.636803 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:02.636272 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c-istio-data\") pod \"e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz\" (UID: \"045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c\") " pod="openshift-ingress/e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz" Jul 12 18:43:02.636803 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:02.636320 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c-credential-socket\") pod \"e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz\" (UID: \"045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c\") " pod="openshift-ingress/e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz" Jul 12 18:43:02.636803 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:02.636369 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c-workload-socket\") pod \"e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz\" (UID: \"045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c\") " pod="openshift-ingress/e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz" Jul 12 18:43:02.636988 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:02.636824 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c-istiod-ca-cert\") pod \"e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz\" (UID: \"045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c\") " pod="openshift-ingress/e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz" Jul 12 18:43:02.638183 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:02.638166 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c-istio-envoy\") pod \"e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz\" (UID: \"045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c\") " pod="openshift-ingress/e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz" Jul 12 18:43:02.638393 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:02.638376 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c-istio-podinfo\") pod \"e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz\" (UID: \"045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c\") " pod="openshift-ingress/e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz" Jul 12 18:43:02.644316 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:02.644286 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c-istio-token\") pod \"e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz\" (UID: \"045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c\") " pod="openshift-ingress/e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz" Jul 12 18:43:02.644486 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:02.644467 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pv8g\" (UniqueName: \"kubernetes.io/projected/045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c-kube-api-access-7pv8g\") pod \"e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz\" (UID: \"045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c\") " pod="openshift-ingress/e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz" Jul 12 18:43:02.745391 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:02.745357 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz" Jul 12 18:43:02.883616 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:02.883584 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz"] Jul 12 18:43:02.886830 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:43:02.886801 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod045cabbc_2fa6_4c99_b9e7_f5ed542a2a4c.slice/crio-8f74172d967cd982ce27af348a2692dfb3468e37816ecbd50ca82ae6e6e5dd7a WatchSource:0}: Error finding container 8f74172d967cd982ce27af348a2692dfb3468e37816ecbd50ca82ae6e6e5dd7a: Status 404 returned error can't find the container with id 8f74172d967cd982ce27af348a2692dfb3468e37816ecbd50ca82ae6e6e5dd7a Jul 12 18:43:02.889128 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:02.889096 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 12 18:43:02.889227 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:02.889173 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 12 18:43:02.889227 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:02.889205 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 12 18:43:03.749793 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:03.749759 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz" event={"ID":"045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c","Type":"ContainerStarted","Data":"6c723abfef228cd97e7357419c77c3903072dce4699f0b226336d3d0deda3a7c"} Jul 12 18:43:03.750174 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:03.749810 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz" event={"ID":"045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c","Type":"ContainerStarted","Data":"8f74172d967cd982ce27af348a2692dfb3468e37816ecbd50ca82ae6e6e5dd7a"} Jul 12 18:43:03.801514 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:03.801455 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz" podStartSLOduration=1.801437254 podStartE2EDuration="1.801437254s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:43:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:43:03.800200511 +0000 UTC m=+1556.927105330" watchObservedRunningTime="2026-07-12 18:43:03.801437254 +0000 UTC m=+1556.928342058" Jul 12 18:43:04.745853 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:04.745815 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz" Jul 12 18:43:04.750757 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:04.750734 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz" Jul 12 18:43:04.757981 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:04.757957 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz" Jul 12 18:43:04.758891 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:04.758869 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz" Jul 12 18:43:08.581278 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:08.581243 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:43:14.711300 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:14.711262 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz"] Jul 12 18:43:14.711730 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:14.711469 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz" podUID="045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c" containerName="istio-proxy" containerID="cri-o://6c723abfef228cd97e7357419c77c3903072dce4699f0b226336d3d0deda3a7c" gracePeriod=30 Jul 12 18:43:16.362933 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:16.362907 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz" Jul 12 18:43:16.466648 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:16.466593 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c-istio-podinfo\") pod \"045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c\" (UID: \"045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c\") " Jul 12 18:43:16.466836 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:16.466656 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c-workload-certs\") pod \"045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c\" (UID: \"045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c\") " Jul 12 18:43:16.466836 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:16.466684 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c-credential-socket\") pod \"045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c\" (UID: \"045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c\") " Jul 12 18:43:16.466836 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:16.466713 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c-istiod-ca-cert\") pod \"045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c\" (UID: \"045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c\") " Jul 12 18:43:16.466836 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:16.466753 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7pv8g\" (UniqueName: \"kubernetes.io/projected/045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c-kube-api-access-7pv8g\") pod \"045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c\" (UID: \"045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c\") " Jul 12 18:43:16.466836 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:16.466803 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c-istio-envoy\") pod \"045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c\" (UID: \"045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c\") " Jul 12 18:43:16.467129 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:16.466861 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c-istio-data\") pod \"045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c\" (UID: \"045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c\") " Jul 12 18:43:16.467129 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:16.466922 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c-istio-token\") pod \"045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c\" (UID: \"045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c\") " Jul 12 18:43:16.467129 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:16.466952 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c-workload-socket\") pod \"045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c\" (UID: \"045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c\") " Jul 12 18:43:16.467129 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:16.467035 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c" (UID: "045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:43:16.467129 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:16.467046 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c" (UID: "045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:43:16.467385 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:16.467137 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c" (UID: "045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:43:16.467385 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:16.467339 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c" (UID: "045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:43:16.467489 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:16.467414 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c-workload-socket\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:43:16.467489 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:16.467436 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c-workload-certs\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:43:16.467489 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:16.467476 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c-credential-socket\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:43:16.467849 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:16.467491 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c-istiod-ca-cert\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:43:16.468002 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:16.467978 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c-istio-data" (OuterVolumeSpecName: "istio-data") pod "045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c" (UID: "045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:43:16.469050 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:16.469024 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c" (UID: "045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 18:43:16.469050 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:16.469041 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c" (UID: "045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:43:16.469254 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:16.469240 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c-istio-token" (OuterVolumeSpecName: "istio-token") pod "045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c" (UID: "045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:43:16.469298 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:16.469274 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c-kube-api-access-7pv8g" (OuterVolumeSpecName: "kube-api-access-7pv8g") pod "045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c" (UID: "045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c"). InnerVolumeSpecName "kube-api-access-7pv8g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:43:16.568742 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:16.568705 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c-istio-podinfo\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:43:16.568742 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:16.568735 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7pv8g\" (UniqueName: \"kubernetes.io/projected/045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c-kube-api-access-7pv8g\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:43:16.568742 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:16.568750 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c-istio-envoy\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:43:16.568965 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:16.568761 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c-istio-data\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:43:16.568965 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:16.568772 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c-istio-token\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:43:16.821042 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:16.821006 2577 generic.go:356] "Generic (PLEG): container finished" podID="045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c" containerID="6c723abfef228cd97e7357419c77c3903072dce4699f0b226336d3d0deda3a7c" exitCode=0 Jul 12 18:43:16.821219 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:16.821076 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz" Jul 12 18:43:16.821219 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:16.821089 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz" event={"ID":"045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c","Type":"ContainerDied","Data":"6c723abfef228cd97e7357419c77c3903072dce4699f0b226336d3d0deda3a7c"} Jul 12 18:43:16.821219 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:16.821179 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz" event={"ID":"045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c","Type":"ContainerDied","Data":"8f74172d967cd982ce27af348a2692dfb3468e37816ecbd50ca82ae6e6e5dd7a"} Jul 12 18:43:16.821219 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:16.821208 2577 scope.go:117] "RemoveContainer" containerID="6c723abfef228cd97e7357419c77c3903072dce4699f0b226336d3d0deda3a7c" Jul 12 18:43:16.830442 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:16.830425 2577 scope.go:117] "RemoveContainer" containerID="6c723abfef228cd97e7357419c77c3903072dce4699f0b226336d3d0deda3a7c" Jul 12 18:43:16.830715 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:43:16.830695 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c723abfef228cd97e7357419c77c3903072dce4699f0b226336d3d0deda3a7c\": container with ID starting with 6c723abfef228cd97e7357419c77c3903072dce4699f0b226336d3d0deda3a7c not found: ID does not exist" containerID="6c723abfef228cd97e7357419c77c3903072dce4699f0b226336d3d0deda3a7c" Jul 12 18:43:16.830795 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:16.830728 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c723abfef228cd97e7357419c77c3903072dce4699f0b226336d3d0deda3a7c"} err="failed to get container status \"6c723abfef228cd97e7357419c77c3903072dce4699f0b226336d3d0deda3a7c\": rpc error: code = NotFound desc = could not find container \"6c723abfef228cd97e7357419c77c3903072dce4699f0b226336d3d0deda3a7c\": container with ID starting with 6c723abfef228cd97e7357419c77c3903072dce4699f0b226336d3d0deda3a7c not found: ID does not exist" Jul 12 18:43:16.842640 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:16.842608 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz"] Jul 12 18:43:16.847066 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:16.847043 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-b7e51ce2-openshift-default-754b6c789f-kt8sz"] Jul 12 18:43:17.075819 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:17.075728 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c" path="/var/lib/kubelet/pods/045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c/volumes" Jul 12 18:43:18.684052 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:18.684018 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:43:20.824302 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:20.824264 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn"] Jul 12 18:43:20.824799 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:20.824781 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c" containerName="istio-proxy" Jul 12 18:43:20.824877 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:20.824802 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c" containerName="istio-proxy" Jul 12 18:43:20.824937 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:20.824909 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="045cabbc-2fa6-4c99-b9e7-f5ed542a2a4c" containerName="istio-proxy" Jul 12 18:43:20.827740 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:20.827721 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn" Jul 12 18:43:20.831654 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:20.831616 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-93b7e9bc-openshift-default-dockercfg-67rzj\"" Jul 12 18:43:20.851351 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:20.851321 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn"] Jul 12 18:43:21.009003 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:21.008966 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4417c3d2-e2ac-423b-aa9d-fdd25f68c72c-workload-certs\") pod \"e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn\" (UID: \"4417c3d2-e2ac-423b-aa9d-fdd25f68c72c\") " pod="openshift-ingress/e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn" Jul 12 18:43:21.009003 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:21.009004 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4417c3d2-e2ac-423b-aa9d-fdd25f68c72c-istio-envoy\") pod \"e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn\" (UID: \"4417c3d2-e2ac-423b-aa9d-fdd25f68c72c\") " pod="openshift-ingress/e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn" Jul 12 18:43:21.009222 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:21.009033 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4417c3d2-e2ac-423b-aa9d-fdd25f68c72c-istio-podinfo\") pod \"e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn\" (UID: \"4417c3d2-e2ac-423b-aa9d-fdd25f68c72c\") " pod="openshift-ingress/e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn" Jul 12 18:43:21.009222 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:21.009081 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4417c3d2-e2ac-423b-aa9d-fdd25f68c72c-istio-token\") pod \"e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn\" (UID: \"4417c3d2-e2ac-423b-aa9d-fdd25f68c72c\") " pod="openshift-ingress/e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn" Jul 12 18:43:21.009222 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:21.009126 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4417c3d2-e2ac-423b-aa9d-fdd25f68c72c-credential-socket\") pod \"e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn\" (UID: \"4417c3d2-e2ac-423b-aa9d-fdd25f68c72c\") " pod="openshift-ingress/e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn" Jul 12 18:43:21.009222 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:21.009144 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4417c3d2-e2ac-423b-aa9d-fdd25f68c72c-istio-data\") pod \"e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn\" (UID: \"4417c3d2-e2ac-423b-aa9d-fdd25f68c72c\") " pod="openshift-ingress/e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn" Jul 12 18:43:21.009222 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:21.009179 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4417c3d2-e2ac-423b-aa9d-fdd25f68c72c-istiod-ca-cert\") pod \"e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn\" (UID: \"4417c3d2-e2ac-423b-aa9d-fdd25f68c72c\") " pod="openshift-ingress/e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn" Jul 12 18:43:21.009383 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:21.009233 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4417c3d2-e2ac-423b-aa9d-fdd25f68c72c-workload-socket\") pod \"e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn\" (UID: \"4417c3d2-e2ac-423b-aa9d-fdd25f68c72c\") " pod="openshift-ingress/e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn" Jul 12 18:43:21.009383 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:21.009254 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hjdv\" (UniqueName: \"kubernetes.io/projected/4417c3d2-e2ac-423b-aa9d-fdd25f68c72c-kube-api-access-4hjdv\") pod \"e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn\" (UID: \"4417c3d2-e2ac-423b-aa9d-fdd25f68c72c\") " pod="openshift-ingress/e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn" Jul 12 18:43:21.110554 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:21.110464 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4417c3d2-e2ac-423b-aa9d-fdd25f68c72c-workload-certs\") pod \"e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn\" (UID: \"4417c3d2-e2ac-423b-aa9d-fdd25f68c72c\") " pod="openshift-ingress/e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn" Jul 12 18:43:21.110554 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:21.110497 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4417c3d2-e2ac-423b-aa9d-fdd25f68c72c-istio-envoy\") pod \"e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn\" (UID: \"4417c3d2-e2ac-423b-aa9d-fdd25f68c72c\") " pod="openshift-ingress/e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn" Jul 12 18:43:21.110554 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:21.110524 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4417c3d2-e2ac-423b-aa9d-fdd25f68c72c-istio-podinfo\") pod \"e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn\" (UID: \"4417c3d2-e2ac-423b-aa9d-fdd25f68c72c\") " pod="openshift-ingress/e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn" Jul 12 18:43:21.110830 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:21.110658 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4417c3d2-e2ac-423b-aa9d-fdd25f68c72c-istio-token\") pod \"e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn\" (UID: \"4417c3d2-e2ac-423b-aa9d-fdd25f68c72c\") " pod="openshift-ingress/e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn" Jul 12 18:43:21.110830 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:21.110709 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4417c3d2-e2ac-423b-aa9d-fdd25f68c72c-credential-socket\") pod \"e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn\" (UID: \"4417c3d2-e2ac-423b-aa9d-fdd25f68c72c\") " pod="openshift-ingress/e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn" Jul 12 18:43:21.110830 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:21.110738 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4417c3d2-e2ac-423b-aa9d-fdd25f68c72c-istio-data\") pod \"e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn\" (UID: \"4417c3d2-e2ac-423b-aa9d-fdd25f68c72c\") " pod="openshift-ingress/e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn" Jul 12 18:43:21.110830 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:21.110796 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4417c3d2-e2ac-423b-aa9d-fdd25f68c72c-istiod-ca-cert\") pod \"e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn\" (UID: \"4417c3d2-e2ac-423b-aa9d-fdd25f68c72c\") " pod="openshift-ingress/e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn" Jul 12 18:43:21.111134 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:21.110857 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4417c3d2-e2ac-423b-aa9d-fdd25f68c72c-workload-socket\") pod \"e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn\" (UID: \"4417c3d2-e2ac-423b-aa9d-fdd25f68c72c\") " pod="openshift-ingress/e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn" Jul 12 18:43:21.111134 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:21.110888 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hjdv\" (UniqueName: \"kubernetes.io/projected/4417c3d2-e2ac-423b-aa9d-fdd25f68c72c-kube-api-access-4hjdv\") pod \"e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn\" (UID: \"4417c3d2-e2ac-423b-aa9d-fdd25f68c72c\") " pod="openshift-ingress/e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn" Jul 12 18:43:21.111134 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:21.111029 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4417c3d2-e2ac-423b-aa9d-fdd25f68c72c-credential-socket\") pod \"e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn\" (UID: \"4417c3d2-e2ac-423b-aa9d-fdd25f68c72c\") " pod="openshift-ingress/e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn" Jul 12 18:43:21.111134 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:21.111080 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4417c3d2-e2ac-423b-aa9d-fdd25f68c72c-istio-data\") pod \"e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn\" (UID: \"4417c3d2-e2ac-423b-aa9d-fdd25f68c72c\") " pod="openshift-ingress/e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn" Jul 12 18:43:21.111365 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:21.111192 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4417c3d2-e2ac-423b-aa9d-fdd25f68c72c-workload-certs\") pod \"e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn\" (UID: \"4417c3d2-e2ac-423b-aa9d-fdd25f68c72c\") " pod="openshift-ingress/e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn" Jul 12 18:43:21.111365 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:21.111265 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4417c3d2-e2ac-423b-aa9d-fdd25f68c72c-workload-socket\") pod \"e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn\" (UID: \"4417c3d2-e2ac-423b-aa9d-fdd25f68c72c\") " pod="openshift-ingress/e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn" Jul 12 18:43:21.111651 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:21.111609 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4417c3d2-e2ac-423b-aa9d-fdd25f68c72c-istiod-ca-cert\") pod \"e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn\" (UID: \"4417c3d2-e2ac-423b-aa9d-fdd25f68c72c\") " pod="openshift-ingress/e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn" Jul 12 18:43:21.112855 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:21.112825 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4417c3d2-e2ac-423b-aa9d-fdd25f68c72c-istio-envoy\") pod \"e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn\" (UID: \"4417c3d2-e2ac-423b-aa9d-fdd25f68c72c\") " pod="openshift-ingress/e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn" Jul 12 18:43:21.112960 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:21.112942 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4417c3d2-e2ac-423b-aa9d-fdd25f68c72c-istio-podinfo\") pod \"e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn\" (UID: \"4417c3d2-e2ac-423b-aa9d-fdd25f68c72c\") " pod="openshift-ingress/e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn" Jul 12 18:43:21.119242 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:21.119218 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4417c3d2-e2ac-423b-aa9d-fdd25f68c72c-istio-token\") pod \"e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn\" (UID: \"4417c3d2-e2ac-423b-aa9d-fdd25f68c72c\") " pod="openshift-ingress/e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn" Jul 12 18:43:21.119339 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:21.119306 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hjdv\" (UniqueName: \"kubernetes.io/projected/4417c3d2-e2ac-423b-aa9d-fdd25f68c72c-kube-api-access-4hjdv\") pod \"e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn\" (UID: \"4417c3d2-e2ac-423b-aa9d-fdd25f68c72c\") " pod="openshift-ingress/e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn" Jul 12 18:43:21.138278 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:21.138252 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn" Jul 12 18:43:21.274474 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:21.274447 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn"] Jul 12 18:43:21.276383 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:43:21.276357 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4417c3d2_e2ac_423b_aa9d_fdd25f68c72c.slice/crio-587a193ef4e171638cb940056799475c2d83e6ee72012de8d04c862bd0c4f84d WatchSource:0}: Error finding container 587a193ef4e171638cb940056799475c2d83e6ee72012de8d04c862bd0c4f84d: Status 404 returned error can't find the container with id 587a193ef4e171638cb940056799475c2d83e6ee72012de8d04c862bd0c4f84d Jul 12 18:43:21.278523 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:21.278490 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 12 18:43:21.278587 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:21.278558 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 12 18:43:21.278622 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:21.278588 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 12 18:43:21.847537 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:21.847502 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn" event={"ID":"4417c3d2-e2ac-423b-aa9d-fdd25f68c72c","Type":"ContainerStarted","Data":"a72f097b3dcb32810330d94031be489888ef65485940bb1c230bbefb4e208112"} Jul 12 18:43:21.847963 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:21.847565 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn" event={"ID":"4417c3d2-e2ac-423b-aa9d-fdd25f68c72c","Type":"ContainerStarted","Data":"587a193ef4e171638cb940056799475c2d83e6ee72012de8d04c862bd0c4f84d"} Jul 12 18:43:21.866321 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:21.866271 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn" podStartSLOduration=1.866251785 podStartE2EDuration="1.866251785s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:43:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:43:21.866119654 +0000 UTC m=+1574.993024459" watchObservedRunningTime="2026-07-12 18:43:21.866251785 +0000 UTC m=+1574.993156591" Jul 12 18:43:22.139137 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:22.139055 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn" Jul 12 18:43:22.143795 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:22.143771 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn" Jul 12 18:43:22.853037 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:22.853006 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn" Jul 12 18:43:22.853934 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:22.853917 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn" Jul 12 18:43:26.526403 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:26.526368 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn"] Jul 12 18:43:26.526840 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:26.526605 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn" podUID="4417c3d2-e2ac-423b-aa9d-fdd25f68c72c" containerName="istio-proxy" containerID="cri-o://a72f097b3dcb32810330d94031be489888ef65485940bb1c230bbefb4e208112" gracePeriod=30 Jul 12 18:43:31.781958 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:31.781933 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn" Jul 12 18:43:31.802134 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:31.801607 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4hjdv\" (UniqueName: \"kubernetes.io/projected/4417c3d2-e2ac-423b-aa9d-fdd25f68c72c-kube-api-access-4hjdv\") pod \"4417c3d2-e2ac-423b-aa9d-fdd25f68c72c\" (UID: \"4417c3d2-e2ac-423b-aa9d-fdd25f68c72c\") " Jul 12 18:43:31.802303 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:31.802128 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4417c3d2-e2ac-423b-aa9d-fdd25f68c72c-istiod-ca-cert\") pod \"4417c3d2-e2ac-423b-aa9d-fdd25f68c72c\" (UID: \"4417c3d2-e2ac-423b-aa9d-fdd25f68c72c\") " Jul 12 18:43:31.802303 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:31.802212 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4417c3d2-e2ac-423b-aa9d-fdd25f68c72c-istio-token\") pod \"4417c3d2-e2ac-423b-aa9d-fdd25f68c72c\" (UID: \"4417c3d2-e2ac-423b-aa9d-fdd25f68c72c\") " Jul 12 18:43:31.802303 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:31.802247 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4417c3d2-e2ac-423b-aa9d-fdd25f68c72c-credential-socket\") pod \"4417c3d2-e2ac-423b-aa9d-fdd25f68c72c\" (UID: \"4417c3d2-e2ac-423b-aa9d-fdd25f68c72c\") " Jul 12 18:43:31.802303 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:31.802296 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4417c3d2-e2ac-423b-aa9d-fdd25f68c72c-istio-podinfo\") pod \"4417c3d2-e2ac-423b-aa9d-fdd25f68c72c\" (UID: \"4417c3d2-e2ac-423b-aa9d-fdd25f68c72c\") " Jul 12 18:43:31.802518 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:31.802333 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4417c3d2-e2ac-423b-aa9d-fdd25f68c72c-workload-certs\") pod \"4417c3d2-e2ac-423b-aa9d-fdd25f68c72c\" (UID: \"4417c3d2-e2ac-423b-aa9d-fdd25f68c72c\") " Jul 12 18:43:31.802518 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:31.802361 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4417c3d2-e2ac-423b-aa9d-fdd25f68c72c-istio-envoy\") pod \"4417c3d2-e2ac-423b-aa9d-fdd25f68c72c\" (UID: \"4417c3d2-e2ac-423b-aa9d-fdd25f68c72c\") " Jul 12 18:43:31.802518 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:31.802397 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4417c3d2-e2ac-423b-aa9d-fdd25f68c72c-istio-data\") pod \"4417c3d2-e2ac-423b-aa9d-fdd25f68c72c\" (UID: \"4417c3d2-e2ac-423b-aa9d-fdd25f68c72c\") " Jul 12 18:43:31.802518 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:31.802419 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4417c3d2-e2ac-423b-aa9d-fdd25f68c72c-workload-socket\") pod \"4417c3d2-e2ac-423b-aa9d-fdd25f68c72c\" (UID: \"4417c3d2-e2ac-423b-aa9d-fdd25f68c72c\") " Jul 12 18:43:31.802756 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:31.802535 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4417c3d2-e2ac-423b-aa9d-fdd25f68c72c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4417c3d2-e2ac-423b-aa9d-fdd25f68c72c" (UID: "4417c3d2-e2ac-423b-aa9d-fdd25f68c72c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:43:31.802756 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:31.802724 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4417c3d2-e2ac-423b-aa9d-fdd25f68c72c-istiod-ca-cert\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:43:31.802859 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:31.802804 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4417c3d2-e2ac-423b-aa9d-fdd25f68c72c-istio-data" (OuterVolumeSpecName: "istio-data") pod "4417c3d2-e2ac-423b-aa9d-fdd25f68c72c" (UID: "4417c3d2-e2ac-423b-aa9d-fdd25f68c72c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:43:31.802859 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:31.802784 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4417c3d2-e2ac-423b-aa9d-fdd25f68c72c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4417c3d2-e2ac-423b-aa9d-fdd25f68c72c" (UID: "4417c3d2-e2ac-423b-aa9d-fdd25f68c72c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:43:31.803003 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:31.802962 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4417c3d2-e2ac-423b-aa9d-fdd25f68c72c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4417c3d2-e2ac-423b-aa9d-fdd25f68c72c" (UID: "4417c3d2-e2ac-423b-aa9d-fdd25f68c72c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:43:31.803134 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:31.803096 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4417c3d2-e2ac-423b-aa9d-fdd25f68c72c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4417c3d2-e2ac-423b-aa9d-fdd25f68c72c" (UID: "4417c3d2-e2ac-423b-aa9d-fdd25f68c72c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:43:31.804546 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:31.804508 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4417c3d2-e2ac-423b-aa9d-fdd25f68c72c-kube-api-access-4hjdv" (OuterVolumeSpecName: "kube-api-access-4hjdv") pod "4417c3d2-e2ac-423b-aa9d-fdd25f68c72c" (UID: "4417c3d2-e2ac-423b-aa9d-fdd25f68c72c"). InnerVolumeSpecName "kube-api-access-4hjdv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:43:31.805145 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:31.805115 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4417c3d2-e2ac-423b-aa9d-fdd25f68c72c-istio-token" (OuterVolumeSpecName: "istio-token") pod "4417c3d2-e2ac-423b-aa9d-fdd25f68c72c" (UID: "4417c3d2-e2ac-423b-aa9d-fdd25f68c72c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:43:31.805145 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:31.805126 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4417c3d2-e2ac-423b-aa9d-fdd25f68c72c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4417c3d2-e2ac-423b-aa9d-fdd25f68c72c" (UID: "4417c3d2-e2ac-423b-aa9d-fdd25f68c72c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:43:31.805309 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:31.805282 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4417c3d2-e2ac-423b-aa9d-fdd25f68c72c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4417c3d2-e2ac-423b-aa9d-fdd25f68c72c" (UID: "4417c3d2-e2ac-423b-aa9d-fdd25f68c72c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 18:43:31.903194 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:31.903164 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4417c3d2-e2ac-423b-aa9d-fdd25f68c72c-workload-certs\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:43:31.903194 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:31.903187 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4417c3d2-e2ac-423b-aa9d-fdd25f68c72c-istio-envoy\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:43:31.903194 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:31.903196 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4417c3d2-e2ac-423b-aa9d-fdd25f68c72c-istio-data\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:43:31.903194 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:31.903203 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4417c3d2-e2ac-423b-aa9d-fdd25f68c72c-workload-socket\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:43:31.903469 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:31.903214 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4hjdv\" (UniqueName: \"kubernetes.io/projected/4417c3d2-e2ac-423b-aa9d-fdd25f68c72c-kube-api-access-4hjdv\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:43:31.903469 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:31.903222 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4417c3d2-e2ac-423b-aa9d-fdd25f68c72c-istio-token\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:43:31.903469 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:31.903232 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4417c3d2-e2ac-423b-aa9d-fdd25f68c72c-credential-socket\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:43:31.903469 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:31.903239 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4417c3d2-e2ac-423b-aa9d-fdd25f68c72c-istio-podinfo\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:43:31.904093 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:31.904066 2577 generic.go:356] "Generic (PLEG): container finished" podID="4417c3d2-e2ac-423b-aa9d-fdd25f68c72c" containerID="a72f097b3dcb32810330d94031be489888ef65485940bb1c230bbefb4e208112" exitCode=0 Jul 12 18:43:31.904186 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:31.904116 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn" Jul 12 18:43:31.904186 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:31.904154 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn" event={"ID":"4417c3d2-e2ac-423b-aa9d-fdd25f68c72c","Type":"ContainerDied","Data":"a72f097b3dcb32810330d94031be489888ef65485940bb1c230bbefb4e208112"} Jul 12 18:43:31.904257 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:31.904210 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn" event={"ID":"4417c3d2-e2ac-423b-aa9d-fdd25f68c72c","Type":"ContainerDied","Data":"587a193ef4e171638cb940056799475c2d83e6ee72012de8d04c862bd0c4f84d"} Jul 12 18:43:31.904257 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:31.904232 2577 scope.go:117] "RemoveContainer" containerID="a72f097b3dcb32810330d94031be489888ef65485940bb1c230bbefb4e208112" Jul 12 18:43:31.913535 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:31.913514 2577 scope.go:117] "RemoveContainer" containerID="a72f097b3dcb32810330d94031be489888ef65485940bb1c230bbefb4e208112" Jul 12 18:43:31.913824 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:43:31.913806 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a72f097b3dcb32810330d94031be489888ef65485940bb1c230bbefb4e208112\": container with ID starting with a72f097b3dcb32810330d94031be489888ef65485940bb1c230bbefb4e208112 not found: ID does not exist" containerID="a72f097b3dcb32810330d94031be489888ef65485940bb1c230bbefb4e208112" Jul 12 18:43:31.913881 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:31.913832 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a72f097b3dcb32810330d94031be489888ef65485940bb1c230bbefb4e208112"} err="failed to get container status \"a72f097b3dcb32810330d94031be489888ef65485940bb1c230bbefb4e208112\": rpc error: code = NotFound desc = could not find container \"a72f097b3dcb32810330d94031be489888ef65485940bb1c230bbefb4e208112\": container with ID starting with a72f097b3dcb32810330d94031be489888ef65485940bb1c230bbefb4e208112 not found: ID does not exist" Jul 12 18:43:31.926731 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:31.926700 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn"] Jul 12 18:43:31.930714 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:31.930686 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-93b7e9bc-openshift-default-5b9fdff75-k6wbn"] Jul 12 18:43:33.077727 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:43:33.077694 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4417c3d2-e2ac-423b-aa9d-fdd25f68c72c" path="/var/lib/kubelet/pods/4417c3d2-e2ac-423b-aa9d-fdd25f68c72c/volumes" Jul 12 18:44:55.386006 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:44:55.385972 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:45:04.784773 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:04.784736 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:45:07.750594 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:07.750566 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-2e12c194-openshift-default-646f467655-86598"] Jul 12 18:45:07.750967 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:07.750913 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4417c3d2-e2ac-423b-aa9d-fdd25f68c72c" containerName="istio-proxy" Jul 12 18:45:07.750967 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:07.750925 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4417c3d2-e2ac-423b-aa9d-fdd25f68c72c" containerName="istio-proxy" Jul 12 18:45:07.751056 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:07.751023 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="4417c3d2-e2ac-423b-aa9d-fdd25f68c72c" containerName="istio-proxy" Jul 12 18:45:07.754427 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:07.754406 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2e12c194-openshift-default-646f467655-86598" Jul 12 18:45:07.756990 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:07.756970 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-2e12c194-openshift-default-dockercfg-r2xtn\"" Jul 12 18:45:07.758690 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:07.758664 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e207479a-3f33-4ef6-83a8-438f9257a029-workload-certs\") pod \"e2e-mt-2e12c194-openshift-default-646f467655-86598\" (UID: \"e207479a-3f33-4ef6-83a8-438f9257a029\") " pod="openshift-ingress/e2e-mt-2e12c194-openshift-default-646f467655-86598" Jul 12 18:45:07.758780 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:07.758702 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e207479a-3f33-4ef6-83a8-438f9257a029-istio-envoy\") pod \"e2e-mt-2e12c194-openshift-default-646f467655-86598\" (UID: \"e207479a-3f33-4ef6-83a8-438f9257a029\") " pod="openshift-ingress/e2e-mt-2e12c194-openshift-default-646f467655-86598" Jul 12 18:45:07.758780 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:07.758741 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e207479a-3f33-4ef6-83a8-438f9257a029-istiod-ca-cert\") pod \"e2e-mt-2e12c194-openshift-default-646f467655-86598\" (UID: \"e207479a-3f33-4ef6-83a8-438f9257a029\") " pod="openshift-ingress/e2e-mt-2e12c194-openshift-default-646f467655-86598" Jul 12 18:45:07.758867 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:07.758801 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e207479a-3f33-4ef6-83a8-438f9257a029-credential-socket\") pod \"e2e-mt-2e12c194-openshift-default-646f467655-86598\" (UID: \"e207479a-3f33-4ef6-83a8-438f9257a029\") " pod="openshift-ingress/e2e-mt-2e12c194-openshift-default-646f467655-86598" Jul 12 18:45:07.758867 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:07.758824 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e207479a-3f33-4ef6-83a8-438f9257a029-istio-data\") pod \"e2e-mt-2e12c194-openshift-default-646f467655-86598\" (UID: \"e207479a-3f33-4ef6-83a8-438f9257a029\") " pod="openshift-ingress/e2e-mt-2e12c194-openshift-default-646f467655-86598" Jul 12 18:45:07.758947 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:07.758871 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e207479a-3f33-4ef6-83a8-438f9257a029-istio-token\") pod \"e2e-mt-2e12c194-openshift-default-646f467655-86598\" (UID: \"e207479a-3f33-4ef6-83a8-438f9257a029\") " pod="openshift-ingress/e2e-mt-2e12c194-openshift-default-646f467655-86598" Jul 12 18:45:07.758947 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:07.758897 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvf2r\" (UniqueName: \"kubernetes.io/projected/e207479a-3f33-4ef6-83a8-438f9257a029-kube-api-access-kvf2r\") pod \"e2e-mt-2e12c194-openshift-default-646f467655-86598\" (UID: \"e207479a-3f33-4ef6-83a8-438f9257a029\") " pod="openshift-ingress/e2e-mt-2e12c194-openshift-default-646f467655-86598" Jul 12 18:45:07.758947 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:07.758927 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e207479a-3f33-4ef6-83a8-438f9257a029-workload-socket\") pod \"e2e-mt-2e12c194-openshift-default-646f467655-86598\" (UID: \"e207479a-3f33-4ef6-83a8-438f9257a029\") " pod="openshift-ingress/e2e-mt-2e12c194-openshift-default-646f467655-86598" Jul 12 18:45:07.759076 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:07.758986 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e207479a-3f33-4ef6-83a8-438f9257a029-istio-podinfo\") pod \"e2e-mt-2e12c194-openshift-default-646f467655-86598\" (UID: \"e207479a-3f33-4ef6-83a8-438f9257a029\") " pod="openshift-ingress/e2e-mt-2e12c194-openshift-default-646f467655-86598" Jul 12 18:45:07.771665 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:07.771613 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-2e12c194-openshift-default-646f467655-86598"] Jul 12 18:45:07.859762 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:07.859726 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e207479a-3f33-4ef6-83a8-438f9257a029-istio-token\") pod \"e2e-mt-2e12c194-openshift-default-646f467655-86598\" (UID: \"e207479a-3f33-4ef6-83a8-438f9257a029\") " pod="openshift-ingress/e2e-mt-2e12c194-openshift-default-646f467655-86598" Jul 12 18:45:07.859762 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:07.859769 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvf2r\" (UniqueName: \"kubernetes.io/projected/e207479a-3f33-4ef6-83a8-438f9257a029-kube-api-access-kvf2r\") pod \"e2e-mt-2e12c194-openshift-default-646f467655-86598\" (UID: \"e207479a-3f33-4ef6-83a8-438f9257a029\") " pod="openshift-ingress/e2e-mt-2e12c194-openshift-default-646f467655-86598" Jul 12 18:45:07.860046 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:07.859805 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e207479a-3f33-4ef6-83a8-438f9257a029-workload-socket\") pod \"e2e-mt-2e12c194-openshift-default-646f467655-86598\" (UID: \"e207479a-3f33-4ef6-83a8-438f9257a029\") " pod="openshift-ingress/e2e-mt-2e12c194-openshift-default-646f467655-86598" Jul 12 18:45:07.860108 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:07.860040 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e207479a-3f33-4ef6-83a8-438f9257a029-istio-podinfo\") pod \"e2e-mt-2e12c194-openshift-default-646f467655-86598\" (UID: \"e207479a-3f33-4ef6-83a8-438f9257a029\") " pod="openshift-ingress/e2e-mt-2e12c194-openshift-default-646f467655-86598" Jul 12 18:45:07.860165 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:07.860147 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e207479a-3f33-4ef6-83a8-438f9257a029-workload-certs\") pod \"e2e-mt-2e12c194-openshift-default-646f467655-86598\" (UID: \"e207479a-3f33-4ef6-83a8-438f9257a029\") " pod="openshift-ingress/e2e-mt-2e12c194-openshift-default-646f467655-86598" Jul 12 18:45:07.860223 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:07.860175 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e207479a-3f33-4ef6-83a8-438f9257a029-istio-envoy\") pod \"e2e-mt-2e12c194-openshift-default-646f467655-86598\" (UID: \"e207479a-3f33-4ef6-83a8-438f9257a029\") " pod="openshift-ingress/e2e-mt-2e12c194-openshift-default-646f467655-86598" Jul 12 18:45:07.860223 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:07.860207 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e207479a-3f33-4ef6-83a8-438f9257a029-workload-socket\") pod \"e2e-mt-2e12c194-openshift-default-646f467655-86598\" (UID: \"e207479a-3f33-4ef6-83a8-438f9257a029\") " pod="openshift-ingress/e2e-mt-2e12c194-openshift-default-646f467655-86598" Jul 12 18:45:07.860223 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:07.860220 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e207479a-3f33-4ef6-83a8-438f9257a029-istiod-ca-cert\") pod \"e2e-mt-2e12c194-openshift-default-646f467655-86598\" (UID: \"e207479a-3f33-4ef6-83a8-438f9257a029\") " pod="openshift-ingress/e2e-mt-2e12c194-openshift-default-646f467655-86598" Jul 12 18:45:07.860387 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:07.860305 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e207479a-3f33-4ef6-83a8-438f9257a029-credential-socket\") pod \"e2e-mt-2e12c194-openshift-default-646f467655-86598\" (UID: \"e207479a-3f33-4ef6-83a8-438f9257a029\") " pod="openshift-ingress/e2e-mt-2e12c194-openshift-default-646f467655-86598" Jul 12 18:45:07.860387 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:07.860332 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e207479a-3f33-4ef6-83a8-438f9257a029-istio-data\") pod \"e2e-mt-2e12c194-openshift-default-646f467655-86598\" (UID: \"e207479a-3f33-4ef6-83a8-438f9257a029\") " pod="openshift-ingress/e2e-mt-2e12c194-openshift-default-646f467655-86598" Jul 12 18:45:07.860497 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:07.860481 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e207479a-3f33-4ef6-83a8-438f9257a029-workload-certs\") pod \"e2e-mt-2e12c194-openshift-default-646f467655-86598\" (UID: \"e207479a-3f33-4ef6-83a8-438f9257a029\") " pod="openshift-ingress/e2e-mt-2e12c194-openshift-default-646f467655-86598" Jul 12 18:45:07.860684 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:07.860664 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e207479a-3f33-4ef6-83a8-438f9257a029-istio-data\") pod \"e2e-mt-2e12c194-openshift-default-646f467655-86598\" (UID: \"e207479a-3f33-4ef6-83a8-438f9257a029\") " pod="openshift-ingress/e2e-mt-2e12c194-openshift-default-646f467655-86598" Jul 12 18:45:07.860844 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:07.860803 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e207479a-3f33-4ef6-83a8-438f9257a029-credential-socket\") pod \"e2e-mt-2e12c194-openshift-default-646f467655-86598\" (UID: \"e207479a-3f33-4ef6-83a8-438f9257a029\") " pod="openshift-ingress/e2e-mt-2e12c194-openshift-default-646f467655-86598" Jul 12 18:45:07.860964 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:07.860941 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e207479a-3f33-4ef6-83a8-438f9257a029-istiod-ca-cert\") pod \"e2e-mt-2e12c194-openshift-default-646f467655-86598\" (UID: \"e207479a-3f33-4ef6-83a8-438f9257a029\") " pod="openshift-ingress/e2e-mt-2e12c194-openshift-default-646f467655-86598" Jul 12 18:45:07.862686 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:07.862666 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e207479a-3f33-4ef6-83a8-438f9257a029-istio-envoy\") pod \"e2e-mt-2e12c194-openshift-default-646f467655-86598\" (UID: \"e207479a-3f33-4ef6-83a8-438f9257a029\") " pod="openshift-ingress/e2e-mt-2e12c194-openshift-default-646f467655-86598" Jul 12 18:45:07.862801 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:07.862749 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e207479a-3f33-4ef6-83a8-438f9257a029-istio-podinfo\") pod \"e2e-mt-2e12c194-openshift-default-646f467655-86598\" (UID: \"e207479a-3f33-4ef6-83a8-438f9257a029\") " pod="openshift-ingress/e2e-mt-2e12c194-openshift-default-646f467655-86598" Jul 12 18:45:07.867726 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:07.867706 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e207479a-3f33-4ef6-83a8-438f9257a029-istio-token\") pod \"e2e-mt-2e12c194-openshift-default-646f467655-86598\" (UID: \"e207479a-3f33-4ef6-83a8-438f9257a029\") " pod="openshift-ingress/e2e-mt-2e12c194-openshift-default-646f467655-86598" Jul 12 18:45:07.867824 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:07.867809 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvf2r\" (UniqueName: \"kubernetes.io/projected/e207479a-3f33-4ef6-83a8-438f9257a029-kube-api-access-kvf2r\") pod \"e2e-mt-2e12c194-openshift-default-646f467655-86598\" (UID: \"e207479a-3f33-4ef6-83a8-438f9257a029\") " pod="openshift-ingress/e2e-mt-2e12c194-openshift-default-646f467655-86598" Jul 12 18:45:08.067579 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:08.067482 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2e12c194-openshift-default-646f467655-86598" Jul 12 18:45:08.209607 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:08.209580 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-2e12c194-openshift-default-646f467655-86598"] Jul 12 18:45:08.211905 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:45:08.211874 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode207479a_3f33_4ef6_83a8_438f9257a029.slice/crio-0be46cf4b0337082655a54da3759dc04e7ce131af2a53e2d47f70e52878a9798 WatchSource:0}: Error finding container 0be46cf4b0337082655a54da3759dc04e7ce131af2a53e2d47f70e52878a9798: Status 404 returned error can't find the container with id 0be46cf4b0337082655a54da3759dc04e7ce131af2a53e2d47f70e52878a9798 Jul 12 18:45:08.214174 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:08.214144 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 12 18:45:08.214240 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:08.214214 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 12 18:45:08.214280 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:08.214241 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 12 18:45:08.404388 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:08.404298 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2e12c194-openshift-default-646f467655-86598" event={"ID":"e207479a-3f33-4ef6-83a8-438f9257a029","Type":"ContainerStarted","Data":"c266b213d7246b04e9d6877bbf9b1ed8de4f3d28fa093d81f69c9ce6c411b31d"} Jul 12 18:45:08.404388 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:08.404360 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2e12c194-openshift-default-646f467655-86598" event={"ID":"e207479a-3f33-4ef6-83a8-438f9257a029","Type":"ContainerStarted","Data":"0be46cf4b0337082655a54da3759dc04e7ce131af2a53e2d47f70e52878a9798"} Jul 12 18:45:08.425767 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:08.425713 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-2e12c194-openshift-default-646f467655-86598" podStartSLOduration=1.42569587 podStartE2EDuration="1.42569587s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:45:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:45:08.423664027 +0000 UTC m=+1681.550568830" watchObservedRunningTime="2026-07-12 18:45:08.42569587 +0000 UTC m=+1681.552600652" Jul 12 18:45:09.067716 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:09.067684 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-2e12c194-openshift-default-646f467655-86598" Jul 12 18:45:09.078240 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:09.078211 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-2e12c194-openshift-default-646f467655-86598" Jul 12 18:45:09.409919 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:09.409840 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2e12c194-openshift-default-646f467655-86598" Jul 12 18:45:09.410740 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:09.410724 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-2e12c194-openshift-default-646f467655-86598" Jul 12 18:45:15.979902 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:15.979864 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:45:24.386432 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:24.386396 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:45:24.411123 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:24.411089 2577 patch_prober.go:28] interesting pod/e2e-mt-2e12c194-openshift-default-646f467655-86598 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Jul 12 18:45:24.411290 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:24.411139 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-2e12c194-openshift-default-646f467655-86598" podUID="e207479a-3f33-4ef6-83a8-438f9257a029" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Jul 12 18:45:24.494957 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:24.494920 2577 generic.go:356] "Generic (PLEG): container finished" podID="e207479a-3f33-4ef6-83a8-438f9257a029" containerID="c266b213d7246b04e9d6877bbf9b1ed8de4f3d28fa093d81f69c9ce6c411b31d" exitCode=0 Jul 12 18:45:24.495153 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:24.494977 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2e12c194-openshift-default-646f467655-86598" event={"ID":"e207479a-3f33-4ef6-83a8-438f9257a029","Type":"ContainerDied","Data":"c266b213d7246b04e9d6877bbf9b1ed8de4f3d28fa093d81f69c9ce6c411b31d"} Jul 12 18:45:24.495425 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:24.495408 2577 scope.go:117] "RemoveContainer" containerID="c266b213d7246b04e9d6877bbf9b1ed8de4f3d28fa093d81f69c9ce6c411b31d" Jul 12 18:45:24.496396 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:24.496364 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 12 18:45:24.496462 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:24.496433 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 12 18:45:24.496500 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:24.496469 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 12 18:45:25.507499 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:25.507466 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2e12c194-openshift-default-646f467655-86598" event={"ID":"e207479a-3f33-4ef6-83a8-438f9257a029","Type":"ContainerStarted","Data":"fbf9801c20b2d94bc27af1b851f3860ae62fae8790ef5a44a008053245bf2646"} Jul 12 18:45:26.068430 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:26.068392 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-2e12c194-openshift-default-646f467655-86598" Jul 12 18:45:26.072668 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:26.072618 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-2e12c194-openshift-default-646f467655-86598" Jul 12 18:45:26.512791 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:26.512761 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2e12c194-openshift-default-646f467655-86598" Jul 12 18:45:26.513689 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:26.513670 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-2e12c194-openshift-default-646f467655-86598" Jul 12 18:45:28.186162 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:28.186125 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-2e12c194-openshift-default-646f467655-86598"] Jul 12 18:45:28.523707 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:28.523615 2577 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-mt-2e12c194-openshift-default-646f467655-86598" secret="" err="secret \"e2e-mt-2e12c194-openshift-default-dockercfg-r2xtn\" not found" Jul 12 18:45:29.528711 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:29.528581 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-2e12c194-openshift-default-646f467655-86598" podUID="e207479a-3f33-4ef6-83a8-438f9257a029" containerName="istio-proxy" containerID="cri-o://fbf9801c20b2d94bc27af1b851f3860ae62fae8790ef5a44a008053245bf2646" gracePeriod=30 Jul 12 18:45:34.767218 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:34.767194 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2e12c194-openshift-default-646f467655-86598" Jul 12 18:45:34.907735 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:34.907614 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kvf2r\" (UniqueName: \"kubernetes.io/projected/e207479a-3f33-4ef6-83a8-438f9257a029-kube-api-access-kvf2r\") pod \"e207479a-3f33-4ef6-83a8-438f9257a029\" (UID: \"e207479a-3f33-4ef6-83a8-438f9257a029\") " Jul 12 18:45:34.907735 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:34.907727 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e207479a-3f33-4ef6-83a8-438f9257a029-istio-envoy\") pod \"e207479a-3f33-4ef6-83a8-438f9257a029\" (UID: \"e207479a-3f33-4ef6-83a8-438f9257a029\") " Jul 12 18:45:34.907981 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:34.907749 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e207479a-3f33-4ef6-83a8-438f9257a029-istio-data\") pod \"e207479a-3f33-4ef6-83a8-438f9257a029\" (UID: \"e207479a-3f33-4ef6-83a8-438f9257a029\") " Jul 12 18:45:34.907981 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:34.907790 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e207479a-3f33-4ef6-83a8-438f9257a029-istiod-ca-cert\") pod \"e207479a-3f33-4ef6-83a8-438f9257a029\" (UID: \"e207479a-3f33-4ef6-83a8-438f9257a029\") " Jul 12 18:45:34.907981 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:34.907824 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e207479a-3f33-4ef6-83a8-438f9257a029-credential-socket\") pod \"e207479a-3f33-4ef6-83a8-438f9257a029\" (UID: \"e207479a-3f33-4ef6-83a8-438f9257a029\") " Jul 12 18:45:34.907981 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:34.907864 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e207479a-3f33-4ef6-83a8-438f9257a029-workload-socket\") pod \"e207479a-3f33-4ef6-83a8-438f9257a029\" (UID: \"e207479a-3f33-4ef6-83a8-438f9257a029\") " Jul 12 18:45:34.907981 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:34.907899 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e207479a-3f33-4ef6-83a8-438f9257a029-istio-podinfo\") pod \"e207479a-3f33-4ef6-83a8-438f9257a029\" (UID: \"e207479a-3f33-4ef6-83a8-438f9257a029\") " Jul 12 18:45:34.907981 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:34.907946 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e207479a-3f33-4ef6-83a8-438f9257a029-workload-certs\") pod \"e207479a-3f33-4ef6-83a8-438f9257a029\" (UID: \"e207479a-3f33-4ef6-83a8-438f9257a029\") " Jul 12 18:45:34.908286 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:34.908053 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e207479a-3f33-4ef6-83a8-438f9257a029-istio-token\") pod \"e207479a-3f33-4ef6-83a8-438f9257a029\" (UID: \"e207479a-3f33-4ef6-83a8-438f9257a029\") " Jul 12 18:45:34.908286 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:34.908165 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e207479a-3f33-4ef6-83a8-438f9257a029-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e207479a-3f33-4ef6-83a8-438f9257a029" (UID: "e207479a-3f33-4ef6-83a8-438f9257a029"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:45:34.908286 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:34.908220 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e207479a-3f33-4ef6-83a8-438f9257a029-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e207479a-3f33-4ef6-83a8-438f9257a029" (UID: "e207479a-3f33-4ef6-83a8-438f9257a029"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:45:34.908286 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:34.908168 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e207479a-3f33-4ef6-83a8-438f9257a029-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e207479a-3f33-4ef6-83a8-438f9257a029" (UID: "e207479a-3f33-4ef6-83a8-438f9257a029"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:45:34.908478 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:34.908368 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e207479a-3f33-4ef6-83a8-438f9257a029-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e207479a-3f33-4ef6-83a8-438f9257a029" (UID: "e207479a-3f33-4ef6-83a8-438f9257a029"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:45:34.908478 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:34.908421 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e207479a-3f33-4ef6-83a8-438f9257a029-istiod-ca-cert\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:45:34.908478 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:34.908441 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e207479a-3f33-4ef6-83a8-438f9257a029-credential-socket\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:45:34.908478 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:34.908455 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e207479a-3f33-4ef6-83a8-438f9257a029-workload-socket\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:45:34.908853 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:34.908831 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e207479a-3f33-4ef6-83a8-438f9257a029-istio-data" (OuterVolumeSpecName: "istio-data") pod "e207479a-3f33-4ef6-83a8-438f9257a029" (UID: "e207479a-3f33-4ef6-83a8-438f9257a029"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:45:34.909911 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:34.909891 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e207479a-3f33-4ef6-83a8-438f9257a029-kube-api-access-kvf2r" (OuterVolumeSpecName: "kube-api-access-kvf2r") pod "e207479a-3f33-4ef6-83a8-438f9257a029" (UID: "e207479a-3f33-4ef6-83a8-438f9257a029"). InnerVolumeSpecName "kube-api-access-kvf2r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:45:34.909999 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:34.909909 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e207479a-3f33-4ef6-83a8-438f9257a029-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e207479a-3f33-4ef6-83a8-438f9257a029" (UID: "e207479a-3f33-4ef6-83a8-438f9257a029"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:45:34.910454 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:34.910438 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e207479a-3f33-4ef6-83a8-438f9257a029-istio-token" (OuterVolumeSpecName: "istio-token") pod "e207479a-3f33-4ef6-83a8-438f9257a029" (UID: "e207479a-3f33-4ef6-83a8-438f9257a029"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:45:34.910494 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:34.910444 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e207479a-3f33-4ef6-83a8-438f9257a029-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e207479a-3f33-4ef6-83a8-438f9257a029" (UID: "e207479a-3f33-4ef6-83a8-438f9257a029"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 18:45:35.009772 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:35.009731 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e207479a-3f33-4ef6-83a8-438f9257a029-istio-podinfo\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:45:35.009772 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:35.009763 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e207479a-3f33-4ef6-83a8-438f9257a029-workload-certs\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:45:35.009772 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:35.009777 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e207479a-3f33-4ef6-83a8-438f9257a029-istio-token\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:45:35.010019 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:35.009789 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kvf2r\" (UniqueName: \"kubernetes.io/projected/e207479a-3f33-4ef6-83a8-438f9257a029-kube-api-access-kvf2r\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:45:35.010019 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:35.009800 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e207479a-3f33-4ef6-83a8-438f9257a029-istio-envoy\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:45:35.010019 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:35.009813 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e207479a-3f33-4ef6-83a8-438f9257a029-istio-data\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:45:35.560021 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:35.559985 2577 generic.go:356] "Generic (PLEG): container finished" podID="e207479a-3f33-4ef6-83a8-438f9257a029" containerID="fbf9801c20b2d94bc27af1b851f3860ae62fae8790ef5a44a008053245bf2646" exitCode=0 Jul 12 18:45:35.560220 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:35.560033 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2e12c194-openshift-default-646f467655-86598" Jul 12 18:45:35.560220 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:35.560042 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2e12c194-openshift-default-646f467655-86598" event={"ID":"e207479a-3f33-4ef6-83a8-438f9257a029","Type":"ContainerDied","Data":"fbf9801c20b2d94bc27af1b851f3860ae62fae8790ef5a44a008053245bf2646"} Jul 12 18:45:35.560220 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:35.560092 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2e12c194-openshift-default-646f467655-86598" event={"ID":"e207479a-3f33-4ef6-83a8-438f9257a029","Type":"ContainerDied","Data":"0be46cf4b0337082655a54da3759dc04e7ce131af2a53e2d47f70e52878a9798"} Jul 12 18:45:35.560220 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:35.560120 2577 scope.go:117] "RemoveContainer" containerID="fbf9801c20b2d94bc27af1b851f3860ae62fae8790ef5a44a008053245bf2646" Jul 12 18:45:35.568713 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:35.568695 2577 scope.go:117] "RemoveContainer" containerID="c266b213d7246b04e9d6877bbf9b1ed8de4f3d28fa093d81f69c9ce6c411b31d" Jul 12 18:45:35.576658 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:35.576639 2577 scope.go:117] "RemoveContainer" containerID="fbf9801c20b2d94bc27af1b851f3860ae62fae8790ef5a44a008053245bf2646" Jul 12 18:45:35.576959 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:45:35.576935 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fbf9801c20b2d94bc27af1b851f3860ae62fae8790ef5a44a008053245bf2646\": container with ID starting with fbf9801c20b2d94bc27af1b851f3860ae62fae8790ef5a44a008053245bf2646 not found: ID does not exist" containerID="fbf9801c20b2d94bc27af1b851f3860ae62fae8790ef5a44a008053245bf2646" Jul 12 18:45:35.577039 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:35.576970 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbf9801c20b2d94bc27af1b851f3860ae62fae8790ef5a44a008053245bf2646"} err="failed to get container status \"fbf9801c20b2d94bc27af1b851f3860ae62fae8790ef5a44a008053245bf2646\": rpc error: code = NotFound desc = could not find container \"fbf9801c20b2d94bc27af1b851f3860ae62fae8790ef5a44a008053245bf2646\": container with ID starting with fbf9801c20b2d94bc27af1b851f3860ae62fae8790ef5a44a008053245bf2646 not found: ID does not exist" Jul 12 18:45:35.577039 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:35.576991 2577 scope.go:117] "RemoveContainer" containerID="c266b213d7246b04e9d6877bbf9b1ed8de4f3d28fa093d81f69c9ce6c411b31d" Jul 12 18:45:35.577287 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:45:35.577267 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c266b213d7246b04e9d6877bbf9b1ed8de4f3d28fa093d81f69c9ce6c411b31d\": container with ID starting with c266b213d7246b04e9d6877bbf9b1ed8de4f3d28fa093d81f69c9ce6c411b31d not found: ID does not exist" containerID="c266b213d7246b04e9d6877bbf9b1ed8de4f3d28fa093d81f69c9ce6c411b31d" Jul 12 18:45:35.577336 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:35.577294 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c266b213d7246b04e9d6877bbf9b1ed8de4f3d28fa093d81f69c9ce6c411b31d"} err="failed to get container status \"c266b213d7246b04e9d6877bbf9b1ed8de4f3d28fa093d81f69c9ce6c411b31d\": rpc error: code = NotFound desc = could not find container \"c266b213d7246b04e9d6877bbf9b1ed8de4f3d28fa093d81f69c9ce6c411b31d\": container with ID starting with c266b213d7246b04e9d6877bbf9b1ed8de4f3d28fa093d81f69c9ce6c411b31d not found: ID does not exist" Jul 12 18:45:35.579173 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:35.579156 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-2e12c194-openshift-default-646f467655-86598"] Jul 12 18:45:35.585278 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:35.585255 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-2e12c194-openshift-default-646f467655-86598"] Jul 12 18:45:37.075963 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:37.075928 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e207479a-3f33-4ef6-83a8-438f9257a029" path="/var/lib/kubelet/pods/e207479a-3f33-4ef6-83a8-438f9257a029/volumes" Jul 12 18:45:47.642255 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:47.642221 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl"] Jul 12 18:45:47.642847 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:47.642672 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e207479a-3f33-4ef6-83a8-438f9257a029" containerName="istio-proxy" Jul 12 18:45:47.642847 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:47.642690 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e207479a-3f33-4ef6-83a8-438f9257a029" containerName="istio-proxy" Jul 12 18:45:47.642847 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:47.642705 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e207479a-3f33-4ef6-83a8-438f9257a029" containerName="istio-proxy" Jul 12 18:45:47.642847 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:47.642714 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e207479a-3f33-4ef6-83a8-438f9257a029" containerName="istio-proxy" Jul 12 18:45:47.642847 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:47.642830 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e207479a-3f33-4ef6-83a8-438f9257a029" containerName="istio-proxy" Jul 12 18:45:47.642847 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:47.642841 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e207479a-3f33-4ef6-83a8-438f9257a029" containerName="istio-proxy" Jul 12 18:45:47.646189 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:47.646167 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl" Jul 12 18:45:47.648939 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:47.648913 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-0328d158-openshift-default-dockercfg-4lrcg\"" Jul 12 18:45:47.656777 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:47.656749 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl"] Jul 12 18:45:47.838173 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:47.838137 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/531f75a1-f485-4f60-91d5-87fef4b58bed-credential-socket\") pod \"e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl\" (UID: \"531f75a1-f485-4f60-91d5-87fef4b58bed\") " pod="openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl" Jul 12 18:45:47.838350 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:47.838185 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/531f75a1-f485-4f60-91d5-87fef4b58bed-istiod-ca-cert\") pod \"e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl\" (UID: \"531f75a1-f485-4f60-91d5-87fef4b58bed\") " pod="openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl" Jul 12 18:45:47.838350 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:47.838210 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/531f75a1-f485-4f60-91d5-87fef4b58bed-istio-data\") pod \"e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl\" (UID: \"531f75a1-f485-4f60-91d5-87fef4b58bed\") " pod="openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl" Jul 12 18:45:47.838350 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:47.838242 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/531f75a1-f485-4f60-91d5-87fef4b58bed-workload-socket\") pod \"e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl\" (UID: \"531f75a1-f485-4f60-91d5-87fef4b58bed\") " pod="openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl" Jul 12 18:45:47.838350 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:47.838267 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8mn2\" (UniqueName: \"kubernetes.io/projected/531f75a1-f485-4f60-91d5-87fef4b58bed-kube-api-access-c8mn2\") pod \"e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl\" (UID: \"531f75a1-f485-4f60-91d5-87fef4b58bed\") " pod="openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl" Jul 12 18:45:47.838350 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:47.838321 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/531f75a1-f485-4f60-91d5-87fef4b58bed-istio-envoy\") pod \"e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl\" (UID: \"531f75a1-f485-4f60-91d5-87fef4b58bed\") " pod="openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl" Jul 12 18:45:47.838350 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:47.838338 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/531f75a1-f485-4f60-91d5-87fef4b58bed-istio-podinfo\") pod \"e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl\" (UID: \"531f75a1-f485-4f60-91d5-87fef4b58bed\") " pod="openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl" Jul 12 18:45:47.838563 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:47.838356 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/531f75a1-f485-4f60-91d5-87fef4b58bed-workload-certs\") pod \"e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl\" (UID: \"531f75a1-f485-4f60-91d5-87fef4b58bed\") " pod="openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl" Jul 12 18:45:47.838563 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:47.838401 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/531f75a1-f485-4f60-91d5-87fef4b58bed-istio-token\") pod \"e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl\" (UID: \"531f75a1-f485-4f60-91d5-87fef4b58bed\") " pod="openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl" Jul 12 18:45:47.939315 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:47.939281 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8mn2\" (UniqueName: \"kubernetes.io/projected/531f75a1-f485-4f60-91d5-87fef4b58bed-kube-api-access-c8mn2\") pod \"e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl\" (UID: \"531f75a1-f485-4f60-91d5-87fef4b58bed\") " pod="openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl" Jul 12 18:45:47.939476 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:47.939352 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/531f75a1-f485-4f60-91d5-87fef4b58bed-istio-envoy\") pod \"e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl\" (UID: \"531f75a1-f485-4f60-91d5-87fef4b58bed\") " pod="openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl" Jul 12 18:45:47.939476 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:47.939377 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/531f75a1-f485-4f60-91d5-87fef4b58bed-istio-podinfo\") pod \"e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl\" (UID: \"531f75a1-f485-4f60-91d5-87fef4b58bed\") " pod="openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl" Jul 12 18:45:47.939476 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:47.939404 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/531f75a1-f485-4f60-91d5-87fef4b58bed-workload-certs\") pod \"e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl\" (UID: \"531f75a1-f485-4f60-91d5-87fef4b58bed\") " pod="openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl" Jul 12 18:45:47.939476 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:47.939429 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/531f75a1-f485-4f60-91d5-87fef4b58bed-istio-token\") pod \"e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl\" (UID: \"531f75a1-f485-4f60-91d5-87fef4b58bed\") " pod="openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl" Jul 12 18:45:47.939718 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:47.939502 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/531f75a1-f485-4f60-91d5-87fef4b58bed-credential-socket\") pod \"e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl\" (UID: \"531f75a1-f485-4f60-91d5-87fef4b58bed\") " pod="openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl" Jul 12 18:45:47.939718 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:47.939542 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/531f75a1-f485-4f60-91d5-87fef4b58bed-istiod-ca-cert\") pod \"e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl\" (UID: \"531f75a1-f485-4f60-91d5-87fef4b58bed\") " pod="openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl" Jul 12 18:45:47.939718 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:47.939677 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/531f75a1-f485-4f60-91d5-87fef4b58bed-istio-data\") pod \"e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl\" (UID: \"531f75a1-f485-4f60-91d5-87fef4b58bed\") " pod="openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl" Jul 12 18:45:47.939867 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:47.939739 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/531f75a1-f485-4f60-91d5-87fef4b58bed-workload-socket\") pod \"e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl\" (UID: \"531f75a1-f485-4f60-91d5-87fef4b58bed\") " pod="openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl" Jul 12 18:45:47.940024 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:47.939995 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/531f75a1-f485-4f60-91d5-87fef4b58bed-credential-socket\") pod \"e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl\" (UID: \"531f75a1-f485-4f60-91d5-87fef4b58bed\") " pod="openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl" Jul 12 18:45:47.940118 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:47.940051 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/531f75a1-f485-4f60-91d5-87fef4b58bed-istio-data\") pod \"e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl\" (UID: \"531f75a1-f485-4f60-91d5-87fef4b58bed\") " pod="openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl" Jul 12 18:45:47.940118 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:47.940068 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/531f75a1-f485-4f60-91d5-87fef4b58bed-workload-socket\") pod \"e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl\" (UID: \"531f75a1-f485-4f60-91d5-87fef4b58bed\") " pod="openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl" Jul 12 18:45:47.940221 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:47.940206 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/531f75a1-f485-4f60-91d5-87fef4b58bed-workload-certs\") pod \"e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl\" (UID: \"531f75a1-f485-4f60-91d5-87fef4b58bed\") " pod="openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl" Jul 12 18:45:47.940261 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:47.940248 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/531f75a1-f485-4f60-91d5-87fef4b58bed-istiod-ca-cert\") pod \"e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl\" (UID: \"531f75a1-f485-4f60-91d5-87fef4b58bed\") " pod="openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl" Jul 12 18:45:47.942225 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:47.942205 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/531f75a1-f485-4f60-91d5-87fef4b58bed-istio-envoy\") pod \"e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl\" (UID: \"531f75a1-f485-4f60-91d5-87fef4b58bed\") " pod="openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl" Jul 12 18:45:47.942379 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:47.942362 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/531f75a1-f485-4f60-91d5-87fef4b58bed-istio-podinfo\") pod \"e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl\" (UID: \"531f75a1-f485-4f60-91d5-87fef4b58bed\") " pod="openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl" Jul 12 18:45:47.948888 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:47.948860 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/531f75a1-f485-4f60-91d5-87fef4b58bed-istio-token\") pod \"e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl\" (UID: \"531f75a1-f485-4f60-91d5-87fef4b58bed\") " pod="openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl" Jul 12 18:45:47.948888 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:47.948879 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8mn2\" (UniqueName: \"kubernetes.io/projected/531f75a1-f485-4f60-91d5-87fef4b58bed-kube-api-access-c8mn2\") pod \"e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl\" (UID: \"531f75a1-f485-4f60-91d5-87fef4b58bed\") " pod="openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl" Jul 12 18:45:47.959402 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:47.959374 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl" Jul 12 18:45:48.100294 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:48.100269 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl"] Jul 12 18:45:48.102676 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:45:48.102649 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod531f75a1_f485_4f60_91d5_87fef4b58bed.slice/crio-565d0f97b6a167851ad5d9a7eebeae35187eda7caffa07511720cd608232f604 WatchSource:0}: Error finding container 565d0f97b6a167851ad5d9a7eebeae35187eda7caffa07511720cd608232f604: Status 404 returned error can't find the container with id 565d0f97b6a167851ad5d9a7eebeae35187eda7caffa07511720cd608232f604 Jul 12 18:45:48.104872 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:48.104835 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 12 18:45:48.104963 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:48.104925 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 12 18:45:48.105009 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:48.104972 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 12 18:45:48.630260 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:48.630230 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl" event={"ID":"531f75a1-f485-4f60-91d5-87fef4b58bed","Type":"ContainerStarted","Data":"6d1c97ad9965d7af2ffea1ca2e7f4fe688a048c6a4596aec5d46071dcfae56d5"} Jul 12 18:45:48.630486 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:48.630286 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl" event={"ID":"531f75a1-f485-4f60-91d5-87fef4b58bed","Type":"ContainerStarted","Data":"565d0f97b6a167851ad5d9a7eebeae35187eda7caffa07511720cd608232f604"} Jul 12 18:45:48.651481 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:48.651424 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl" podStartSLOduration=1.651408593 podStartE2EDuration="1.651408593s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:45:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:45:48.648441875 +0000 UTC m=+1721.775346691" watchObservedRunningTime="2026-07-12 18:45:48.651408593 +0000 UTC m=+1721.778313398" Jul 12 18:45:48.959659 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:48.959614 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl" Jul 12 18:45:49.965715 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:49.965685 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl" Jul 12 18:45:50.641689 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:50.641655 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl" Jul 12 18:45:50.642455 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:50.642439 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl" Jul 12 18:45:57.585748 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:45:57.585714 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:46:05.643577 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:46:05.643499 2577 patch_prober.go:28] interesting pod/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Jul 12 18:46:05.643577 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:46:05.643550 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl" podUID="531f75a1-f485-4f60-91d5-87fef4b58bed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Jul 12 18:46:05.719489 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:46:05.719453 2577 generic.go:356] "Generic (PLEG): container finished" podID="531f75a1-f485-4f60-91d5-87fef4b58bed" containerID="6d1c97ad9965d7af2ffea1ca2e7f4fe688a048c6a4596aec5d46071dcfae56d5" exitCode=0 Jul 12 18:46:05.719668 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:46:05.719528 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl" event={"ID":"531f75a1-f485-4f60-91d5-87fef4b58bed","Type":"ContainerDied","Data":"6d1c97ad9965d7af2ffea1ca2e7f4fe688a048c6a4596aec5d46071dcfae56d5"} Jul 12 18:46:05.719976 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:46:05.719959 2577 scope.go:117] "RemoveContainer" containerID="6d1c97ad9965d7af2ffea1ca2e7f4fe688a048c6a4596aec5d46071dcfae56d5" Jul 12 18:46:05.721037 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:46:05.721008 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 12 18:46:05.721139 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:46:05.721071 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 12 18:46:05.721139 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:46:05.721105 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 12 18:46:06.682729 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:46:06.682693 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:46:06.727150 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:46:06.727106 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl" event={"ID":"531f75a1-f485-4f60-91d5-87fef4b58bed","Type":"ContainerStarted","Data":"333d949d080cd1d8161a6cc4f10734d1f8d2d26f9060beebe6e871c81b76590c"} Jul 12 18:46:06.959911 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:46:06.959872 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl" Jul 12 18:46:06.960108 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:46:06.960091 2577 patch_prober.go:28] interesting pod/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Jul 12 18:46:06.960169 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:46:06.960149 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl" podUID="531f75a1-f485-4f60-91d5-87fef4b58bed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Jul 12 18:46:07.960171 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:46:07.960134 2577 patch_prober.go:28] interesting pod/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Jul 12 18:46:07.960586 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:46:07.960187 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl" podUID="531f75a1-f485-4f60-91d5-87fef4b58bed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Jul 12 18:46:08.960177 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:46:08.960137 2577 patch_prober.go:28] interesting pod/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Jul 12 18:46:08.960692 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:46:08.960195 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl" podUID="531f75a1-f485-4f60-91d5-87fef4b58bed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Jul 12 18:46:09.960408 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:46:09.960378 2577 patch_prober.go:28] interesting pod/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Jul 12 18:46:09.960888 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:46:09.960865 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl" podUID="531f75a1-f485-4f60-91d5-87fef4b58bed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Jul 12 18:46:10.964798 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:46:10.964769 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl" Jul 12 18:46:10.965329 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:46:10.965048 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl" Jul 12 18:46:10.965912 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:46:10.965894 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl" Jul 12 18:46:21.079689 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:46:21.079650 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:46:32.784859 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:46:32.784823 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:46:42.479552 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:46:42.479512 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:47:09.991328 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:47:09.991292 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-spzqd_ed115460-b220-476d-b9d5-65f95ec3d059/ovn-acl-logging/0.log" Jul 12 18:47:09.998213 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:47:09.998193 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-spzqd_ed115460-b220-476d-b9d5-65f95ec3d059/ovn-acl-logging/0.log" Jul 12 18:47:18.578414 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:47:18.578361 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:47:21.077227 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:47:21.077199 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl"] Jul 12 18:47:21.077733 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:47:21.077400 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl" podUID="531f75a1-f485-4f60-91d5-87fef4b58bed" containerName="istio-proxy" containerID="cri-o://333d949d080cd1d8161a6cc4f10734d1f8d2d26f9060beebe6e871c81b76590c" gracePeriod=30 Jul 12 18:47:25.967322 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:47:25.967284 2577 patch_prober.go:28] interesting pod/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 12 18:47:25.967796 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:47:25.967357 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl" podUID="531f75a1-f485-4f60-91d5-87fef4b58bed" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 12 18:47:26.337573 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:47:26.337541 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl" Jul 12 18:47:26.446066 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:47:26.446030 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/531f75a1-f485-4f60-91d5-87fef4b58bed-workload-certs\") pod \"531f75a1-f485-4f60-91d5-87fef4b58bed\" (UID: \"531f75a1-f485-4f60-91d5-87fef4b58bed\") " Jul 12 18:47:26.446266 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:47:26.446078 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/531f75a1-f485-4f60-91d5-87fef4b58bed-credential-socket\") pod \"531f75a1-f485-4f60-91d5-87fef4b58bed\" (UID: \"531f75a1-f485-4f60-91d5-87fef4b58bed\") " Jul 12 18:47:26.446266 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:47:26.446147 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c8mn2\" (UniqueName: \"kubernetes.io/projected/531f75a1-f485-4f60-91d5-87fef4b58bed-kube-api-access-c8mn2\") pod \"531f75a1-f485-4f60-91d5-87fef4b58bed\" (UID: \"531f75a1-f485-4f60-91d5-87fef4b58bed\") " Jul 12 18:47:26.446266 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:47:26.446188 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/531f75a1-f485-4f60-91d5-87fef4b58bed-istio-podinfo\") pod \"531f75a1-f485-4f60-91d5-87fef4b58bed\" (UID: \"531f75a1-f485-4f60-91d5-87fef4b58bed\") " Jul 12 18:47:26.446266 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:47:26.446224 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/531f75a1-f485-4f60-91d5-87fef4b58bed-istiod-ca-cert\") pod \"531f75a1-f485-4f60-91d5-87fef4b58bed\" (UID: \"531f75a1-f485-4f60-91d5-87fef4b58bed\") " Jul 12 18:47:26.446266 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:47:26.446242 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/531f75a1-f485-4f60-91d5-87fef4b58bed-istio-data\") pod \"531f75a1-f485-4f60-91d5-87fef4b58bed\" (UID: \"531f75a1-f485-4f60-91d5-87fef4b58bed\") " Jul 12 18:47:26.446266 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:47:26.446257 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/531f75a1-f485-4f60-91d5-87fef4b58bed-istio-token\") pod \"531f75a1-f485-4f60-91d5-87fef4b58bed\" (UID: \"531f75a1-f485-4f60-91d5-87fef4b58bed\") " Jul 12 18:47:26.446608 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:47:26.446306 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/531f75a1-f485-4f60-91d5-87fef4b58bed-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "531f75a1-f485-4f60-91d5-87fef4b58bed" (UID: "531f75a1-f485-4f60-91d5-87fef4b58bed"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:47:26.446608 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:47:26.446287 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/531f75a1-f485-4f60-91d5-87fef4b58bed-workload-socket\") pod \"531f75a1-f485-4f60-91d5-87fef4b58bed\" (UID: \"531f75a1-f485-4f60-91d5-87fef4b58bed\") " Jul 12 18:47:26.446608 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:47:26.446392 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/531f75a1-f485-4f60-91d5-87fef4b58bed-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "531f75a1-f485-4f60-91d5-87fef4b58bed" (UID: "531f75a1-f485-4f60-91d5-87fef4b58bed"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:47:26.446608 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:47:26.446420 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/531f75a1-f485-4f60-91d5-87fef4b58bed-istio-envoy\") pod \"531f75a1-f485-4f60-91d5-87fef4b58bed\" (UID: \"531f75a1-f485-4f60-91d5-87fef4b58bed\") " Jul 12 18:47:26.446865 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:47:26.446617 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/531f75a1-f485-4f60-91d5-87fef4b58bed-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "531f75a1-f485-4f60-91d5-87fef4b58bed" (UID: "531f75a1-f485-4f60-91d5-87fef4b58bed"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:47:26.446865 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:47:26.446683 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/531f75a1-f485-4f60-91d5-87fef4b58bed-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "531f75a1-f485-4f60-91d5-87fef4b58bed" (UID: "531f75a1-f485-4f60-91d5-87fef4b58bed"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:47:26.446865 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:47:26.446859 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/531f75a1-f485-4f60-91d5-87fef4b58bed-istiod-ca-cert\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:47:26.447020 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:47:26.446879 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/531f75a1-f485-4f60-91d5-87fef4b58bed-workload-socket\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:47:26.447020 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:47:26.446895 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/531f75a1-f485-4f60-91d5-87fef4b58bed-workload-certs\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:47:26.447020 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:47:26.446908 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/531f75a1-f485-4f60-91d5-87fef4b58bed-credential-socket\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:47:26.447343 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:47:26.447319 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/531f75a1-f485-4f60-91d5-87fef4b58bed-istio-data" (OuterVolumeSpecName: "istio-data") pod "531f75a1-f485-4f60-91d5-87fef4b58bed" (UID: "531f75a1-f485-4f60-91d5-87fef4b58bed"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:47:26.448930 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:47:26.448894 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/531f75a1-f485-4f60-91d5-87fef4b58bed-kube-api-access-c8mn2" (OuterVolumeSpecName: "kube-api-access-c8mn2") pod "531f75a1-f485-4f60-91d5-87fef4b58bed" (UID: "531f75a1-f485-4f60-91d5-87fef4b58bed"). InnerVolumeSpecName "kube-api-access-c8mn2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:47:26.449043 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:47:26.448948 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/531f75a1-f485-4f60-91d5-87fef4b58bed-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "531f75a1-f485-4f60-91d5-87fef4b58bed" (UID: "531f75a1-f485-4f60-91d5-87fef4b58bed"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 18:47:26.449043 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:47:26.448958 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/531f75a1-f485-4f60-91d5-87fef4b58bed-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "531f75a1-f485-4f60-91d5-87fef4b58bed" (UID: "531f75a1-f485-4f60-91d5-87fef4b58bed"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:47:26.449043 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:47:26.448971 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/531f75a1-f485-4f60-91d5-87fef4b58bed-istio-token" (OuterVolumeSpecName: "istio-token") pod "531f75a1-f485-4f60-91d5-87fef4b58bed" (UID: "531f75a1-f485-4f60-91d5-87fef4b58bed"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:47:26.548274 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:47:26.548170 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/531f75a1-f485-4f60-91d5-87fef4b58bed-istio-envoy\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:47:26.548274 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:47:26.548211 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c8mn2\" (UniqueName: \"kubernetes.io/projected/531f75a1-f485-4f60-91d5-87fef4b58bed-kube-api-access-c8mn2\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:47:26.548274 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:47:26.548225 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/531f75a1-f485-4f60-91d5-87fef4b58bed-istio-podinfo\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:47:26.548274 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:47:26.548237 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/531f75a1-f485-4f60-91d5-87fef4b58bed-istio-data\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:47:26.548274 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:47:26.548249 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/531f75a1-f485-4f60-91d5-87fef4b58bed-istio-token\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:47:27.152991 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:47:27.152957 2577 generic.go:356] "Generic (PLEG): container finished" podID="531f75a1-f485-4f60-91d5-87fef4b58bed" containerID="333d949d080cd1d8161a6cc4f10734d1f8d2d26f9060beebe6e871c81b76590c" exitCode=0 Jul 12 18:47:27.153401 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:47:27.153006 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl" Jul 12 18:47:27.153401 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:47:27.153020 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl" event={"ID":"531f75a1-f485-4f60-91d5-87fef4b58bed","Type":"ContainerDied","Data":"333d949d080cd1d8161a6cc4f10734d1f8d2d26f9060beebe6e871c81b76590c"} Jul 12 18:47:27.153401 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:47:27.153062 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl" event={"ID":"531f75a1-f485-4f60-91d5-87fef4b58bed","Type":"ContainerDied","Data":"565d0f97b6a167851ad5d9a7eebeae35187eda7caffa07511720cd608232f604"} Jul 12 18:47:27.153401 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:47:27.153082 2577 scope.go:117] "RemoveContainer" containerID="333d949d080cd1d8161a6cc4f10734d1f8d2d26f9060beebe6e871c81b76590c" Jul 12 18:47:27.162050 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:47:27.162034 2577 scope.go:117] "RemoveContainer" containerID="6d1c97ad9965d7af2ffea1ca2e7f4fe688a048c6a4596aec5d46071dcfae56d5" Jul 12 18:47:27.170566 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:47:27.170548 2577 scope.go:117] "RemoveContainer" containerID="333d949d080cd1d8161a6cc4f10734d1f8d2d26f9060beebe6e871c81b76590c" Jul 12 18:47:27.170830 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:47:27.170809 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"333d949d080cd1d8161a6cc4f10734d1f8d2d26f9060beebe6e871c81b76590c\": container with ID starting with 333d949d080cd1d8161a6cc4f10734d1f8d2d26f9060beebe6e871c81b76590c not found: ID does not exist" containerID="333d949d080cd1d8161a6cc4f10734d1f8d2d26f9060beebe6e871c81b76590c" Jul 12 18:47:27.170924 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:47:27.170838 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"333d949d080cd1d8161a6cc4f10734d1f8d2d26f9060beebe6e871c81b76590c"} err="failed to get container status \"333d949d080cd1d8161a6cc4f10734d1f8d2d26f9060beebe6e871c81b76590c\": rpc error: code = NotFound desc = could not find container \"333d949d080cd1d8161a6cc4f10734d1f8d2d26f9060beebe6e871c81b76590c\": container with ID starting with 333d949d080cd1d8161a6cc4f10734d1f8d2d26f9060beebe6e871c81b76590c not found: ID does not exist" Jul 12 18:47:27.170924 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:47:27.170853 2577 scope.go:117] "RemoveContainer" containerID="6d1c97ad9965d7af2ffea1ca2e7f4fe688a048c6a4596aec5d46071dcfae56d5" Jul 12 18:47:27.171234 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:47:27.171208 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d1c97ad9965d7af2ffea1ca2e7f4fe688a048c6a4596aec5d46071dcfae56d5\": container with ID starting with 6d1c97ad9965d7af2ffea1ca2e7f4fe688a048c6a4596aec5d46071dcfae56d5 not found: ID does not exist" containerID="6d1c97ad9965d7af2ffea1ca2e7f4fe688a048c6a4596aec5d46071dcfae56d5" Jul 12 18:47:27.171330 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:47:27.171243 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d1c97ad9965d7af2ffea1ca2e7f4fe688a048c6a4596aec5d46071dcfae56d5"} err="failed to get container status \"6d1c97ad9965d7af2ffea1ca2e7f4fe688a048c6a4596aec5d46071dcfae56d5\": rpc error: code = NotFound desc = could not find container \"6d1c97ad9965d7af2ffea1ca2e7f4fe688a048c6a4596aec5d46071dcfae56d5\": container with ID starting with 6d1c97ad9965d7af2ffea1ca2e7f4fe688a048c6a4596aec5d46071dcfae56d5 not found: ID does not exist" Jul 12 18:47:27.173972 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:47:27.173953 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl"] Jul 12 18:47:27.180288 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:47:27.180264 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-0328d158-openshift-default-66c4dfb75f-wfdgl"] Jul 12 18:47:29.076871 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:47:29.076840 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="531f75a1-f485-4f60-91d5-87fef4b58bed" path="/var/lib/kubelet/pods/531f75a1-f485-4f60-91d5-87fef4b58bed/volumes" Jul 12 18:48:15.995214 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:15.995178 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8"] Jul 12 18:48:15.995811 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:15.995703 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="531f75a1-f485-4f60-91d5-87fef4b58bed" containerName="istio-proxy" Jul 12 18:48:15.995811 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:15.995723 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="531f75a1-f485-4f60-91d5-87fef4b58bed" containerName="istio-proxy" Jul 12 18:48:15.996316 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:15.995833 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="531f75a1-f485-4f60-91d5-87fef4b58bed" containerName="istio-proxy" Jul 12 18:48:16.000182 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:16.000153 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8" Jul 12 18:48:16.003069 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:16.003048 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-14e82f-openshift-default-dockercfg-kpmrq\"" Jul 12 18:48:16.013517 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:16.013493 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8"] Jul 12 18:48:16.073390 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:16.073352 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8daa20dc-98ec-493b-a530-eaa278c707f3-workload-certs\") pod \"e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8\" (UID: \"8daa20dc-98ec-493b-a530-eaa278c707f3\") " pod="openshift-ingress/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8" Jul 12 18:48:16.073390 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:16.073383 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8daa20dc-98ec-493b-a530-eaa278c707f3-istio-podinfo\") pod \"e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8\" (UID: \"8daa20dc-98ec-493b-a530-eaa278c707f3\") " pod="openshift-ingress/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8" Jul 12 18:48:16.073672 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:16.073404 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8phm\" (UniqueName: \"kubernetes.io/projected/8daa20dc-98ec-493b-a530-eaa278c707f3-kube-api-access-g8phm\") pod \"e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8\" (UID: \"8daa20dc-98ec-493b-a530-eaa278c707f3\") " pod="openshift-ingress/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8" Jul 12 18:48:16.073672 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:16.073516 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8daa20dc-98ec-493b-a530-eaa278c707f3-istio-envoy\") pod \"e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8\" (UID: \"8daa20dc-98ec-493b-a530-eaa278c707f3\") " pod="openshift-ingress/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8" Jul 12 18:48:16.073672 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:16.073557 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8daa20dc-98ec-493b-a530-eaa278c707f3-istio-data\") pod \"e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8\" (UID: \"8daa20dc-98ec-493b-a530-eaa278c707f3\") " pod="openshift-ingress/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8" Jul 12 18:48:16.073672 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:16.073619 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8daa20dc-98ec-493b-a530-eaa278c707f3-istiod-ca-cert\") pod \"e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8\" (UID: \"8daa20dc-98ec-493b-a530-eaa278c707f3\") " pod="openshift-ingress/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8" Jul 12 18:48:16.073672 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:16.073667 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8daa20dc-98ec-493b-a530-eaa278c707f3-workload-socket\") pod \"e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8\" (UID: \"8daa20dc-98ec-493b-a530-eaa278c707f3\") " pod="openshift-ingress/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8" Jul 12 18:48:16.073950 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:16.073685 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8daa20dc-98ec-493b-a530-eaa278c707f3-istio-token\") pod \"e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8\" (UID: \"8daa20dc-98ec-493b-a530-eaa278c707f3\") " pod="openshift-ingress/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8" Jul 12 18:48:16.073950 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:16.073714 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8daa20dc-98ec-493b-a530-eaa278c707f3-credential-socket\") pod \"e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8\" (UID: \"8daa20dc-98ec-493b-a530-eaa278c707f3\") " pod="openshift-ingress/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8" Jul 12 18:48:16.175050 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:16.174997 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8daa20dc-98ec-493b-a530-eaa278c707f3-workload-certs\") pod \"e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8\" (UID: \"8daa20dc-98ec-493b-a530-eaa278c707f3\") " pod="openshift-ingress/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8" Jul 12 18:48:16.175050 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:16.175055 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8daa20dc-98ec-493b-a530-eaa278c707f3-istio-podinfo\") pod \"e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8\" (UID: \"8daa20dc-98ec-493b-a530-eaa278c707f3\") " pod="openshift-ingress/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8" Jul 12 18:48:16.175321 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:16.175081 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8phm\" (UniqueName: \"kubernetes.io/projected/8daa20dc-98ec-493b-a530-eaa278c707f3-kube-api-access-g8phm\") pod \"e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8\" (UID: \"8daa20dc-98ec-493b-a530-eaa278c707f3\") " pod="openshift-ingress/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8" Jul 12 18:48:16.175321 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:16.175152 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8daa20dc-98ec-493b-a530-eaa278c707f3-istio-envoy\") pod \"e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8\" (UID: \"8daa20dc-98ec-493b-a530-eaa278c707f3\") " pod="openshift-ingress/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8" Jul 12 18:48:16.175321 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:16.175178 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8daa20dc-98ec-493b-a530-eaa278c707f3-istio-data\") pod \"e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8\" (UID: \"8daa20dc-98ec-493b-a530-eaa278c707f3\") " pod="openshift-ingress/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8" Jul 12 18:48:16.175492 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:16.175450 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8daa20dc-98ec-493b-a530-eaa278c707f3-workload-certs\") pod \"e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8\" (UID: \"8daa20dc-98ec-493b-a530-eaa278c707f3\") " pod="openshift-ingress/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8" Jul 12 18:48:16.175492 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:16.175462 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8daa20dc-98ec-493b-a530-eaa278c707f3-istiod-ca-cert\") pod \"e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8\" (UID: \"8daa20dc-98ec-493b-a530-eaa278c707f3\") " pod="openshift-ingress/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8" Jul 12 18:48:16.175600 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:16.175528 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8daa20dc-98ec-493b-a530-eaa278c707f3-workload-socket\") pod \"e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8\" (UID: \"8daa20dc-98ec-493b-a530-eaa278c707f3\") " pod="openshift-ingress/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8" Jul 12 18:48:16.175600 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:16.175565 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8daa20dc-98ec-493b-a530-eaa278c707f3-istio-token\") pod \"e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8\" (UID: \"8daa20dc-98ec-493b-a530-eaa278c707f3\") " pod="openshift-ingress/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8" Jul 12 18:48:16.175740 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:16.175604 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8daa20dc-98ec-493b-a530-eaa278c707f3-credential-socket\") pod \"e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8\" (UID: \"8daa20dc-98ec-493b-a530-eaa278c707f3\") " pod="openshift-ingress/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8" Jul 12 18:48:16.175740 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:16.175726 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8daa20dc-98ec-493b-a530-eaa278c707f3-istio-data\") pod \"e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8\" (UID: \"8daa20dc-98ec-493b-a530-eaa278c707f3\") " pod="openshift-ingress/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8" Jul 12 18:48:16.175977 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:16.175951 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8daa20dc-98ec-493b-a530-eaa278c707f3-credential-socket\") pod \"e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8\" (UID: \"8daa20dc-98ec-493b-a530-eaa278c707f3\") " pod="openshift-ingress/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8" Jul 12 18:48:16.176168 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:16.176142 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8daa20dc-98ec-493b-a530-eaa278c707f3-workload-socket\") pod \"e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8\" (UID: \"8daa20dc-98ec-493b-a530-eaa278c707f3\") " pod="openshift-ingress/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8" Jul 12 18:48:16.176168 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:16.176159 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8daa20dc-98ec-493b-a530-eaa278c707f3-istiod-ca-cert\") pod \"e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8\" (UID: \"8daa20dc-98ec-493b-a530-eaa278c707f3\") " pod="openshift-ingress/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8" Jul 12 18:48:16.177713 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:16.177694 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8daa20dc-98ec-493b-a530-eaa278c707f3-istio-podinfo\") pod \"e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8\" (UID: \"8daa20dc-98ec-493b-a530-eaa278c707f3\") " pod="openshift-ingress/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8" Jul 12 18:48:16.177805 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:16.177760 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8daa20dc-98ec-493b-a530-eaa278c707f3-istio-envoy\") pod \"e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8\" (UID: \"8daa20dc-98ec-493b-a530-eaa278c707f3\") " pod="openshift-ingress/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8" Jul 12 18:48:16.183229 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:16.183199 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8daa20dc-98ec-493b-a530-eaa278c707f3-istio-token\") pod \"e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8\" (UID: \"8daa20dc-98ec-493b-a530-eaa278c707f3\") " pod="openshift-ingress/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8" Jul 12 18:48:16.183802 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:16.183780 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8phm\" (UniqueName: \"kubernetes.io/projected/8daa20dc-98ec-493b-a530-eaa278c707f3-kube-api-access-g8phm\") pod \"e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8\" (UID: \"8daa20dc-98ec-493b-a530-eaa278c707f3\") " pod="openshift-ingress/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8" Jul 12 18:48:16.310797 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:16.310701 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8" Jul 12 18:48:16.462360 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:16.462333 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8"] Jul 12 18:48:16.464273 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:48:16.464235 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8daa20dc_98ec_493b_a530_eaa278c707f3.slice/crio-78ef650f76310ad5748ddce87c174bad2488b4e99194661776609d4beaf2effa WatchSource:0}: Error finding container 78ef650f76310ad5748ddce87c174bad2488b4e99194661776609d4beaf2effa: Status 404 returned error can't find the container with id 78ef650f76310ad5748ddce87c174bad2488b4e99194661776609d4beaf2effa Jul 12 18:48:16.466430 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:16.466399 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 12 18:48:16.466514 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:16.466464 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 12 18:48:16.466514 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:16.466508 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 12 18:48:17.411146 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:17.411107 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8" event={"ID":"8daa20dc-98ec-493b-a530-eaa278c707f3","Type":"ContainerStarted","Data":"18333f2ba2ee8dbdf3091bc02f5bb1cc1ab829a61de54eeed5fd87b0c127f44a"} Jul 12 18:48:17.411539 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:17.411169 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8" event={"ID":"8daa20dc-98ec-493b-a530-eaa278c707f3","Type":"ContainerStarted","Data":"78ef650f76310ad5748ddce87c174bad2488b4e99194661776609d4beaf2effa"} Jul 12 18:48:17.431753 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:17.431705 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8" podStartSLOduration=2.431690276 podStartE2EDuration="2.431690276s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:48:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:48:17.429017561 +0000 UTC m=+1870.555922368" watchObservedRunningTime="2026-07-12 18:48:17.431690276 +0000 UTC m=+1870.558595093" Jul 12 18:48:18.311025 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:18.310986 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8" Jul 12 18:48:18.316196 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:18.316169 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8" Jul 12 18:48:18.416059 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:18.416028 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8" Jul 12 18:48:18.417012 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:18.416995 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8" Jul 12 18:48:27.410497 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:27.410462 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:48:34.458803 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:34.458759 2577 patch_prober.go:28] interesting pod/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 12 18:48:34.459264 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:34.458840 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8" podUID="8daa20dc-98ec-493b-a530-eaa278c707f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 12 18:48:35.505180 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:35.505147 2577 generic.go:356] "Generic (PLEG): container finished" podID="8daa20dc-98ec-493b-a530-eaa278c707f3" containerID="18333f2ba2ee8dbdf3091bc02f5bb1cc1ab829a61de54eeed5fd87b0c127f44a" exitCode=0 Jul 12 18:48:35.505584 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:35.505207 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8" event={"ID":"8daa20dc-98ec-493b-a530-eaa278c707f3","Type":"ContainerDied","Data":"18333f2ba2ee8dbdf3091bc02f5bb1cc1ab829a61de54eeed5fd87b0c127f44a"} Jul 12 18:48:35.505584 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:35.505572 2577 scope.go:117] "RemoveContainer" containerID="18333f2ba2ee8dbdf3091bc02f5bb1cc1ab829a61de54eeed5fd87b0c127f44a" Jul 12 18:48:35.506544 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:35.506514 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 12 18:48:35.506607 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:35.506594 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 12 18:48:35.506677 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:35.506664 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 12 18:48:36.511762 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:36.511729 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8" event={"ID":"8daa20dc-98ec-493b-a530-eaa278c707f3","Type":"ContainerStarted","Data":"73d9e3c936a3f5a15cda2b48808044336a08fd09429f870e52e8905070ebadc1"} Jul 12 18:48:37.311133 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:37.311092 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8" Jul 12 18:48:37.311369 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:37.311346 2577 patch_prober.go:28] interesting pod/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Jul 12 18:48:37.311429 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:37.311409 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8" podUID="8daa20dc-98ec-493b-a530-eaa278c707f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Jul 12 18:48:38.311441 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:38.311408 2577 patch_prober.go:28] interesting pod/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Jul 12 18:48:38.311872 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:38.311461 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8" podUID="8daa20dc-98ec-493b-a530-eaa278c707f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Jul 12 18:48:39.311128 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:39.311092 2577 patch_prober.go:28] interesting pod/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Jul 12 18:48:39.311278 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:39.311149 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8" podUID="8daa20dc-98ec-493b-a530-eaa278c707f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Jul 12 18:48:39.882902 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:39.882863 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:48:40.314542 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:40.314514 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8" Jul 12 18:48:40.314867 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:40.314846 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8" Jul 12 18:48:40.315525 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:40.315506 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8" Jul 12 18:48:50.881354 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:48:50.881324 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:49:17.482090 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:49:17.482051 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:49:35.486748 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:49:35.486710 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:50:13.444925 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:13.444892 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8"] Jul 12 18:50:13.445468 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:13.445172 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8" podUID="8daa20dc-98ec-493b-a530-eaa278c707f3" containerName="istio-proxy" containerID="cri-o://73d9e3c936a3f5a15cda2b48808044336a08fd09429f870e52e8905070ebadc1" gracePeriod=30 Jul 12 18:50:17.780642 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:17.780596 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:50:18.704771 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:18.704712 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8" Jul 12 18:50:18.793345 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:18.793315 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8daa20dc-98ec-493b-a530-eaa278c707f3-istio-token\") pod \"8daa20dc-98ec-493b-a530-eaa278c707f3\" (UID: \"8daa20dc-98ec-493b-a530-eaa278c707f3\") " Jul 12 18:50:18.793345 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:18.793353 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g8phm\" (UniqueName: \"kubernetes.io/projected/8daa20dc-98ec-493b-a530-eaa278c707f3-kube-api-access-g8phm\") pod \"8daa20dc-98ec-493b-a530-eaa278c707f3\" (UID: \"8daa20dc-98ec-493b-a530-eaa278c707f3\") " Jul 12 18:50:18.793873 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:18.793379 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8daa20dc-98ec-493b-a530-eaa278c707f3-istio-podinfo\") pod \"8daa20dc-98ec-493b-a530-eaa278c707f3\" (UID: \"8daa20dc-98ec-493b-a530-eaa278c707f3\") " Jul 12 18:50:18.793873 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:18.793407 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8daa20dc-98ec-493b-a530-eaa278c707f3-istio-data\") pod \"8daa20dc-98ec-493b-a530-eaa278c707f3\" (UID: \"8daa20dc-98ec-493b-a530-eaa278c707f3\") " Jul 12 18:50:18.793873 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:18.793522 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8daa20dc-98ec-493b-a530-eaa278c707f3-workload-socket\") pod \"8daa20dc-98ec-493b-a530-eaa278c707f3\" (UID: \"8daa20dc-98ec-493b-a530-eaa278c707f3\") " Jul 12 18:50:18.793873 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:18.793552 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8daa20dc-98ec-493b-a530-eaa278c707f3-credential-socket\") pod \"8daa20dc-98ec-493b-a530-eaa278c707f3\" (UID: \"8daa20dc-98ec-493b-a530-eaa278c707f3\") " Jul 12 18:50:18.793873 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:18.793609 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8daa20dc-98ec-493b-a530-eaa278c707f3-workload-certs\") pod \"8daa20dc-98ec-493b-a530-eaa278c707f3\" (UID: \"8daa20dc-98ec-493b-a530-eaa278c707f3\") " Jul 12 18:50:18.793873 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:18.793661 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8daa20dc-98ec-493b-a530-eaa278c707f3-istio-envoy\") pod \"8daa20dc-98ec-493b-a530-eaa278c707f3\" (UID: \"8daa20dc-98ec-493b-a530-eaa278c707f3\") " Jul 12 18:50:18.793873 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:18.793701 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8daa20dc-98ec-493b-a530-eaa278c707f3-istiod-ca-cert\") pod \"8daa20dc-98ec-493b-a530-eaa278c707f3\" (UID: \"8daa20dc-98ec-493b-a530-eaa278c707f3\") " Jul 12 18:50:18.793873 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:18.793838 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8daa20dc-98ec-493b-a530-eaa278c707f3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8daa20dc-98ec-493b-a530-eaa278c707f3" (UID: "8daa20dc-98ec-493b-a530-eaa278c707f3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:50:18.793873 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:18.793871 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8daa20dc-98ec-493b-a530-eaa278c707f3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8daa20dc-98ec-493b-a530-eaa278c707f3" (UID: "8daa20dc-98ec-493b-a530-eaa278c707f3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:50:18.794346 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:18.794054 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8daa20dc-98ec-493b-a530-eaa278c707f3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8daa20dc-98ec-493b-a530-eaa278c707f3" (UID: "8daa20dc-98ec-493b-a530-eaa278c707f3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:50:18.794346 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:18.794098 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8daa20dc-98ec-493b-a530-eaa278c707f3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8daa20dc-98ec-493b-a530-eaa278c707f3" (UID: "8daa20dc-98ec-493b-a530-eaa278c707f3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:50:18.794346 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:18.794104 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8daa20dc-98ec-493b-a530-eaa278c707f3-workload-socket\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:50:18.794346 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:18.794125 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8daa20dc-98ec-493b-a530-eaa278c707f3-credential-socket\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:50:18.794569 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:18.794421 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8daa20dc-98ec-493b-a530-eaa278c707f3-istio-data" (OuterVolumeSpecName: "istio-data") pod "8daa20dc-98ec-493b-a530-eaa278c707f3" (UID: "8daa20dc-98ec-493b-a530-eaa278c707f3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:50:18.795810 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:18.795787 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8daa20dc-98ec-493b-a530-eaa278c707f3-kube-api-access-g8phm" (OuterVolumeSpecName: "kube-api-access-g8phm") pod "8daa20dc-98ec-493b-a530-eaa278c707f3" (UID: "8daa20dc-98ec-493b-a530-eaa278c707f3"). InnerVolumeSpecName "kube-api-access-g8phm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:50:18.795919 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:18.795836 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8daa20dc-98ec-493b-a530-eaa278c707f3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8daa20dc-98ec-493b-a530-eaa278c707f3" (UID: "8daa20dc-98ec-493b-a530-eaa278c707f3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:50:18.796011 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:18.795995 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8daa20dc-98ec-493b-a530-eaa278c707f3-istio-token" (OuterVolumeSpecName: "istio-token") pod "8daa20dc-98ec-493b-a530-eaa278c707f3" (UID: "8daa20dc-98ec-493b-a530-eaa278c707f3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:50:18.796054 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:18.795995 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8daa20dc-98ec-493b-a530-eaa278c707f3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8daa20dc-98ec-493b-a530-eaa278c707f3" (UID: "8daa20dc-98ec-493b-a530-eaa278c707f3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 18:50:18.895571 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:18.895539 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8daa20dc-98ec-493b-a530-eaa278c707f3-workload-certs\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:50:18.895571 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:18.895564 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8daa20dc-98ec-493b-a530-eaa278c707f3-istio-envoy\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:50:18.895571 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:18.895573 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8daa20dc-98ec-493b-a530-eaa278c707f3-istiod-ca-cert\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:50:18.895571 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:18.895582 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8daa20dc-98ec-493b-a530-eaa278c707f3-istio-token\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:50:18.895867 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:18.895591 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g8phm\" (UniqueName: \"kubernetes.io/projected/8daa20dc-98ec-493b-a530-eaa278c707f3-kube-api-access-g8phm\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:50:18.895867 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:18.895598 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8daa20dc-98ec-493b-a530-eaa278c707f3-istio-podinfo\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:50:18.895867 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:18.895607 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8daa20dc-98ec-493b-a530-eaa278c707f3-istio-data\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:50:19.049836 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:19.049750 2577 generic.go:356] "Generic (PLEG): container finished" podID="8daa20dc-98ec-493b-a530-eaa278c707f3" containerID="73d9e3c936a3f5a15cda2b48808044336a08fd09429f870e52e8905070ebadc1" exitCode=0 Jul 12 18:50:19.049836 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:19.049788 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8" Jul 12 18:50:19.049836 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:19.049803 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8" event={"ID":"8daa20dc-98ec-493b-a530-eaa278c707f3","Type":"ContainerDied","Data":"73d9e3c936a3f5a15cda2b48808044336a08fd09429f870e52e8905070ebadc1"} Jul 12 18:50:19.050061 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:19.049851 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8" event={"ID":"8daa20dc-98ec-493b-a530-eaa278c707f3","Type":"ContainerDied","Data":"78ef650f76310ad5748ddce87c174bad2488b4e99194661776609d4beaf2effa"} Jul 12 18:50:19.050061 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:19.049877 2577 scope.go:117] "RemoveContainer" containerID="73d9e3c936a3f5a15cda2b48808044336a08fd09429f870e52e8905070ebadc1" Jul 12 18:50:19.059230 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:19.059210 2577 scope.go:117] "RemoveContainer" containerID="18333f2ba2ee8dbdf3091bc02f5bb1cc1ab829a61de54eeed5fd87b0c127f44a" Jul 12 18:50:19.067205 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:19.067186 2577 scope.go:117] "RemoveContainer" containerID="73d9e3c936a3f5a15cda2b48808044336a08fd09429f870e52e8905070ebadc1" Jul 12 18:50:19.067445 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:50:19.067427 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73d9e3c936a3f5a15cda2b48808044336a08fd09429f870e52e8905070ebadc1\": container with ID starting with 73d9e3c936a3f5a15cda2b48808044336a08fd09429f870e52e8905070ebadc1 not found: ID does not exist" containerID="73d9e3c936a3f5a15cda2b48808044336a08fd09429f870e52e8905070ebadc1" Jul 12 18:50:19.067516 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:19.067457 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73d9e3c936a3f5a15cda2b48808044336a08fd09429f870e52e8905070ebadc1"} err="failed to get container status \"73d9e3c936a3f5a15cda2b48808044336a08fd09429f870e52e8905070ebadc1\": rpc error: code = NotFound desc = could not find container \"73d9e3c936a3f5a15cda2b48808044336a08fd09429f870e52e8905070ebadc1\": container with ID starting with 73d9e3c936a3f5a15cda2b48808044336a08fd09429f870e52e8905070ebadc1 not found: ID does not exist" Jul 12 18:50:19.067516 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:19.067484 2577 scope.go:117] "RemoveContainer" containerID="18333f2ba2ee8dbdf3091bc02f5bb1cc1ab829a61de54eeed5fd87b0c127f44a" Jul 12 18:50:19.067820 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:50:19.067793 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18333f2ba2ee8dbdf3091bc02f5bb1cc1ab829a61de54eeed5fd87b0c127f44a\": container with ID starting with 18333f2ba2ee8dbdf3091bc02f5bb1cc1ab829a61de54eeed5fd87b0c127f44a not found: ID does not exist" containerID="18333f2ba2ee8dbdf3091bc02f5bb1cc1ab829a61de54eeed5fd87b0c127f44a" Jul 12 18:50:19.068157 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:19.067825 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18333f2ba2ee8dbdf3091bc02f5bb1cc1ab829a61de54eeed5fd87b0c127f44a"} err="failed to get container status \"18333f2ba2ee8dbdf3091bc02f5bb1cc1ab829a61de54eeed5fd87b0c127f44a\": rpc error: code = NotFound desc = could not find container \"18333f2ba2ee8dbdf3091bc02f5bb1cc1ab829a61de54eeed5fd87b0c127f44a\": container with ID starting with 18333f2ba2ee8dbdf3091bc02f5bb1cc1ab829a61de54eeed5fd87b0c127f44a not found: ID does not exist" Jul 12 18:50:19.075956 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:19.075938 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8"] Jul 12 18:50:19.076059 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:19.075965 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-14e82f-openshift-default-7fd5b6459c-7h4f8"] Jul 12 18:50:20.938501 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:20.938467 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t"] Jul 12 18:50:20.938989 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:20.938973 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8daa20dc-98ec-493b-a530-eaa278c707f3" containerName="istio-proxy" Jul 12 18:50:20.938989 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:20.938990 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8daa20dc-98ec-493b-a530-eaa278c707f3" containerName="istio-proxy" Jul 12 18:50:20.939121 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:20.939011 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="531f75a1-f485-4f60-91d5-87fef4b58bed" containerName="istio-proxy" Jul 12 18:50:20.939121 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:20.939018 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="531f75a1-f485-4f60-91d5-87fef4b58bed" containerName="istio-proxy" Jul 12 18:50:20.939121 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:20.939029 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8daa20dc-98ec-493b-a530-eaa278c707f3" containerName="istio-proxy" Jul 12 18:50:20.939121 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:20.939035 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8daa20dc-98ec-493b-a530-eaa278c707f3" containerName="istio-proxy" Jul 12 18:50:20.939725 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:20.939700 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8daa20dc-98ec-493b-a530-eaa278c707f3" containerName="istio-proxy" Jul 12 18:50:20.939897 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:20.939885 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8daa20dc-98ec-493b-a530-eaa278c707f3" containerName="istio-proxy" Jul 12 18:50:20.939980 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:20.939971 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="531f75a1-f485-4f60-91d5-87fef4b58bed" containerName="istio-proxy" Jul 12 18:50:20.945758 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:20.945738 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t" Jul 12 18:50:20.948569 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:20.948540 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-675ba4-openshift-default-dockercfg-sxhdd\"" Jul 12 18:50:20.952619 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:20.952591 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t"] Jul 12 18:50:21.015902 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:21.015867 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19b67bd6-f4b0-4fab-bd1c-182a84625f32-istio-podinfo\") pod \"e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t\" (UID: \"19b67bd6-f4b0-4fab-bd1c-182a84625f32\") " pod="openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t" Jul 12 18:50:21.016099 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:21.015909 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19b67bd6-f4b0-4fab-bd1c-182a84625f32-workload-socket\") pod \"e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t\" (UID: \"19b67bd6-f4b0-4fab-bd1c-182a84625f32\") " pod="openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t" Jul 12 18:50:21.016099 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:21.015932 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19b67bd6-f4b0-4fab-bd1c-182a84625f32-istio-envoy\") pod \"e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t\" (UID: \"19b67bd6-f4b0-4fab-bd1c-182a84625f32\") " pod="openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t" Jul 12 18:50:21.016099 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:21.016048 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19b67bd6-f4b0-4fab-bd1c-182a84625f32-istio-data\") pod \"e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t\" (UID: \"19b67bd6-f4b0-4fab-bd1c-182a84625f32\") " pod="openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t" Jul 12 18:50:21.016099 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:21.016082 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19b67bd6-f4b0-4fab-bd1c-182a84625f32-workload-certs\") pod \"e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t\" (UID: \"19b67bd6-f4b0-4fab-bd1c-182a84625f32\") " pod="openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t" Jul 12 18:50:21.016099 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:21.016100 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19b67bd6-f4b0-4fab-bd1c-182a84625f32-istio-token\") pod \"e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t\" (UID: \"19b67bd6-f4b0-4fab-bd1c-182a84625f32\") " pod="openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t" Jul 12 18:50:21.016306 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:21.016115 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19b67bd6-f4b0-4fab-bd1c-182a84625f32-istiod-ca-cert\") pod \"e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t\" (UID: \"19b67bd6-f4b0-4fab-bd1c-182a84625f32\") " pod="openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t" Jul 12 18:50:21.016306 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:21.016147 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19b67bd6-f4b0-4fab-bd1c-182a84625f32-credential-socket\") pod \"e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t\" (UID: \"19b67bd6-f4b0-4fab-bd1c-182a84625f32\") " pod="openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t" Jul 12 18:50:21.016306 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:21.016163 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwbjk\" (UniqueName: \"kubernetes.io/projected/19b67bd6-f4b0-4fab-bd1c-182a84625f32-kube-api-access-nwbjk\") pod \"e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t\" (UID: \"19b67bd6-f4b0-4fab-bd1c-182a84625f32\") " pod="openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t" Jul 12 18:50:21.075721 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:21.075685 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8daa20dc-98ec-493b-a530-eaa278c707f3" path="/var/lib/kubelet/pods/8daa20dc-98ec-493b-a530-eaa278c707f3/volumes" Jul 12 18:50:21.117721 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:21.117683 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19b67bd6-f4b0-4fab-bd1c-182a84625f32-istio-podinfo\") pod \"e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t\" (UID: \"19b67bd6-f4b0-4fab-bd1c-182a84625f32\") " pod="openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t" Jul 12 18:50:21.117721 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:21.117722 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19b67bd6-f4b0-4fab-bd1c-182a84625f32-workload-socket\") pod \"e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t\" (UID: \"19b67bd6-f4b0-4fab-bd1c-182a84625f32\") " pod="openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t" Jul 12 18:50:21.117959 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:21.117742 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19b67bd6-f4b0-4fab-bd1c-182a84625f32-istio-envoy\") pod \"e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t\" (UID: \"19b67bd6-f4b0-4fab-bd1c-182a84625f32\") " pod="openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t" Jul 12 18:50:21.117959 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:21.117792 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19b67bd6-f4b0-4fab-bd1c-182a84625f32-istio-data\") pod \"e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t\" (UID: \"19b67bd6-f4b0-4fab-bd1c-182a84625f32\") " pod="openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t" Jul 12 18:50:21.117959 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:21.117826 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19b67bd6-f4b0-4fab-bd1c-182a84625f32-workload-certs\") pod \"e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t\" (UID: \"19b67bd6-f4b0-4fab-bd1c-182a84625f32\") " pod="openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t" Jul 12 18:50:21.117959 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:21.117850 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19b67bd6-f4b0-4fab-bd1c-182a84625f32-istio-token\") pod \"e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t\" (UID: \"19b67bd6-f4b0-4fab-bd1c-182a84625f32\") " pod="openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t" Jul 12 18:50:21.117959 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:21.117873 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19b67bd6-f4b0-4fab-bd1c-182a84625f32-istiod-ca-cert\") pod \"e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t\" (UID: \"19b67bd6-f4b0-4fab-bd1c-182a84625f32\") " pod="openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t" Jul 12 18:50:21.117959 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:21.117933 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19b67bd6-f4b0-4fab-bd1c-182a84625f32-credential-socket\") pod \"e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t\" (UID: \"19b67bd6-f4b0-4fab-bd1c-182a84625f32\") " pod="openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t" Jul 12 18:50:21.118275 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:21.118054 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwbjk\" (UniqueName: \"kubernetes.io/projected/19b67bd6-f4b0-4fab-bd1c-182a84625f32-kube-api-access-nwbjk\") pod \"e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t\" (UID: \"19b67bd6-f4b0-4fab-bd1c-182a84625f32\") " pod="openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t" Jul 12 18:50:21.118275 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:21.118237 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19b67bd6-f4b0-4fab-bd1c-182a84625f32-workload-socket\") pod \"e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t\" (UID: \"19b67bd6-f4b0-4fab-bd1c-182a84625f32\") " pod="openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t" Jul 12 18:50:21.118384 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:21.118296 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19b67bd6-f4b0-4fab-bd1c-182a84625f32-istio-data\") pod \"e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t\" (UID: \"19b67bd6-f4b0-4fab-bd1c-182a84625f32\") " pod="openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t" Jul 12 18:50:21.118384 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:21.118335 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19b67bd6-f4b0-4fab-bd1c-182a84625f32-workload-certs\") pod \"e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t\" (UID: \"19b67bd6-f4b0-4fab-bd1c-182a84625f32\") " pod="openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t" Jul 12 18:50:21.118600 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:21.118581 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19b67bd6-f4b0-4fab-bd1c-182a84625f32-credential-socket\") pod \"e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t\" (UID: \"19b67bd6-f4b0-4fab-bd1c-182a84625f32\") " pod="openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t" Jul 12 18:50:21.118788 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:21.118769 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19b67bd6-f4b0-4fab-bd1c-182a84625f32-istiod-ca-cert\") pod \"e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t\" (UID: \"19b67bd6-f4b0-4fab-bd1c-182a84625f32\") " pod="openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t" Jul 12 18:50:21.120061 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:21.120037 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19b67bd6-f4b0-4fab-bd1c-182a84625f32-istio-envoy\") pod \"e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t\" (UID: \"19b67bd6-f4b0-4fab-bd1c-182a84625f32\") " pod="openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t" Jul 12 18:50:21.120693 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:21.120674 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19b67bd6-f4b0-4fab-bd1c-182a84625f32-istio-podinfo\") pod \"e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t\" (UID: \"19b67bd6-f4b0-4fab-bd1c-182a84625f32\") " pod="openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t" Jul 12 18:50:21.125696 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:21.125673 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19b67bd6-f4b0-4fab-bd1c-182a84625f32-istio-token\") pod \"e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t\" (UID: \"19b67bd6-f4b0-4fab-bd1c-182a84625f32\") " pod="openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t" Jul 12 18:50:21.125775 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:21.125760 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwbjk\" (UniqueName: \"kubernetes.io/projected/19b67bd6-f4b0-4fab-bd1c-182a84625f32-kube-api-access-nwbjk\") pod \"e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t\" (UID: \"19b67bd6-f4b0-4fab-bd1c-182a84625f32\") " pod="openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t" Jul 12 18:50:21.258306 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:21.258211 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t" Jul 12 18:50:21.397778 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:21.397754 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t"] Jul 12 18:50:21.399366 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:50:21.399340 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19b67bd6_f4b0_4fab_bd1c_182a84625f32.slice/crio-5edc527371e92944e31f46f072ff8a91c2252bf6b7cfee290d4cc97961ef45e5 WatchSource:0}: Error finding container 5edc527371e92944e31f46f072ff8a91c2252bf6b7cfee290d4cc97961ef45e5: Status 404 returned error can't find the container with id 5edc527371e92944e31f46f072ff8a91c2252bf6b7cfee290d4cc97961ef45e5 Jul 12 18:50:21.401591 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:21.401546 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 12 18:50:21.401698 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:21.401658 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 12 18:50:21.401744 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:21.401704 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 12 18:50:22.067194 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:22.067160 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t" event={"ID":"19b67bd6-f4b0-4fab-bd1c-182a84625f32","Type":"ContainerStarted","Data":"f32d11b40f0bd8d527ea7e8db097a1fe7c56400bf637f8a3de38b3c152c04316"} Jul 12 18:50:22.067620 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:22.067218 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t" event={"ID":"19b67bd6-f4b0-4fab-bd1c-182a84625f32","Type":"ContainerStarted","Data":"5edc527371e92944e31f46f072ff8a91c2252bf6b7cfee290d4cc97961ef45e5"} Jul 12 18:50:22.089428 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:22.089382 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t" podStartSLOduration=2.089365559 podStartE2EDuration="2.089365559s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:50:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:50:22.086177595 +0000 UTC m=+1995.213082416" watchObservedRunningTime="2026-07-12 18:50:22.089365559 +0000 UTC m=+1995.216270344" Jul 12 18:50:22.258596 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:22.258560 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t" Jul 12 18:50:22.263546 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:22.263522 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t" Jul 12 18:50:23.075394 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:23.075369 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t" Jul 12 18:50:23.075776 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:23.075430 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t" Jul 12 18:50:32.493154 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:32.493119 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:50:39.114818 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:39.114772 2577 patch_prober.go:28] interesting pod/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 12 18:50:39.115239 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:39.114836 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t" podUID="19b67bd6-f4b0-4fab-bd1c-182a84625f32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 12 18:50:41.181824 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:41.181712 2577 generic.go:356] "Generic (PLEG): container finished" podID="19b67bd6-f4b0-4fab-bd1c-182a84625f32" containerID="f32d11b40f0bd8d527ea7e8db097a1fe7c56400bf637f8a3de38b3c152c04316" exitCode=0 Jul 12 18:50:41.181824 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:41.181777 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t" event={"ID":"19b67bd6-f4b0-4fab-bd1c-182a84625f32","Type":"ContainerDied","Data":"f32d11b40f0bd8d527ea7e8db097a1fe7c56400bf637f8a3de38b3c152c04316"} Jul 12 18:50:41.182395 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:41.182276 2577 scope.go:117] "RemoveContainer" containerID="f32d11b40f0bd8d527ea7e8db097a1fe7c56400bf637f8a3de38b3c152c04316" Jul 12 18:50:41.183481 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:41.183438 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 12 18:50:41.183579 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:41.183547 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 12 18:50:41.183656 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:41.183610 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 12 18:50:42.189022 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:42.188988 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t" event={"ID":"19b67bd6-f4b0-4fab-bd1c-182a84625f32","Type":"ContainerStarted","Data":"63913710c738ec5083b020f8877774ef6aadc922cb2e345e98040a5a03a721ad"} Jul 12 18:50:42.258812 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:42.258774 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t" Jul 12 18:50:42.259011 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:42.258990 2577 patch_prober.go:28] interesting pod/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Jul 12 18:50:42.259065 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:42.259042 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t" podUID="19b67bd6-f4b0-4fab-bd1c-182a84625f32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Jul 12 18:50:43.259257 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:43.259224 2577 patch_prober.go:28] interesting pod/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Jul 12 18:50:43.259681 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:43.259269 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t" podUID="19b67bd6-f4b0-4fab-bd1c-182a84625f32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Jul 12 18:50:44.258553 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:44.258520 2577 patch_prober.go:28] interesting pod/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Jul 12 18:50:44.258856 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:44.258571 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t" podUID="19b67bd6-f4b0-4fab-bd1c-182a84625f32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Jul 12 18:50:45.259040 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:45.258999 2577 patch_prober.go:28] interesting pod/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Jul 12 18:50:45.259523 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:45.259071 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t" podUID="19b67bd6-f4b0-4fab-bd1c-182a84625f32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Jul 12 18:50:46.262641 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:46.262603 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t" Jul 12 18:50:46.263090 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:46.262884 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t" Jul 12 18:50:46.263640 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:46.263605 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t" Jul 12 18:50:47.680763 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:47.680730 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:50:50.341446 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:50.341413 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t"] Jul 12 18:50:50.341958 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:50.341613 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t" podUID="19b67bd6-f4b0-4fab-bd1c-182a84625f32" containerName="istio-proxy" containerID="cri-o://63913710c738ec5083b020f8877774ef6aadc922cb2e345e98040a5a03a721ad" gracePeriod=30 Jul 12 18:50:55.597966 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:55.597941 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t" Jul 12 18:50:55.656053 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:55.655967 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19b67bd6-f4b0-4fab-bd1c-182a84625f32-istio-envoy\") pod \"19b67bd6-f4b0-4fab-bd1c-182a84625f32\" (UID: \"19b67bd6-f4b0-4fab-bd1c-182a84625f32\") " Jul 12 18:50:55.656053 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:55.656011 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19b67bd6-f4b0-4fab-bd1c-182a84625f32-credential-socket\") pod \"19b67bd6-f4b0-4fab-bd1c-182a84625f32\" (UID: \"19b67bd6-f4b0-4fab-bd1c-182a84625f32\") " Jul 12 18:50:55.656275 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:55.656062 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19b67bd6-f4b0-4fab-bd1c-182a84625f32-workload-certs\") pod \"19b67bd6-f4b0-4fab-bd1c-182a84625f32\" (UID: \"19b67bd6-f4b0-4fab-bd1c-182a84625f32\") " Jul 12 18:50:55.656275 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:55.656118 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nwbjk\" (UniqueName: \"kubernetes.io/projected/19b67bd6-f4b0-4fab-bd1c-182a84625f32-kube-api-access-nwbjk\") pod \"19b67bd6-f4b0-4fab-bd1c-182a84625f32\" (UID: \"19b67bd6-f4b0-4fab-bd1c-182a84625f32\") " Jul 12 18:50:55.656275 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:55.656169 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19b67bd6-f4b0-4fab-bd1c-182a84625f32-istio-token\") pod \"19b67bd6-f4b0-4fab-bd1c-182a84625f32\" (UID: \"19b67bd6-f4b0-4fab-bd1c-182a84625f32\") " Jul 12 18:50:55.656275 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:55.656202 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19b67bd6-f4b0-4fab-bd1c-182a84625f32-istio-podinfo\") pod \"19b67bd6-f4b0-4fab-bd1c-182a84625f32\" (UID: \"19b67bd6-f4b0-4fab-bd1c-182a84625f32\") " Jul 12 18:50:55.656275 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:55.656224 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19b67bd6-f4b0-4fab-bd1c-182a84625f32-istiod-ca-cert\") pod \"19b67bd6-f4b0-4fab-bd1c-182a84625f32\" (UID: \"19b67bd6-f4b0-4fab-bd1c-182a84625f32\") " Jul 12 18:50:55.656275 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:55.656256 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19b67bd6-f4b0-4fab-bd1c-182a84625f32-workload-socket\") pod \"19b67bd6-f4b0-4fab-bd1c-182a84625f32\" (UID: \"19b67bd6-f4b0-4fab-bd1c-182a84625f32\") " Jul 12 18:50:55.656658 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:55.656313 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19b67bd6-f4b0-4fab-bd1c-182a84625f32-istio-data\") pod \"19b67bd6-f4b0-4fab-bd1c-182a84625f32\" (UID: \"19b67bd6-f4b0-4fab-bd1c-182a84625f32\") " Jul 12 18:50:55.656658 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:55.656344 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19b67bd6-f4b0-4fab-bd1c-182a84625f32-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "19b67bd6-f4b0-4fab-bd1c-182a84625f32" (UID: "19b67bd6-f4b0-4fab-bd1c-182a84625f32"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:50:55.656658 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:55.656361 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19b67bd6-f4b0-4fab-bd1c-182a84625f32-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "19b67bd6-f4b0-4fab-bd1c-182a84625f32" (UID: "19b67bd6-f4b0-4fab-bd1c-182a84625f32"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:50:55.656658 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:55.656609 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19b67bd6-f4b0-4fab-bd1c-182a84625f32-credential-socket\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:50:55.656868 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:55.656675 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19b67bd6-f4b0-4fab-bd1c-182a84625f32-workload-certs\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:50:55.656868 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:55.656607 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19b67bd6-f4b0-4fab-bd1c-182a84625f32-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "19b67bd6-f4b0-4fab-bd1c-182a84625f32" (UID: "19b67bd6-f4b0-4fab-bd1c-182a84625f32"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:50:55.656868 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:55.656620 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19b67bd6-f4b0-4fab-bd1c-182a84625f32-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "19b67bd6-f4b0-4fab-bd1c-182a84625f32" (UID: "19b67bd6-f4b0-4fab-bd1c-182a84625f32"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:50:55.657328 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:55.657307 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19b67bd6-f4b0-4fab-bd1c-182a84625f32-istio-data" (OuterVolumeSpecName: "istio-data") pod "19b67bd6-f4b0-4fab-bd1c-182a84625f32" (UID: "19b67bd6-f4b0-4fab-bd1c-182a84625f32"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:50:55.658208 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:55.658183 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19b67bd6-f4b0-4fab-bd1c-182a84625f32-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "19b67bd6-f4b0-4fab-bd1c-182a84625f32" (UID: "19b67bd6-f4b0-4fab-bd1c-182a84625f32"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:50:55.658524 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:55.658502 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19b67bd6-f4b0-4fab-bd1c-182a84625f32-kube-api-access-nwbjk" (OuterVolumeSpecName: "kube-api-access-nwbjk") pod "19b67bd6-f4b0-4fab-bd1c-182a84625f32" (UID: "19b67bd6-f4b0-4fab-bd1c-182a84625f32"). InnerVolumeSpecName "kube-api-access-nwbjk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:50:55.658818 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:55.658797 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/19b67bd6-f4b0-4fab-bd1c-182a84625f32-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "19b67bd6-f4b0-4fab-bd1c-182a84625f32" (UID: "19b67bd6-f4b0-4fab-bd1c-182a84625f32"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 18:50:55.658876 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:55.658797 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19b67bd6-f4b0-4fab-bd1c-182a84625f32-istio-token" (OuterVolumeSpecName: "istio-token") pod "19b67bd6-f4b0-4fab-bd1c-182a84625f32" (UID: "19b67bd6-f4b0-4fab-bd1c-182a84625f32"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:50:55.758122 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:55.758080 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19b67bd6-f4b0-4fab-bd1c-182a84625f32-istio-token\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:50:55.758122 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:55.758115 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19b67bd6-f4b0-4fab-bd1c-182a84625f32-istio-podinfo\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:50:55.758122 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:55.758128 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19b67bd6-f4b0-4fab-bd1c-182a84625f32-istiod-ca-cert\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:50:55.758372 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:55.758139 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19b67bd6-f4b0-4fab-bd1c-182a84625f32-workload-socket\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:50:55.758372 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:55.758151 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19b67bd6-f4b0-4fab-bd1c-182a84625f32-istio-data\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:50:55.758372 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:55.758162 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19b67bd6-f4b0-4fab-bd1c-182a84625f32-istio-envoy\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:50:55.758372 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:55.758174 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nwbjk\" (UniqueName: \"kubernetes.io/projected/19b67bd6-f4b0-4fab-bd1c-182a84625f32-kube-api-access-nwbjk\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:50:56.267415 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:56.267380 2577 generic.go:356] "Generic (PLEG): container finished" podID="19b67bd6-f4b0-4fab-bd1c-182a84625f32" containerID="63913710c738ec5083b020f8877774ef6aadc922cb2e345e98040a5a03a721ad" exitCode=0 Jul 12 18:50:56.267597 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:56.267424 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t" Jul 12 18:50:56.267597 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:56.267468 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t" event={"ID":"19b67bd6-f4b0-4fab-bd1c-182a84625f32","Type":"ContainerDied","Data":"63913710c738ec5083b020f8877774ef6aadc922cb2e345e98040a5a03a721ad"} Jul 12 18:50:56.267597 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:56.267526 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t" event={"ID":"19b67bd6-f4b0-4fab-bd1c-182a84625f32","Type":"ContainerDied","Data":"5edc527371e92944e31f46f072ff8a91c2252bf6b7cfee290d4cc97961ef45e5"} Jul 12 18:50:56.267597 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:56.267553 2577 scope.go:117] "RemoveContainer" containerID="63913710c738ec5083b020f8877774ef6aadc922cb2e345e98040a5a03a721ad" Jul 12 18:50:56.277039 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:56.277021 2577 scope.go:117] "RemoveContainer" containerID="f32d11b40f0bd8d527ea7e8db097a1fe7c56400bf637f8a3de38b3c152c04316" Jul 12 18:50:56.287972 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:56.287953 2577 scope.go:117] "RemoveContainer" containerID="63913710c738ec5083b020f8877774ef6aadc922cb2e345e98040a5a03a721ad" Jul 12 18:50:56.288370 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:50:56.288342 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63913710c738ec5083b020f8877774ef6aadc922cb2e345e98040a5a03a721ad\": container with ID starting with 63913710c738ec5083b020f8877774ef6aadc922cb2e345e98040a5a03a721ad not found: ID does not exist" containerID="63913710c738ec5083b020f8877774ef6aadc922cb2e345e98040a5a03a721ad" Jul 12 18:50:56.288524 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:56.288383 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63913710c738ec5083b020f8877774ef6aadc922cb2e345e98040a5a03a721ad"} err="failed to get container status \"63913710c738ec5083b020f8877774ef6aadc922cb2e345e98040a5a03a721ad\": rpc error: code = NotFound desc = could not find container \"63913710c738ec5083b020f8877774ef6aadc922cb2e345e98040a5a03a721ad\": container with ID starting with 63913710c738ec5083b020f8877774ef6aadc922cb2e345e98040a5a03a721ad not found: ID does not exist" Jul 12 18:50:56.288524 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:56.288409 2577 scope.go:117] "RemoveContainer" containerID="f32d11b40f0bd8d527ea7e8db097a1fe7c56400bf637f8a3de38b3c152c04316" Jul 12 18:50:56.288852 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:50:56.288834 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f32d11b40f0bd8d527ea7e8db097a1fe7c56400bf637f8a3de38b3c152c04316\": container with ID starting with f32d11b40f0bd8d527ea7e8db097a1fe7c56400bf637f8a3de38b3c152c04316 not found: ID does not exist" containerID="f32d11b40f0bd8d527ea7e8db097a1fe7c56400bf637f8a3de38b3c152c04316" Jul 12 18:50:56.288904 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:56.288860 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f32d11b40f0bd8d527ea7e8db097a1fe7c56400bf637f8a3de38b3c152c04316"} err="failed to get container status \"f32d11b40f0bd8d527ea7e8db097a1fe7c56400bf637f8a3de38b3c152c04316\": rpc error: code = NotFound desc = could not find container \"f32d11b40f0bd8d527ea7e8db097a1fe7c56400bf637f8a3de38b3c152c04316\": container with ID starting with f32d11b40f0bd8d527ea7e8db097a1fe7c56400bf637f8a3de38b3c152c04316 not found: ID does not exist" Jul 12 18:50:56.291309 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:56.291289 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t"] Jul 12 18:50:56.293438 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:56.293417 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-675ba4-openshift-default-7759784dc7-6sv6t"] Jul 12 18:50:57.075877 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:57.075843 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19b67bd6-f4b0-4fab-bd1c-182a84625f32" path="/var/lib/kubelet/pods/19b67bd6-f4b0-4fab-bd1c-182a84625f32/volumes" Jul 12 18:50:57.986746 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:57.986712 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx"] Jul 12 18:50:57.987097 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:57.987083 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19b67bd6-f4b0-4fab-bd1c-182a84625f32" containerName="istio-proxy" Jul 12 18:50:57.987157 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:57.987100 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="19b67bd6-f4b0-4fab-bd1c-182a84625f32" containerName="istio-proxy" Jul 12 18:50:57.987211 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:57.987174 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="19b67bd6-f4b0-4fab-bd1c-182a84625f32" containerName="istio-proxy" Jul 12 18:50:57.987211 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:57.987186 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="19b67bd6-f4b0-4fab-bd1c-182a84625f32" containerName="istio-proxy" Jul 12 18:50:57.990051 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:57.990032 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx" Jul 12 18:50:57.992539 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:57.992522 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-1a89be-openshift-default-dockercfg-jl2p2\"" Jul 12 18:50:58.001603 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:58.001577 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx"] Jul 12 18:50:58.078567 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:58.078529 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aec1dc2c-2bf0-4af3-acef-57040036072a-istiod-ca-cert\") pod \"e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx\" (UID: \"aec1dc2c-2bf0-4af3-acef-57040036072a\") " pod="openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx" Jul 12 18:50:58.078567 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:58.078566 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aec1dc2c-2bf0-4af3-acef-57040036072a-istio-envoy\") pod \"e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx\" (UID: \"aec1dc2c-2bf0-4af3-acef-57040036072a\") " pod="openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx" Jul 12 18:50:58.079063 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:58.078584 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aec1dc2c-2bf0-4af3-acef-57040036072a-istio-data\") pod \"e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx\" (UID: \"aec1dc2c-2bf0-4af3-acef-57040036072a\") " pod="openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx" Jul 12 18:50:58.079063 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:58.078615 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gt4v9\" (UniqueName: \"kubernetes.io/projected/aec1dc2c-2bf0-4af3-acef-57040036072a-kube-api-access-gt4v9\") pod \"e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx\" (UID: \"aec1dc2c-2bf0-4af3-acef-57040036072a\") " pod="openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx" Jul 12 18:50:58.079063 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:58.078706 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aec1dc2c-2bf0-4af3-acef-57040036072a-istio-podinfo\") pod \"e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx\" (UID: \"aec1dc2c-2bf0-4af3-acef-57040036072a\") " pod="openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx" Jul 12 18:50:58.079063 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:58.078735 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aec1dc2c-2bf0-4af3-acef-57040036072a-workload-certs\") pod \"e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx\" (UID: \"aec1dc2c-2bf0-4af3-acef-57040036072a\") " pod="openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx" Jul 12 18:50:58.079063 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:58.078762 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aec1dc2c-2bf0-4af3-acef-57040036072a-credential-socket\") pod \"e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx\" (UID: \"aec1dc2c-2bf0-4af3-acef-57040036072a\") " pod="openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx" Jul 12 18:50:58.079063 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:58.078881 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aec1dc2c-2bf0-4af3-acef-57040036072a-istio-token\") pod \"e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx\" (UID: \"aec1dc2c-2bf0-4af3-acef-57040036072a\") " pod="openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx" Jul 12 18:50:58.079063 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:58.078929 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aec1dc2c-2bf0-4af3-acef-57040036072a-workload-socket\") pod \"e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx\" (UID: \"aec1dc2c-2bf0-4af3-acef-57040036072a\") " pod="openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx" Jul 12 18:50:58.180177 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:58.180135 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aec1dc2c-2bf0-4af3-acef-57040036072a-istio-podinfo\") pod \"e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx\" (UID: \"aec1dc2c-2bf0-4af3-acef-57040036072a\") " pod="openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx" Jul 12 18:50:58.180405 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:58.180193 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aec1dc2c-2bf0-4af3-acef-57040036072a-workload-certs\") pod \"e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx\" (UID: \"aec1dc2c-2bf0-4af3-acef-57040036072a\") " pod="openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx" Jul 12 18:50:58.180405 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:58.180231 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aec1dc2c-2bf0-4af3-acef-57040036072a-credential-socket\") pod \"e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx\" (UID: \"aec1dc2c-2bf0-4af3-acef-57040036072a\") " pod="openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx" Jul 12 18:50:58.180405 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:58.180309 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aec1dc2c-2bf0-4af3-acef-57040036072a-istio-token\") pod \"e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx\" (UID: \"aec1dc2c-2bf0-4af3-acef-57040036072a\") " pod="openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx" Jul 12 18:50:58.180405 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:58.180348 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aec1dc2c-2bf0-4af3-acef-57040036072a-workload-socket\") pod \"e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx\" (UID: \"aec1dc2c-2bf0-4af3-acef-57040036072a\") " pod="openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx" Jul 12 18:50:58.180405 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:58.180376 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aec1dc2c-2bf0-4af3-acef-57040036072a-istiod-ca-cert\") pod \"e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx\" (UID: \"aec1dc2c-2bf0-4af3-acef-57040036072a\") " pod="openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx" Jul 12 18:50:58.180405 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:58.180407 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aec1dc2c-2bf0-4af3-acef-57040036072a-istio-envoy\") pod \"e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx\" (UID: \"aec1dc2c-2bf0-4af3-acef-57040036072a\") " pod="openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx" Jul 12 18:50:58.180685 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:58.180434 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aec1dc2c-2bf0-4af3-acef-57040036072a-istio-data\") pod \"e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx\" (UID: \"aec1dc2c-2bf0-4af3-acef-57040036072a\") " pod="openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx" Jul 12 18:50:58.180685 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:58.180494 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gt4v9\" (UniqueName: \"kubernetes.io/projected/aec1dc2c-2bf0-4af3-acef-57040036072a-kube-api-access-gt4v9\") pod \"e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx\" (UID: \"aec1dc2c-2bf0-4af3-acef-57040036072a\") " pod="openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx" Jul 12 18:50:58.181099 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:58.181072 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aec1dc2c-2bf0-4af3-acef-57040036072a-workload-certs\") pod \"e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx\" (UID: \"aec1dc2c-2bf0-4af3-acef-57040036072a\") " pod="openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx" Jul 12 18:50:58.181240 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:58.181184 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aec1dc2c-2bf0-4af3-acef-57040036072a-workload-socket\") pod \"e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx\" (UID: \"aec1dc2c-2bf0-4af3-acef-57040036072a\") " pod="openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx" Jul 12 18:50:58.181240 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:58.181111 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aec1dc2c-2bf0-4af3-acef-57040036072a-credential-socket\") pod \"e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx\" (UID: \"aec1dc2c-2bf0-4af3-acef-57040036072a\") " pod="openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx" Jul 12 18:50:58.181240 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:58.181153 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aec1dc2c-2bf0-4af3-acef-57040036072a-istio-data\") pod \"e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx\" (UID: \"aec1dc2c-2bf0-4af3-acef-57040036072a\") " pod="openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx" Jul 12 18:50:58.181525 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:58.181506 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aec1dc2c-2bf0-4af3-acef-57040036072a-istiod-ca-cert\") pod \"e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx\" (UID: \"aec1dc2c-2bf0-4af3-acef-57040036072a\") " pod="openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx" Jul 12 18:50:58.182891 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:58.182873 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aec1dc2c-2bf0-4af3-acef-57040036072a-istio-podinfo\") pod \"e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx\" (UID: \"aec1dc2c-2bf0-4af3-acef-57040036072a\") " pod="openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx" Jul 12 18:50:58.183065 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:58.183043 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aec1dc2c-2bf0-4af3-acef-57040036072a-istio-envoy\") pod \"e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx\" (UID: \"aec1dc2c-2bf0-4af3-acef-57040036072a\") " pod="openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx" Jul 12 18:50:58.190742 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:58.190718 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aec1dc2c-2bf0-4af3-acef-57040036072a-istio-token\") pod \"e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx\" (UID: \"aec1dc2c-2bf0-4af3-acef-57040036072a\") " pod="openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx" Jul 12 18:50:58.191499 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:58.191467 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gt4v9\" (UniqueName: \"kubernetes.io/projected/aec1dc2c-2bf0-4af3-acef-57040036072a-kube-api-access-gt4v9\") pod \"e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx\" (UID: \"aec1dc2c-2bf0-4af3-acef-57040036072a\") " pod="openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx" Jul 12 18:50:58.302187 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:58.302091 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx" Jul 12 18:50:58.442752 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:58.442725 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx"] Jul 12 18:50:58.444850 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:50:58.444823 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaec1dc2c_2bf0_4af3_acef_57040036072a.slice/crio-6a4edb1e11cf02fd780f53a6bdb97fd2f9223d8a4c8a598bdb5be97e5609a9c4 WatchSource:0}: Error finding container 6a4edb1e11cf02fd780f53a6bdb97fd2f9223d8a4c8a598bdb5be97e5609a9c4: Status 404 returned error can't find the container with id 6a4edb1e11cf02fd780f53a6bdb97fd2f9223d8a4c8a598bdb5be97e5609a9c4 Jul 12 18:50:58.447075 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:58.447046 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 12 18:50:58.447144 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:58.447111 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 12 18:50:58.447144 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:58.447139 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 12 18:50:59.285402 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:59.285365 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx" event={"ID":"aec1dc2c-2bf0-4af3-acef-57040036072a","Type":"ContainerStarted","Data":"4df5d07e8577c61e15f5d0544147899139eb6c3bd1fdaf44c29e5c0b96e79d98"} Jul 12 18:50:59.285822 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:59.285428 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx" event={"ID":"aec1dc2c-2bf0-4af3-acef-57040036072a","Type":"ContainerStarted","Data":"6a4edb1e11cf02fd780f53a6bdb97fd2f9223d8a4c8a598bdb5be97e5609a9c4"} Jul 12 18:50:59.302392 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:59.302362 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx" Jul 12 18:50:59.306138 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:59.306094 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx" podStartSLOduration=2.306079647 podStartE2EDuration="2.306079647s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 18:50:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:50:59.303556853 +0000 UTC m=+2032.430461662" watchObservedRunningTime="2026-07-12 18:50:59.306079647 +0000 UTC m=+2032.432984444" Jul 12 18:50:59.308081 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:50:59.308061 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx" Jul 12 18:51:00.290888 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:51:00.290856 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx" Jul 12 18:51:00.291788 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:51:00.291768 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx" Jul 12 18:51:02.084736 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:51:02.084697 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:51:16.184986 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:51:16.184940 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:51:16.332828 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:51:16.332782 2577 patch_prober.go:28] interesting pod/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 12 18:51:16.333003 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:51:16.332867 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx" podUID="aec1dc2c-2bf0-4af3-acef-57040036072a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 12 18:51:18.393040 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:51:18.393006 2577 generic.go:356] "Generic (PLEG): container finished" podID="aec1dc2c-2bf0-4af3-acef-57040036072a" containerID="4df5d07e8577c61e15f5d0544147899139eb6c3bd1fdaf44c29e5c0b96e79d98" exitCode=0 Jul 12 18:51:18.393486 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:51:18.393071 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx" event={"ID":"aec1dc2c-2bf0-4af3-acef-57040036072a","Type":"ContainerDied","Data":"4df5d07e8577c61e15f5d0544147899139eb6c3bd1fdaf44c29e5c0b96e79d98"} Jul 12 18:51:18.393486 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:51:18.393478 2577 scope.go:117] "RemoveContainer" containerID="4df5d07e8577c61e15f5d0544147899139eb6c3bd1fdaf44c29e5c0b96e79d98" Jul 12 18:51:18.394437 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:51:18.394405 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 12 18:51:18.394537 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:51:18.394490 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 12 18:51:18.394596 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:51:18.394535 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 12 18:51:19.400550 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:51:19.400516 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx" event={"ID":"aec1dc2c-2bf0-4af3-acef-57040036072a","Type":"ContainerStarted","Data":"6faff41150ffbf0c9ceece8be1e72248f4811c5adbeedd123658ec14669895b1"} Jul 12 18:51:20.302966 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:51:20.302930 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx" Jul 12 18:51:20.303223 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:51:20.303200 2577 patch_prober.go:28] interesting pod/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 12 18:51:20.303313 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:51:20.303273 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx" podUID="aec1dc2c-2bf0-4af3-acef-57040036072a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 12 18:51:21.302443 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:51:21.302414 2577 patch_prober.go:28] interesting pod/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 12 18:51:21.302882 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:51:21.302475 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx" podUID="aec1dc2c-2bf0-4af3-acef-57040036072a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 12 18:51:22.302917 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:51:22.302881 2577 patch_prober.go:28] interesting pod/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 12 18:51:22.303332 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:51:22.302950 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx" podUID="aec1dc2c-2bf0-4af3-acef-57040036072a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 12 18:51:23.303116 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:51:23.303084 2577 patch_prober.go:28] interesting pod/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 12 18:51:23.303493 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:51:23.303148 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx" podUID="aec1dc2c-2bf0-4af3-acef-57040036072a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 12 18:51:24.307149 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:51:24.307117 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx" Jul 12 18:51:24.307556 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:51:24.307484 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx" Jul 12 18:51:24.308078 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:51:24.308062 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx" Jul 12 18:52:10.023870 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:52:10.023838 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-spzqd_ed115460-b220-476d-b9d5-65f95ec3d059/ovn-acl-logging/0.log" Jul 12 18:52:10.031895 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:52:10.031872 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-spzqd_ed115460-b220-476d-b9d5-65f95ec3d059/ovn-acl-logging/0.log" Jul 12 18:54:22.375359 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:22.375324 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:54:24.617050 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:24.617017 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx"] Jul 12 18:54:24.617452 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:24.617250 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx" podUID="aec1dc2c-2bf0-4af3-acef-57040036072a" containerName="istio-proxy" containerID="cri-o://6faff41150ffbf0c9ceece8be1e72248f4811c5adbeedd123658ec14669895b1" gracePeriod=30 Jul 12 18:54:29.872557 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:29.872526 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx" Jul 12 18:54:30.010428 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:30.010389 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aec1dc2c-2bf0-4af3-acef-57040036072a-istio-data\") pod \"aec1dc2c-2bf0-4af3-acef-57040036072a\" (UID: \"aec1dc2c-2bf0-4af3-acef-57040036072a\") " Jul 12 18:54:30.010428 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:30.010437 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aec1dc2c-2bf0-4af3-acef-57040036072a-credential-socket\") pod \"aec1dc2c-2bf0-4af3-acef-57040036072a\" (UID: \"aec1dc2c-2bf0-4af3-acef-57040036072a\") " Jul 12 18:54:30.010742 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:30.010474 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aec1dc2c-2bf0-4af3-acef-57040036072a-istiod-ca-cert\") pod \"aec1dc2c-2bf0-4af3-acef-57040036072a\" (UID: \"aec1dc2c-2bf0-4af3-acef-57040036072a\") " Jul 12 18:54:30.010742 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:30.010494 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gt4v9\" (UniqueName: \"kubernetes.io/projected/aec1dc2c-2bf0-4af3-acef-57040036072a-kube-api-access-gt4v9\") pod \"aec1dc2c-2bf0-4af3-acef-57040036072a\" (UID: \"aec1dc2c-2bf0-4af3-acef-57040036072a\") " Jul 12 18:54:30.010742 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:30.010510 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aec1dc2c-2bf0-4af3-acef-57040036072a-istio-podinfo\") pod \"aec1dc2c-2bf0-4af3-acef-57040036072a\" (UID: \"aec1dc2c-2bf0-4af3-acef-57040036072a\") " Jul 12 18:54:30.010742 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:30.010526 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aec1dc2c-2bf0-4af3-acef-57040036072a-workload-socket\") pod \"aec1dc2c-2bf0-4af3-acef-57040036072a\" (UID: \"aec1dc2c-2bf0-4af3-acef-57040036072a\") " Jul 12 18:54:30.010742 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:30.010564 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aec1dc2c-2bf0-4af3-acef-57040036072a-istio-envoy\") pod \"aec1dc2c-2bf0-4af3-acef-57040036072a\" (UID: \"aec1dc2c-2bf0-4af3-acef-57040036072a\") " Jul 12 18:54:30.010742 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:30.010590 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aec1dc2c-2bf0-4af3-acef-57040036072a-workload-certs\") pod \"aec1dc2c-2bf0-4af3-acef-57040036072a\" (UID: \"aec1dc2c-2bf0-4af3-acef-57040036072a\") " Jul 12 18:54:30.010742 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:30.010616 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aec1dc2c-2bf0-4af3-acef-57040036072a-istio-token\") pod \"aec1dc2c-2bf0-4af3-acef-57040036072a\" (UID: \"aec1dc2c-2bf0-4af3-acef-57040036072a\") " Jul 12 18:54:30.011163 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:30.010837 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aec1dc2c-2bf0-4af3-acef-57040036072a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "aec1dc2c-2bf0-4af3-acef-57040036072a" (UID: "aec1dc2c-2bf0-4af3-acef-57040036072a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:54:30.011163 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:30.010970 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aec1dc2c-2bf0-4af3-acef-57040036072a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "aec1dc2c-2bf0-4af3-acef-57040036072a" (UID: "aec1dc2c-2bf0-4af3-acef-57040036072a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 18:54:30.011163 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:30.010973 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aec1dc2c-2bf0-4af3-acef-57040036072a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "aec1dc2c-2bf0-4af3-acef-57040036072a" (UID: "aec1dc2c-2bf0-4af3-acef-57040036072a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:54:30.011163 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:30.010997 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aec1dc2c-2bf0-4af3-acef-57040036072a-credential-socket\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:54:30.011163 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:30.011153 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aec1dc2c-2bf0-4af3-acef-57040036072a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "aec1dc2c-2bf0-4af3-acef-57040036072a" (UID: "aec1dc2c-2bf0-4af3-acef-57040036072a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:54:30.011415 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:30.011374 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aec1dc2c-2bf0-4af3-acef-57040036072a-istio-data" (OuterVolumeSpecName: "istio-data") pod "aec1dc2c-2bf0-4af3-acef-57040036072a" (UID: "aec1dc2c-2bf0-4af3-acef-57040036072a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:54:30.012866 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:30.012841 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aec1dc2c-2bf0-4af3-acef-57040036072a-istio-token" (OuterVolumeSpecName: "istio-token") pod "aec1dc2c-2bf0-4af3-acef-57040036072a" (UID: "aec1dc2c-2bf0-4af3-acef-57040036072a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:54:30.012989 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:30.012904 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/aec1dc2c-2bf0-4af3-acef-57040036072a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "aec1dc2c-2bf0-4af3-acef-57040036072a" (UID: "aec1dc2c-2bf0-4af3-acef-57040036072a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 12 18:54:30.013201 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:30.013182 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aec1dc2c-2bf0-4af3-acef-57040036072a-kube-api-access-gt4v9" (OuterVolumeSpecName: "kube-api-access-gt4v9") pod "aec1dc2c-2bf0-4af3-acef-57040036072a" (UID: "aec1dc2c-2bf0-4af3-acef-57040036072a"). InnerVolumeSpecName "kube-api-access-gt4v9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 18:54:30.013264 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:30.013190 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aec1dc2c-2bf0-4af3-acef-57040036072a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "aec1dc2c-2bf0-4af3-acef-57040036072a" (UID: "aec1dc2c-2bf0-4af3-acef-57040036072a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 18:54:30.111486 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:30.111451 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aec1dc2c-2bf0-4af3-acef-57040036072a-istiod-ca-cert\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:54:30.111486 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:30.111481 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gt4v9\" (UniqueName: \"kubernetes.io/projected/aec1dc2c-2bf0-4af3-acef-57040036072a-kube-api-access-gt4v9\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:54:30.111486 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:30.111492 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aec1dc2c-2bf0-4af3-acef-57040036072a-istio-podinfo\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:54:30.111749 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:30.111501 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aec1dc2c-2bf0-4af3-acef-57040036072a-workload-socket\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:54:30.111749 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:30.111510 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aec1dc2c-2bf0-4af3-acef-57040036072a-istio-envoy\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:54:30.111749 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:30.111518 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aec1dc2c-2bf0-4af3-acef-57040036072a-workload-certs\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:54:30.111749 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:30.111525 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aec1dc2c-2bf0-4af3-acef-57040036072a-istio-token\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:54:30.111749 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:30.111532 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aec1dc2c-2bf0-4af3-acef-57040036072a-istio-data\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jul 12 18:54:30.395309 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:30.395218 2577 generic.go:356] "Generic (PLEG): container finished" podID="aec1dc2c-2bf0-4af3-acef-57040036072a" containerID="6faff41150ffbf0c9ceece8be1e72248f4811c5adbeedd123658ec14669895b1" exitCode=0 Jul 12 18:54:30.395309 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:30.395271 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx" Jul 12 18:54:30.395484 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:30.395306 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx" event={"ID":"aec1dc2c-2bf0-4af3-acef-57040036072a","Type":"ContainerDied","Data":"6faff41150ffbf0c9ceece8be1e72248f4811c5adbeedd123658ec14669895b1"} Jul 12 18:54:30.395484 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:30.395364 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx" event={"ID":"aec1dc2c-2bf0-4af3-acef-57040036072a","Type":"ContainerDied","Data":"6a4edb1e11cf02fd780f53a6bdb97fd2f9223d8a4c8a598bdb5be97e5609a9c4"} Jul 12 18:54:30.395484 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:30.395385 2577 scope.go:117] "RemoveContainer" containerID="6faff41150ffbf0c9ceece8be1e72248f4811c5adbeedd123658ec14669895b1" Jul 12 18:54:30.405010 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:30.404988 2577 scope.go:117] "RemoveContainer" containerID="4df5d07e8577c61e15f5d0544147899139eb6c3bd1fdaf44c29e5c0b96e79d98" Jul 12 18:54:30.413082 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:30.413060 2577 scope.go:117] "RemoveContainer" containerID="6faff41150ffbf0c9ceece8be1e72248f4811c5adbeedd123658ec14669895b1" Jul 12 18:54:30.413338 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:54:30.413318 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6faff41150ffbf0c9ceece8be1e72248f4811c5adbeedd123658ec14669895b1\": container with ID starting with 6faff41150ffbf0c9ceece8be1e72248f4811c5adbeedd123658ec14669895b1 not found: ID does not exist" containerID="6faff41150ffbf0c9ceece8be1e72248f4811c5adbeedd123658ec14669895b1" Jul 12 18:54:30.413416 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:30.413350 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6faff41150ffbf0c9ceece8be1e72248f4811c5adbeedd123658ec14669895b1"} err="failed to get container status \"6faff41150ffbf0c9ceece8be1e72248f4811c5adbeedd123658ec14669895b1\": rpc error: code = NotFound desc = could not find container \"6faff41150ffbf0c9ceece8be1e72248f4811c5adbeedd123658ec14669895b1\": container with ID starting with 6faff41150ffbf0c9ceece8be1e72248f4811c5adbeedd123658ec14669895b1 not found: ID does not exist" Jul 12 18:54:30.413416 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:30.413377 2577 scope.go:117] "RemoveContainer" containerID="4df5d07e8577c61e15f5d0544147899139eb6c3bd1fdaf44c29e5c0b96e79d98" Jul 12 18:54:30.413641 ip-10-0-141-133 kubenswrapper[2577]: E0712 18:54:30.413609 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4df5d07e8577c61e15f5d0544147899139eb6c3bd1fdaf44c29e5c0b96e79d98\": container with ID starting with 4df5d07e8577c61e15f5d0544147899139eb6c3bd1fdaf44c29e5c0b96e79d98 not found: ID does not exist" containerID="4df5d07e8577c61e15f5d0544147899139eb6c3bd1fdaf44c29e5c0b96e79d98" Jul 12 18:54:30.413686 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:30.413652 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4df5d07e8577c61e15f5d0544147899139eb6c3bd1fdaf44c29e5c0b96e79d98"} err="failed to get container status \"4df5d07e8577c61e15f5d0544147899139eb6c3bd1fdaf44c29e5c0b96e79d98\": rpc error: code = NotFound desc = could not find container \"4df5d07e8577c61e15f5d0544147899139eb6c3bd1fdaf44c29e5c0b96e79d98\": container with ID starting with 4df5d07e8577c61e15f5d0544147899139eb6c3bd1fdaf44c29e5c0b96e79d98 not found: ID does not exist" Jul 12 18:54:30.416914 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:30.416894 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx"] Jul 12 18:54:30.421103 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:30.421082 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-1a89be-openshift-default-567c448dd7-bxscx"] Jul 12 18:54:31.080219 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:31.080181 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aec1dc2c-2bf0-4af3-acef-57040036072a" path="/var/lib/kubelet/pods/aec1dc2c-2bf0-4af3-acef-57040036072a/volumes" Jul 12 18:54:33.778524 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:33.778488 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4httc"] Jul 12 18:54:37.049882 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:37.049849 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-g4dgx_c035e80f-4c27-48d9-bb9a-59dcb0218fd9/manager/0.log" Jul 12 18:54:37.158240 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:37.158214 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-db78fd4c7-bq5pl_1ac29973-5a2e-4f0a-854e-e85369b8157a/copy-manifests/0.log" Jul 12 18:54:37.164405 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:37.164381 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-db78fd4c7-bq5pl_1ac29973-5a2e-4f0a-854e-e85369b8157a/manager/0.log" Jul 12 18:54:38.162888 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:38.162854 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f4r57_c3e07319-173b-4c5c-85ef-3af760954600/util/0.log" Jul 12 18:54:38.168827 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:38.168804 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f4r57_c3e07319-173b-4c5c-85ef-3af760954600/pull/0.log" Jul 12 18:54:38.174327 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:38.174306 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f4r57_c3e07319-173b-4c5c-85ef-3af760954600/extract/0.log" Jul 12 18:54:38.280916 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:38.280886 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vpk6h_124f8ebf-36d0-469a-b3b3-c932743898c9/util/0.log" Jul 12 18:54:38.294106 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:38.289000 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vpk6h_124f8ebf-36d0-469a-b3b3-c932743898c9/pull/0.log" Jul 12 18:54:38.299791 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:38.299767 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vpk6h_124f8ebf-36d0-469a-b3b3-c932743898c9/extract/0.log" Jul 12 18:54:38.408284 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:38.408256 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fnqbh_5b65d5ed-ac9a-48bc-8aef-3c608afe4f79/util/0.log" Jul 12 18:54:38.414172 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:38.414103 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fnqbh_5b65d5ed-ac9a-48bc-8aef-3c608afe4f79/pull/0.log" Jul 12 18:54:38.419876 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:38.419853 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fnqbh_5b65d5ed-ac9a-48bc-8aef-3c608afe4f79/extract/0.log" Jul 12 18:54:38.523773 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:38.523743 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gp9dz_f6d5bf1a-f136-45cf-9c46-1d4cad723409/util/0.log" Jul 12 18:54:38.529802 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:38.529778 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gp9dz_f6d5bf1a-f136-45cf-9c46-1d4cad723409/pull/0.log" Jul 12 18:54:38.534899 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:38.534874 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gp9dz_f6d5bf1a-f136-45cf-9c46-1d4cad723409/extract/0.log" Jul 12 18:54:39.312281 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:39.312236 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-4httc_47a4888e-37d0-47e2-84c6-d8b8b02db23a/limitador/0.log" Jul 12 18:54:39.750729 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:39.750699 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678qjql2_05e3c37f-45b4-47dc-9c35-cc81ee6dd97a/istio-proxy/0.log" Jul 12 18:54:39.874134 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:39.874102 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-dvc6v_e4faeb51-6114-46cc-a7df-ff844503afac/discovery/0.log" Jul 12 18:54:41.308261 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:41.308228 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-lwcn2_732b3a95-f015-4cda-b6c6-56cf546be766/main/0.log" Jul 12 18:54:47.965962 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:47.965935 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-2598z_9f8419c6-07be-4876-9762-d98cd772054b/global-pull-secret-syncer/0.log" Jul 12 18:54:48.114602 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:48.114559 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-hqhkm_e1c57461-73ff-431a-a529-60fa73bf37b5/konnectivity-agent/0.log" Jul 12 18:54:48.200285 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:48.200252 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-133.ec2.internal_98d2a84da404683679bff0f03b9d595c/haproxy/0.log" Jul 12 18:54:52.794987 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:52.794954 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f4r57_c3e07319-173b-4c5c-85ef-3af760954600/extract/0.log" Jul 12 18:54:52.825559 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:52.825528 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f4r57_c3e07319-173b-4c5c-85ef-3af760954600/util/0.log" Jul 12 18:54:52.843988 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:52.843959 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759f4r57_c3e07319-173b-4c5c-85ef-3af760954600/pull/0.log" Jul 12 18:54:52.869404 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:52.869378 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vpk6h_124f8ebf-36d0-469a-b3b3-c932743898c9/extract/0.log" Jul 12 18:54:52.888064 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:52.888028 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vpk6h_124f8ebf-36d0-469a-b3b3-c932743898c9/util/0.log" Jul 12 18:54:52.910963 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:52.910934 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0vpk6h_124f8ebf-36d0-469a-b3b3-c932743898c9/pull/0.log" Jul 12 18:54:52.944476 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:52.944449 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fnqbh_5b65d5ed-ac9a-48bc-8aef-3c608afe4f79/extract/0.log" Jul 12 18:54:52.962801 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:52.962776 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fnqbh_5b65d5ed-ac9a-48bc-8aef-3c608afe4f79/util/0.log" Jul 12 18:54:52.981521 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:52.981482 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73fnqbh_5b65d5ed-ac9a-48bc-8aef-3c608afe4f79/pull/0.log" Jul 12 18:54:53.010002 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:53.009979 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gp9dz_f6d5bf1a-f136-45cf-9c46-1d4cad723409/extract/0.log" Jul 12 18:54:53.029206 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:53.029182 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gp9dz_f6d5bf1a-f136-45cf-9c46-1d4cad723409/util/0.log" Jul 12 18:54:53.047866 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:53.047755 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gp9dz_f6d5bf1a-f136-45cf-9c46-1d4cad723409/pull/0.log" Jul 12 18:54:53.561653 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:53.561603 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-4httc_47a4888e-37d0-47e2-84c6-d8b8b02db23a/limitador/0.log" Jul 12 18:54:55.116483 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:55.116456 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_21e7f811-7d95-4335-a3c7-f10528ce7a0a/alertmanager/0.log" Jul 12 18:54:55.136237 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:55.136205 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_21e7f811-7d95-4335-a3c7-f10528ce7a0a/config-reloader/0.log" Jul 12 18:54:55.153825 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:55.153784 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_21e7f811-7d95-4335-a3c7-f10528ce7a0a/kube-rbac-proxy-web/0.log" Jul 12 18:54:55.170868 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:55.170848 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_21e7f811-7d95-4335-a3c7-f10528ce7a0a/kube-rbac-proxy/0.log" Jul 12 18:54:55.193916 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:55.193893 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_21e7f811-7d95-4335-a3c7-f10528ce7a0a/kube-rbac-proxy-metric/0.log" Jul 12 18:54:55.211430 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:55.211403 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_21e7f811-7d95-4335-a3c7-f10528ce7a0a/prom-label-proxy/0.log" Jul 12 18:54:55.228926 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:55.228898 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_21e7f811-7d95-4335-a3c7-f10528ce7a0a/init-config-reloader/0.log" Jul 12 18:54:55.306908 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:55.306882 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-wvfwn_ea7d5a92-5fad-43db-b210-956555ed01e8/kube-state-metrics/0.log" Jul 12 18:54:55.330351 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:55.330327 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-wvfwn_ea7d5a92-5fad-43db-b210-956555ed01e8/kube-rbac-proxy-main/0.log" Jul 12 18:54:55.366794 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:55.366721 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-wvfwn_ea7d5a92-5fad-43db-b210-956555ed01e8/kube-rbac-proxy-self/0.log" Jul 12 18:54:55.391479 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:55.391451 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-59bb7dbd4-wj8zl_c2b013fb-cf46-487b-9ee4-0109234e9fd1/metrics-server/0.log" Jul 12 18:54:55.453255 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:55.453232 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-22qtq_04f3aab6-b724-4e1c-95b4-1a85691a5cca/node-exporter/0.log" Jul 12 18:54:55.474418 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:55.474397 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-22qtq_04f3aab6-b724-4e1c-95b4-1a85691a5cca/kube-rbac-proxy/0.log" Jul 12 18:54:55.497219 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:55.497195 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-22qtq_04f3aab6-b724-4e1c-95b4-1a85691a5cca/init-textfile/0.log" Jul 12 18:54:55.975272 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:55.975237 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5d45bc54b5-m5nrq_56a707ab-2a08-46c0-8ded-547b26ad7da8/telemeter-client/0.log" Jul 12 18:54:55.992873 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:55.992848 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5d45bc54b5-m5nrq_56a707ab-2a08-46c0-8ded-547b26ad7da8/reload/0.log" Jul 12 18:54:56.009355 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:56.009331 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5d45bc54b5-m5nrq_56a707ab-2a08-46c0-8ded-547b26ad7da8/kube-rbac-proxy/0.log" Jul 12 18:54:57.532436 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:57.532407 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-lcvxn/perf-node-gather-daemonset-q5f57"] Jul 12 18:54:57.532939 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:57.532920 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19b67bd6-f4b0-4fab-bd1c-182a84625f32" containerName="istio-proxy" Jul 12 18:54:57.532939 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:57.532939 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="19b67bd6-f4b0-4fab-bd1c-182a84625f32" containerName="istio-proxy" Jul 12 18:54:57.533063 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:57.532957 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aec1dc2c-2bf0-4af3-acef-57040036072a" containerName="istio-proxy" Jul 12 18:54:57.533063 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:57.532966 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="aec1dc2c-2bf0-4af3-acef-57040036072a" containerName="istio-proxy" Jul 12 18:54:57.533063 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:57.532985 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aec1dc2c-2bf0-4af3-acef-57040036072a" containerName="istio-proxy" Jul 12 18:54:57.533063 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:57.532994 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="aec1dc2c-2bf0-4af3-acef-57040036072a" containerName="istio-proxy" Jul 12 18:54:57.533259 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:57.533084 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="aec1dc2c-2bf0-4af3-acef-57040036072a" containerName="istio-proxy" Jul 12 18:54:57.533259 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:57.533095 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="aec1dc2c-2bf0-4af3-acef-57040036072a" containerName="istio-proxy" Jul 12 18:54:57.536240 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:57.536219 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lcvxn/perf-node-gather-daemonset-q5f57" Jul 12 18:54:57.538704 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:57.538684 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-lcvxn\"/\"kube-root-ca.crt\"" Jul 12 18:54:57.538859 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:57.538829 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-lcvxn\"/\"openshift-service-ca.crt\"" Jul 12 18:54:57.539830 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:57.539811 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-lcvxn\"/\"default-dockercfg-flgp8\"" Jul 12 18:54:57.547039 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:57.547005 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lcvxn/perf-node-gather-daemonset-q5f57"] Jul 12 18:54:57.658850 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:57.658813 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3b13ac2a-1b85-44d2-9c08-7fc77514b86b-sys\") pod \"perf-node-gather-daemonset-q5f57\" (UID: \"3b13ac2a-1b85-44d2-9c08-7fc77514b86b\") " pod="openshift-must-gather-lcvxn/perf-node-gather-daemonset-q5f57" Jul 12 18:54:57.659049 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:57.658870 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3b13ac2a-1b85-44d2-9c08-7fc77514b86b-podres\") pod \"perf-node-gather-daemonset-q5f57\" (UID: \"3b13ac2a-1b85-44d2-9c08-7fc77514b86b\") " pod="openshift-must-gather-lcvxn/perf-node-gather-daemonset-q5f57" Jul 12 18:54:57.659049 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:57.658925 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d42g9\" (UniqueName: \"kubernetes.io/projected/3b13ac2a-1b85-44d2-9c08-7fc77514b86b-kube-api-access-d42g9\") pod \"perf-node-gather-daemonset-q5f57\" (UID: \"3b13ac2a-1b85-44d2-9c08-7fc77514b86b\") " pod="openshift-must-gather-lcvxn/perf-node-gather-daemonset-q5f57" Jul 12 18:54:57.659049 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:57.658994 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3b13ac2a-1b85-44d2-9c08-7fc77514b86b-lib-modules\") pod \"perf-node-gather-daemonset-q5f57\" (UID: \"3b13ac2a-1b85-44d2-9c08-7fc77514b86b\") " pod="openshift-must-gather-lcvxn/perf-node-gather-daemonset-q5f57" Jul 12 18:54:57.659179 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:57.659075 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3b13ac2a-1b85-44d2-9c08-7fc77514b86b-proc\") pod \"perf-node-gather-daemonset-q5f57\" (UID: \"3b13ac2a-1b85-44d2-9c08-7fc77514b86b\") " pod="openshift-must-gather-lcvxn/perf-node-gather-daemonset-q5f57" Jul 12 18:54:57.760385 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:57.760347 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3b13ac2a-1b85-44d2-9c08-7fc77514b86b-lib-modules\") pod \"perf-node-gather-daemonset-q5f57\" (UID: \"3b13ac2a-1b85-44d2-9c08-7fc77514b86b\") " pod="openshift-must-gather-lcvxn/perf-node-gather-daemonset-q5f57" Jul 12 18:54:57.760565 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:57.760398 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3b13ac2a-1b85-44d2-9c08-7fc77514b86b-proc\") pod \"perf-node-gather-daemonset-q5f57\" (UID: \"3b13ac2a-1b85-44d2-9c08-7fc77514b86b\") " pod="openshift-must-gather-lcvxn/perf-node-gather-daemonset-q5f57" Jul 12 18:54:57.760565 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:57.760425 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3b13ac2a-1b85-44d2-9c08-7fc77514b86b-sys\") pod \"perf-node-gather-daemonset-q5f57\" (UID: \"3b13ac2a-1b85-44d2-9c08-7fc77514b86b\") " pod="openshift-must-gather-lcvxn/perf-node-gather-daemonset-q5f57" Jul 12 18:54:57.760565 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:57.760458 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3b13ac2a-1b85-44d2-9c08-7fc77514b86b-podres\") pod \"perf-node-gather-daemonset-q5f57\" (UID: \"3b13ac2a-1b85-44d2-9c08-7fc77514b86b\") " pod="openshift-must-gather-lcvxn/perf-node-gather-daemonset-q5f57" Jul 12 18:54:57.760565 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:57.760474 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d42g9\" (UniqueName: \"kubernetes.io/projected/3b13ac2a-1b85-44d2-9c08-7fc77514b86b-kube-api-access-d42g9\") pod \"perf-node-gather-daemonset-q5f57\" (UID: \"3b13ac2a-1b85-44d2-9c08-7fc77514b86b\") " pod="openshift-must-gather-lcvxn/perf-node-gather-daemonset-q5f57" Jul 12 18:54:57.760565 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:57.760543 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3b13ac2a-1b85-44d2-9c08-7fc77514b86b-lib-modules\") pod \"perf-node-gather-daemonset-q5f57\" (UID: \"3b13ac2a-1b85-44d2-9c08-7fc77514b86b\") " pod="openshift-must-gather-lcvxn/perf-node-gather-daemonset-q5f57" Jul 12 18:54:57.760823 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:57.760543 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3b13ac2a-1b85-44d2-9c08-7fc77514b86b-sys\") pod \"perf-node-gather-daemonset-q5f57\" (UID: \"3b13ac2a-1b85-44d2-9c08-7fc77514b86b\") " pod="openshift-must-gather-lcvxn/perf-node-gather-daemonset-q5f57" Jul 12 18:54:57.760823 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:57.760549 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3b13ac2a-1b85-44d2-9c08-7fc77514b86b-proc\") pod \"perf-node-gather-daemonset-q5f57\" (UID: \"3b13ac2a-1b85-44d2-9c08-7fc77514b86b\") " pod="openshift-must-gather-lcvxn/perf-node-gather-daemonset-q5f57" Jul 12 18:54:57.760823 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:57.760611 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3b13ac2a-1b85-44d2-9c08-7fc77514b86b-podres\") pod \"perf-node-gather-daemonset-q5f57\" (UID: \"3b13ac2a-1b85-44d2-9c08-7fc77514b86b\") " pod="openshift-must-gather-lcvxn/perf-node-gather-daemonset-q5f57" Jul 12 18:54:57.767959 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:57.767939 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d42g9\" (UniqueName: \"kubernetes.io/projected/3b13ac2a-1b85-44d2-9c08-7fc77514b86b-kube-api-access-d42g9\") pod \"perf-node-gather-daemonset-q5f57\" (UID: \"3b13ac2a-1b85-44d2-9c08-7fc77514b86b\") " pod="openshift-must-gather-lcvxn/perf-node-gather-daemonset-q5f57" Jul 12 18:54:57.848050 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:57.847961 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lcvxn/perf-node-gather-daemonset-q5f57" Jul 12 18:54:57.978442 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:57.978420 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lcvxn/perf-node-gather-daemonset-q5f57"] Jul 12 18:54:57.980592 ip-10-0-141-133 kubenswrapper[2577]: W0712 18:54:57.980563 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod3b13ac2a_1b85_44d2_9c08_7fc77514b86b.slice/crio-06c6e50f5513f338e7cdadc50bd01f2de0f07f0d979fb159e96e6a96f33a440b WatchSource:0}: Error finding container 06c6e50f5513f338e7cdadc50bd01f2de0f07f0d979fb159e96e6a96f33a440b: Status 404 returned error can't find the container with id 06c6e50f5513f338e7cdadc50bd01f2de0f07f0d979fb159e96e6a96f33a440b Jul 12 18:54:58.382195 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:58.382166 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-747d58bd59-b7nw9_96ab96da-ce89-4d82-91ef-46bebade2984/console/0.log" Jul 12 18:54:58.538619 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:58.538585 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lcvxn/perf-node-gather-daemonset-q5f57" event={"ID":"3b13ac2a-1b85-44d2-9c08-7fc77514b86b","Type":"ContainerStarted","Data":"9a46a5d25e67a8f31c6616d411b1f858a5990b84684cb13babf2720f4a9acb71"} Jul 12 18:54:58.539057 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:58.538658 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lcvxn/perf-node-gather-daemonset-q5f57" event={"ID":"3b13ac2a-1b85-44d2-9c08-7fc77514b86b","Type":"ContainerStarted","Data":"06c6e50f5513f338e7cdadc50bd01f2de0f07f0d979fb159e96e6a96f33a440b"} Jul 12 18:54:58.539057 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:58.538683 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-lcvxn/perf-node-gather-daemonset-q5f57" Jul 12 18:54:58.552565 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:58.552519 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-lcvxn/perf-node-gather-daemonset-q5f57" podStartSLOduration=1.55250455 podStartE2EDuration="1.55250455s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 18:54:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 18:54:58.551592755 +0000 UTC m=+2271.678497571" watchObservedRunningTime="2026-07-12 18:54:58.55250455 +0000 UTC m=+2271.679409354" Jul 12 18:54:59.828393 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:59.828368 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ld5vx_e812eecd-24a1-443b-aee0-0372c0324c56/dns/0.log" Jul 12 18:54:59.846719 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:59.846690 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ld5vx_e812eecd-24a1-443b-aee0-0372c0324c56/kube-rbac-proxy/0.log" Jul 12 18:54:59.987906 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:54:59.987880 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xwcbp_cd8ab0de-04ec-4aa8-8d33-87aa4bbef597/dns-node-resolver/0.log" Jul 12 18:55:00.488723 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:55:00.488691 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-khqqs_1b501a87-6ac0-4f0b-9072-aeecd64d5bc0/node-ca/0.log" Jul 12 18:55:01.406440 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:55:01.406405 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678qjql2_05e3c37f-45b4-47dc-9c35-cc81ee6dd97a/istio-proxy/0.log" Jul 12 18:55:01.569234 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:55:01.569210 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-dvc6v_e4faeb51-6114-46cc-a7df-ff844503afac/discovery/0.log" Jul 12 18:55:02.453494 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:55:02.453462 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-zf4zr_d7643e94-b3c1-428f-842e-4079174ea430/serve-healthcheck-canary/0.log" Jul 12 18:55:02.902742 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:55:02.902663 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6s2lc_1cd288a5-920c-4b68-ad97-27d449622108/kube-rbac-proxy/0.log" Jul 12 18:55:02.921793 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:55:02.921766 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6s2lc_1cd288a5-920c-4b68-ad97-27d449622108/exporter/0.log" Jul 12 18:55:02.939705 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:55:02.939681 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6s2lc_1cd288a5-920c-4b68-ad97-27d449622108/extractor/0.log" Jul 12 18:55:04.552700 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:55:04.552676 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-lcvxn/perf-node-gather-daemonset-q5f57" Jul 12 18:55:05.345715 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:55:05.345684 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-g4dgx_c035e80f-4c27-48d9-bb9a-59dcb0218fd9/manager/0.log" Jul 12 18:55:05.365927 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:55:05.365896 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-db78fd4c7-bq5pl_1ac29973-5a2e-4f0a-854e-e85369b8157a/manager/0.log" Jul 12 18:55:05.383003 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:55:05.382980 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-db78fd4c7-bq5pl_1ac29973-5a2e-4f0a-854e-e85369b8157a/copy-manifests/0.log" Jul 12 18:55:06.833252 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:55:06.833222 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-9757cf8df-rcmmq_19d37878-abe2-47a1-9194-6400f779fe95/manager/0.log" Jul 12 18:55:06.858418 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:55:06.858388 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-wjrwc_ae75c702-3547-4324-86df-ada6e3b4b443/openshift-lws-operator/0.log" Jul 12 18:55:08.065509 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:55:08.065481 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678qjql2_05e3c37f-45b4-47dc-9c35-cc81ee6dd97a/istio-proxy/0.log" Jul 12 18:55:08.219720 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:55:08.219685 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-dvc6v_e4faeb51-6114-46cc-a7df-ff844503afac/discovery/0.log" Jul 12 18:55:09.550072 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:55:09.550044 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678qjql2_05e3c37f-45b4-47dc-9c35-cc81ee6dd97a/istio-proxy/0.log" Jul 12 18:55:09.723200 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:55:09.723164 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-dvc6v_e4faeb51-6114-46cc-a7df-ff844503afac/discovery/0.log" Jul 12 18:55:11.738362 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:55:11.738324 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-hvczh_aa7d16a7-5382-47dd-9455-07282d523685/migrator/0.log" Jul 12 18:55:11.754851 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:55:11.754825 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-hvczh_aa7d16a7-5382-47dd-9455-07282d523685/graceful-termination/0.log" Jul 12 18:55:12.922251 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:55:12.922227 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-29hd5_1a19952f-4732-46da-98ee-05c476b44908/kube-multus-additional-cni-plugins/0.log" Jul 12 18:55:12.941774 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:55:12.941744 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-29hd5_1a19952f-4732-46da-98ee-05c476b44908/egress-router-binary-copy/0.log" Jul 12 18:55:12.958769 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:55:12.958741 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-29hd5_1a19952f-4732-46da-98ee-05c476b44908/cni-plugins/0.log" Jul 12 18:55:12.975248 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:55:12.975217 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-29hd5_1a19952f-4732-46da-98ee-05c476b44908/bond-cni-plugin/0.log" Jul 12 18:55:12.992178 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:55:12.992143 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-29hd5_1a19952f-4732-46da-98ee-05c476b44908/routeoverride-cni/0.log" Jul 12 18:55:13.008825 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:55:13.008795 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-29hd5_1a19952f-4732-46da-98ee-05c476b44908/whereabouts-cni-bincopy/0.log" Jul 12 18:55:13.025077 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:55:13.025052 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-29hd5_1a19952f-4732-46da-98ee-05c476b44908/whereabouts-cni/0.log" Jul 12 18:55:13.375692 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:55:13.375659 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-t9jvk_a05fea52-1297-4bd7-82a2-d863030b675b/kube-multus/0.log" Jul 12 18:55:13.431202 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:55:13.431174 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-r66xp_7d69e388-ddc9-4dd8-b5dc-a81a59dc1698/network-metrics-daemon/0.log" Jul 12 18:55:13.445137 ip-10-0-141-133 kubenswrapper[2577]: I0712 18:55:13.445109 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-r66xp_7d69e388-ddc9-4dd8-b5dc-a81a59dc1698/kube-rbac-proxy/0.log"