Jul 08 13:23:11.092345 ip-10-0-137-164 systemd[1]: Starting Kubernetes Kubelet... Jul 08 13:23:11.561828 ip-10-0-137-164 kubenswrapper[2567]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 13:23:11.561828 ip-10-0-137-164 kubenswrapper[2567]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 08 13:23:11.561828 ip-10-0-137-164 kubenswrapper[2567]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 13:23:11.561828 ip-10-0-137-164 kubenswrapper[2567]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 08 13:23:11.561828 ip-10-0-137-164 kubenswrapper[2567]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 13:23:11.561828 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.527974 2567 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 08 13:23:11.561828 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534117 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 13:23:11.561828 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534129 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 13:23:11.561828 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534133 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 13:23:11.561828 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534137 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 13:23:11.561828 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534140 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 13:23:11.561828 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534144 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 13:23:11.561828 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534147 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 13:23:11.561828 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534150 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 13:23:11.563330 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534153 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 13:23:11.563330 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534158 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 13:23:11.563330 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534162 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 13:23:11.563330 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534166 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 13:23:11.563330 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534168 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 13:23:11.563330 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534171 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 13:23:11.563330 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534174 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 13:23:11.563330 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534177 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 13:23:11.563330 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534179 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 13:23:11.563330 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534182 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 13:23:11.563330 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534185 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 13:23:11.563330 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534188 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 13:23:11.563330 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534190 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 13:23:11.563330 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534193 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 13:23:11.563330 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534195 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 13:23:11.563330 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534198 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 13:23:11.563330 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534202 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 13:23:11.563330 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534206 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 13:23:11.563330 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534209 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 13:23:11.564765 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534213 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 13:23:11.564765 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534216 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 13:23:11.564765 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534219 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 13:23:11.564765 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534222 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 13:23:11.564765 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534224 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 13:23:11.564765 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534227 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 13:23:11.564765 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534229 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 13:23:11.564765 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534232 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 13:23:11.564765 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534234 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 13:23:11.564765 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534237 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 13:23:11.564765 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534240 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 13:23:11.564765 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534243 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 13:23:11.564765 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534246 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 13:23:11.564765 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534249 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 13:23:11.564765 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534251 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 13:23:11.564765 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534254 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 13:23:11.564765 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534256 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 13:23:11.564765 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534258 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 13:23:11.564765 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534261 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 13:23:11.564765 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534264 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 13:23:11.565632 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534266 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 13:23:11.565632 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534269 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 13:23:11.565632 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534271 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 13:23:11.565632 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534274 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 13:23:11.565632 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534277 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 13:23:11.565632 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534279 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 13:23:11.565632 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534282 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 13:23:11.565632 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534284 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 13:23:11.565632 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534287 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 13:23:11.565632 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534289 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 13:23:11.565632 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534292 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 13:23:11.565632 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534294 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 13:23:11.565632 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534297 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 13:23:11.565632 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534299 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 13:23:11.565632 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534304 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 13:23:11.565632 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534308 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 13:23:11.565632 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534311 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 13:23:11.565632 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534313 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 13:23:11.565632 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534316 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 13:23:11.566685 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534319 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 13:23:11.566685 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534321 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 13:23:11.566685 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534323 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 13:23:11.566685 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534326 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 13:23:11.566685 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534329 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 13:23:11.566685 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534332 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 13:23:11.566685 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534335 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 13:23:11.566685 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534338 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 13:23:11.566685 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534340 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 13:23:11.566685 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534343 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 13:23:11.566685 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534346 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 13:23:11.566685 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534348 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 13:23:11.566685 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534351 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 13:23:11.566685 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534353 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 13:23:11.566685 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534356 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 13:23:11.566685 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534358 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 13:23:11.566685 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534360 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 13:23:11.566685 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534363 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 13:23:11.566685 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534365 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 13:23:11.566685 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534368 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 13:23:11.567331 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534370 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 13:23:11.567331 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534373 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 13:23:11.567331 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534375 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 13:23:11.567331 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534378 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 13:23:11.567331 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534380 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 13:23:11.567331 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534384 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 13:23:11.567331 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534387 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 13:23:11.567331 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534389 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 13:23:11.567331 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534392 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 13:23:11.567331 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534394 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 13:23:11.567331 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534397 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 13:23:11.567331 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534399 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 13:23:11.567331 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.534402 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 08 13:23:11.567331 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540377 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 13:23:11.567331 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540392 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 13:23:11.567331 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540397 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 13:23:11.567331 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540401 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 13:23:11.567331 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540405 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 13:23:11.567331 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540410 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 13:23:11.567331 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540414 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 13:23:11.567952 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540419 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 13:23:11.567952 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540423 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 13:23:11.567952 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540427 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 13:23:11.567952 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540431 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 13:23:11.567952 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540434 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 13:23:11.567952 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540437 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 13:23:11.567952 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540441 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 13:23:11.567952 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540445 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 13:23:11.567952 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540449 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 13:23:11.567952 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540454 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 13:23:11.567952 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540459 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 13:23:11.567952 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540462 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 13:23:11.567952 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540466 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 13:23:11.567952 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540471 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 13:23:11.567952 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540475 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 13:23:11.567952 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540479 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 13:23:11.567952 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540482 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 13:23:11.567952 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540487 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 13:23:11.567952 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540491 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 13:23:11.567952 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540495 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 13:23:11.568600 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540499 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 13:23:11.568600 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540503 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 13:23:11.568600 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540507 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 13:23:11.568600 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540510 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 13:23:11.568600 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540514 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 13:23:11.568600 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540519 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 13:23:11.568600 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540522 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 13:23:11.568600 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540527 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 13:23:11.568600 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540530 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 13:23:11.568600 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540535 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 13:23:11.568600 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540538 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 13:23:11.568600 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540544 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 13:23:11.568600 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540550 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 13:23:11.568600 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540554 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 13:23:11.568600 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540558 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 13:23:11.568600 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540563 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 13:23:11.568600 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540567 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 13:23:11.568600 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540574 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 13:23:11.568600 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540579 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 13:23:11.569321 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540583 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 13:23:11.569321 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540587 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 13:23:11.569321 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540593 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 13:23:11.569321 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540597 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 13:23:11.569321 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540602 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 13:23:11.569321 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540605 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 13:23:11.569321 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540609 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 13:23:11.569321 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540614 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 13:23:11.569321 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540617 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 13:23:11.569321 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540622 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 13:23:11.569321 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540625 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 13:23:11.569321 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540629 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 13:23:11.569321 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540633 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 13:23:11.569321 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540637 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 13:23:11.569321 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540640 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 13:23:11.569321 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540643 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 13:23:11.569321 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540646 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 13:23:11.569321 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540649 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 13:23:11.569321 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540651 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 13:23:11.570101 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540655 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 13:23:11.570101 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540659 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 13:23:11.570101 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540662 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 13:23:11.570101 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540666 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 13:23:11.570101 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540670 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 13:23:11.570101 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540673 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 13:23:11.570101 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540677 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 13:23:11.570101 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540681 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 13:23:11.570101 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540684 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 13:23:11.570101 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540688 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 13:23:11.570101 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540692 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 13:23:11.570101 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540695 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 13:23:11.570101 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540699 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 13:23:11.570101 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540703 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 13:23:11.570101 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540707 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 13:23:11.570101 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540710 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 13:23:11.570101 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540715 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 13:23:11.570101 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540720 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 13:23:11.570101 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540724 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 13:23:11.570101 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540728 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 13:23:11.570947 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540731 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 13:23:11.570947 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540735 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 13:23:11.570947 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540739 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 13:23:11.570947 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540743 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 13:23:11.570947 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540746 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 13:23:11.570947 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540750 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 13:23:11.570947 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540754 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 13:23:11.570947 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540758 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 08 13:23:11.570947 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540761 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 13:23:11.570947 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540766 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 13:23:11.570947 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540769 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 13:23:11.570947 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540774 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 13:23:11.570947 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540778 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 13:23:11.570947 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.540784 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 13:23:11.570947 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541334 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 13:23:11.570947 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541342 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 13:23:11.570947 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541346 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 13:23:11.570947 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541350 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 13:23:11.570947 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541354 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 13:23:11.570947 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541360 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 13:23:11.571652 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541365 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 13:23:11.571652 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541369 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 13:23:11.571652 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541373 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 13:23:11.571652 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541378 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 13:23:11.571652 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541382 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 13:23:11.571652 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541385 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 13:23:11.571652 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541390 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 13:23:11.571652 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541394 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 13:23:11.571652 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541399 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 13:23:11.571652 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541405 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 13:23:11.571652 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541411 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 13:23:11.571652 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541416 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 13:23:11.571652 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541420 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 13:23:11.571652 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541424 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 13:23:11.571652 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541428 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 13:23:11.571652 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541432 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 13:23:11.571652 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541436 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 13:23:11.571652 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541440 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 13:23:11.571652 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541444 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 13:23:11.574867 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541447 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 13:23:11.574867 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541451 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 13:23:11.574867 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541455 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 13:23:11.574867 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541459 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 13:23:11.574867 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541463 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 13:23:11.574867 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541467 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 13:23:11.574867 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541470 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 13:23:11.574867 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541475 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 13:23:11.574867 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541479 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 13:23:11.574867 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541483 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 13:23:11.574867 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541487 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 13:23:11.574867 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541491 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 13:23:11.574867 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541495 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 13:23:11.574867 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541499 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 13:23:11.574867 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541503 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 13:23:11.574867 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541506 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 13:23:11.574867 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541509 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 13:23:11.574867 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541512 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 13:23:11.574867 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541514 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 13:23:11.574867 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541517 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 13:23:11.575510 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541520 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 13:23:11.575510 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541524 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 13:23:11.575510 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541528 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 13:23:11.575510 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541532 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 13:23:11.575510 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541536 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 13:23:11.575510 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541540 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 13:23:11.575510 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541544 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 13:23:11.575510 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541548 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 13:23:11.575510 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541551 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 13:23:11.575510 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541555 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 13:23:11.575510 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541559 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 13:23:11.575510 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541562 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 08 13:23:11.575510 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541566 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 13:23:11.575510 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541570 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 13:23:11.575510 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541573 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 13:23:11.575510 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541576 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 13:23:11.575510 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541580 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 13:23:11.575510 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541584 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 13:23:11.575510 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541588 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 13:23:11.575510 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541591 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 13:23:11.577205 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541598 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 13:23:11.577205 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541601 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 13:23:11.577205 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541605 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 13:23:11.577205 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541608 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 13:23:11.577205 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541612 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 13:23:11.577205 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541616 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 13:23:11.577205 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541620 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 13:23:11.577205 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541623 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 13:23:11.577205 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541627 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 13:23:11.577205 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541630 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 13:23:11.577205 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541637 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 13:23:11.577205 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541641 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 13:23:11.577205 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541645 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 13:23:11.577205 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541650 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 13:23:11.577205 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541653 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 13:23:11.577205 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541657 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 13:23:11.577205 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541660 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 13:23:11.577205 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541665 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 13:23:11.577205 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541669 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 13:23:11.577205 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541673 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 13:23:11.578042 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541677 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 13:23:11.578042 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541680 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 13:23:11.578042 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541684 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 13:23:11.578042 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541687 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 13:23:11.578042 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541691 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 13:23:11.578042 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541695 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 13:23:11.578042 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541699 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 13:23:11.578042 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541702 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 13:23:11.578042 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541706 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 13:23:11.578042 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541710 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 13:23:11.578042 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541713 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 13:23:11.578042 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541717 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 13:23:11.578042 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541722 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 13:23:11.578042 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.541726 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 13:23:11.578042 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.541813 2567 flags.go:64] FLAG: --address="0.0.0.0" Jul 08 13:23:11.578042 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.541822 2567 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 08 13:23:11.578042 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.541830 2567 flags.go:64] FLAG: --anonymous-auth="true" Jul 08 13:23:11.578042 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.541835 2567 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 08 13:23:11.578042 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.541840 2567 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 08 13:23:11.578042 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.541845 2567 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 08 13:23:11.578042 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.541851 2567 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 08 13:23:11.579304 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.541857 2567 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 08 13:23:11.579304 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.541863 2567 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 08 13:23:11.579304 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.541867 2567 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 08 13:23:11.579304 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.541872 2567 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 08 13:23:11.579304 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.541877 2567 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 08 13:23:11.579304 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.541882 2567 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 08 13:23:11.579304 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.541887 2567 flags.go:64] FLAG: --cgroup-root="" Jul 08 13:23:11.579304 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.541890 2567 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 08 13:23:11.579304 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.541894 2567 flags.go:64] FLAG: --client-ca-file="" Jul 08 13:23:11.579304 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.541899 2567 flags.go:64] FLAG: --cloud-provider="external" Jul 08 13:23:11.579304 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.541904 2567 flags.go:64] FLAG: --cluster-dns="[]" Jul 08 13:23:11.579304 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.541910 2567 flags.go:64] FLAG: --cluster-domain="" Jul 08 13:23:11.579304 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.541914 2567 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 08 13:23:11.579304 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.541919 2567 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 08 13:23:11.579304 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.541923 2567 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 08 13:23:11.579304 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.541928 2567 flags.go:64] FLAG: --container-log-max-files="5" Jul 08 13:23:11.579304 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.541933 2567 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 08 13:23:11.579304 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.541937 2567 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 08 13:23:11.579304 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.541942 2567 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 08 13:23:11.579304 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.541947 2567 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 08 13:23:11.579304 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.541951 2567 flags.go:64] FLAG: --contention-profiling="false" Jul 08 13:23:11.579304 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.541956 2567 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 08 13:23:11.579304 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.541959 2567 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 08 13:23:11.579304 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.541964 2567 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 08 13:23:11.580379 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.541966 2567 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 08 13:23:11.580379 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.541971 2567 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 08 13:23:11.580379 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.541974 2567 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 08 13:23:11.580379 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.541977 2567 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 08 13:23:11.580379 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.541980 2567 flags.go:64] FLAG: --enable-load-reader="false" Jul 08 13:23:11.580379 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.541983 2567 flags.go:64] FLAG: --enable-server="true" Jul 08 13:23:11.580379 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.541986 2567 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 08 13:23:11.580379 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.541990 2567 flags.go:64] FLAG: --event-burst="100" Jul 08 13:23:11.580379 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.541993 2567 flags.go:64] FLAG: --event-qps="50" Jul 08 13:23:11.580379 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.541997 2567 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 08 13:23:11.580379 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542000 2567 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 08 13:23:11.580379 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542005 2567 flags.go:64] FLAG: --eviction-hard="" Jul 08 13:23:11.580379 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542010 2567 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 08 13:23:11.580379 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542015 2567 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 08 13:23:11.580379 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542019 2567 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 08 13:23:11.580379 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542024 2567 flags.go:64] FLAG: --eviction-soft="" Jul 08 13:23:11.580379 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542028 2567 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 08 13:23:11.580379 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542033 2567 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 08 13:23:11.580379 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542038 2567 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 08 13:23:11.580379 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542041 2567 flags.go:64] FLAG: --experimental-mounter-path="" Jul 08 13:23:11.580379 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542044 2567 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 08 13:23:11.580379 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542047 2567 flags.go:64] FLAG: --fail-swap-on="true" Jul 08 13:23:11.580379 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542050 2567 flags.go:64] FLAG: --feature-gates="" Jul 08 13:23:11.580379 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542054 2567 flags.go:64] FLAG: --file-check-frequency="20s" Jul 08 13:23:11.580379 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542057 2567 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 08 13:23:11.581883 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542060 2567 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 08 13:23:11.581883 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542063 2567 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 08 13:23:11.581883 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542066 2567 flags.go:64] FLAG: --healthz-port="10248" Jul 08 13:23:11.581883 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542069 2567 flags.go:64] FLAG: --help="false" Jul 08 13:23:11.581883 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542086 2567 flags.go:64] FLAG: --hostname-override="ip-10-0-137-164.ec2.internal" Jul 08 13:23:11.581883 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542090 2567 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 08 13:23:11.581883 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542095 2567 flags.go:64] FLAG: --http-check-frequency="20s" Jul 08 13:23:11.581883 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542098 2567 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 08 13:23:11.581883 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542101 2567 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 08 13:23:11.581883 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542105 2567 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 08 13:23:11.581883 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542108 2567 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 08 13:23:11.581883 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542110 2567 flags.go:64] FLAG: --image-service-endpoint="" Jul 08 13:23:11.581883 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542113 2567 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 08 13:23:11.581883 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542116 2567 flags.go:64] FLAG: --kube-api-burst="100" Jul 08 13:23:11.581883 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542119 2567 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 08 13:23:11.581883 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542122 2567 flags.go:64] FLAG: --kube-api-qps="50" Jul 08 13:23:11.581883 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542126 2567 flags.go:64] FLAG: --kube-reserved="" Jul 08 13:23:11.581883 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542129 2567 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 08 13:23:11.581883 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542132 2567 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 08 13:23:11.581883 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542135 2567 flags.go:64] FLAG: --kubelet-cgroups="" Jul 08 13:23:11.581883 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542138 2567 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 08 13:23:11.581883 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542141 2567 flags.go:64] FLAG: --lock-file="" Jul 08 13:23:11.581883 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542143 2567 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 08 13:23:11.581883 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542146 2567 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 08 13:23:11.582708 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542149 2567 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 08 13:23:11.582708 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542154 2567 flags.go:64] FLAG: --log-json-split-stream="false" Jul 08 13:23:11.582708 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542158 2567 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 08 13:23:11.582708 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542160 2567 flags.go:64] FLAG: --log-text-split-stream="false" Jul 08 13:23:11.582708 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542163 2567 flags.go:64] FLAG: --logging-format="text" Jul 08 13:23:11.582708 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542166 2567 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 08 13:23:11.582708 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542169 2567 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 08 13:23:11.582708 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542172 2567 flags.go:64] FLAG: --manifest-url="" Jul 08 13:23:11.582708 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542174 2567 flags.go:64] FLAG: --manifest-url-header="" Jul 08 13:23:11.582708 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542179 2567 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 08 13:23:11.582708 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542182 2567 flags.go:64] FLAG: --max-open-files="1000000" Jul 08 13:23:11.582708 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542186 2567 flags.go:64] FLAG: --max-pods="110" Jul 08 13:23:11.582708 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542189 2567 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 08 13:23:11.582708 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542192 2567 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 08 13:23:11.582708 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542196 2567 flags.go:64] FLAG: --memory-manager-policy="None" Jul 08 13:23:11.582708 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542199 2567 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 08 13:23:11.582708 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542202 2567 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 08 13:23:11.582708 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542205 2567 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 08 13:23:11.582708 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542208 2567 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 08 13:23:11.582708 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542215 2567 flags.go:64] FLAG: --node-status-max-images="50" Jul 08 13:23:11.582708 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542218 2567 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 08 13:23:11.582708 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542221 2567 flags.go:64] FLAG: --oom-score-adj="-999" Jul 08 13:23:11.582708 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542224 2567 flags.go:64] FLAG: --pod-cidr="" Jul 08 13:23:11.584010 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542226 2567 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 08 13:23:11.584010 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542231 2567 flags.go:64] FLAG: --pod-manifest-path="" Jul 08 13:23:11.584010 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542235 2567 flags.go:64] FLAG: --pod-max-pids="-1" Jul 08 13:23:11.584010 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542238 2567 flags.go:64] FLAG: --pods-per-core="0" Jul 08 13:23:11.584010 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542241 2567 flags.go:64] FLAG: --port="10250" Jul 08 13:23:11.584010 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542244 2567 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 08 13:23:11.584010 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542247 2567 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0e0b4f2a68f1d95db" Jul 08 13:23:11.584010 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542250 2567 flags.go:64] FLAG: --qos-reserved="" Jul 08 13:23:11.584010 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542253 2567 flags.go:64] FLAG: --read-only-port="10255" Jul 08 13:23:11.584010 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542256 2567 flags.go:64] FLAG: --register-node="true" Jul 08 13:23:11.584010 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542259 2567 flags.go:64] FLAG: --register-with-taints="" Jul 08 13:23:11.584010 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542262 2567 flags.go:64] FLAG: --registry-burst="10" Jul 08 13:23:11.584010 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542266 2567 flags.go:64] FLAG: --registry-qps="5" Jul 08 13:23:11.584010 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542268 2567 flags.go:64] FLAG: --reserved-cpus="" Jul 08 13:23:11.584010 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542271 2567 flags.go:64] FLAG: --reserved-memory="" Jul 08 13:23:11.584010 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542275 2567 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 08 13:23:11.584010 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542278 2567 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 08 13:23:11.584010 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542281 2567 flags.go:64] FLAG: --rotate-certificates="false" Jul 08 13:23:11.584010 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542284 2567 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 08 13:23:11.584010 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542286 2567 flags.go:64] FLAG: --runonce="false" Jul 08 13:23:11.584010 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542289 2567 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 08 13:23:11.584010 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542292 2567 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 08 13:23:11.584010 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542295 2567 flags.go:64] FLAG: --seccomp-default="false" Jul 08 13:23:11.584010 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542299 2567 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 08 13:23:11.584010 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542301 2567 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 08 13:23:11.584847 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542304 2567 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 08 13:23:11.584847 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542307 2567 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 08 13:23:11.584847 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542310 2567 flags.go:64] FLAG: --storage-driver-password="root" Jul 08 13:23:11.584847 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542313 2567 flags.go:64] FLAG: --storage-driver-secure="false" Jul 08 13:23:11.584847 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542316 2567 flags.go:64] FLAG: --storage-driver-table="stats" Jul 08 13:23:11.584847 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542318 2567 flags.go:64] FLAG: --storage-driver-user="root" Jul 08 13:23:11.584847 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542321 2567 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 08 13:23:11.584847 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542324 2567 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 08 13:23:11.584847 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542327 2567 flags.go:64] FLAG: --system-cgroups="" Jul 08 13:23:11.584847 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542330 2567 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 08 13:23:11.584847 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542336 2567 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 08 13:23:11.584847 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542338 2567 flags.go:64] FLAG: --tls-cert-file="" Jul 08 13:23:11.584847 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542341 2567 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 08 13:23:11.584847 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542345 2567 flags.go:64] FLAG: --tls-min-version="" Jul 08 13:23:11.584847 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542349 2567 flags.go:64] FLAG: --tls-private-key-file="" Jul 08 13:23:11.584847 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542351 2567 flags.go:64] FLAG: --topology-manager-policy="none" Jul 08 13:23:11.584847 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542354 2567 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 08 13:23:11.584847 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542357 2567 flags.go:64] FLAG: --topology-manager-scope="container" Jul 08 13:23:11.584847 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542359 2567 flags.go:64] FLAG: --v="2" Jul 08 13:23:11.584847 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542363 2567 flags.go:64] FLAG: --version="false" Jul 08 13:23:11.584847 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542368 2567 flags.go:64] FLAG: --vmodule="" Jul 08 13:23:11.584847 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542371 2567 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 08 13:23:11.584847 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.542374 2567 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 08 13:23:11.584847 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542503 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 13:23:11.585710 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542508 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 13:23:11.585710 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542512 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 13:23:11.585710 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542515 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 13:23:11.585710 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542518 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 13:23:11.585710 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542521 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 13:23:11.585710 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542524 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 13:23:11.585710 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542528 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 13:23:11.585710 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542530 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 13:23:11.585710 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542533 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 13:23:11.585710 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542536 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 13:23:11.585710 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542538 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 13:23:11.585710 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542541 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 13:23:11.585710 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542544 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 13:23:11.585710 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542546 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 13:23:11.585710 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542548 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 13:23:11.585710 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542551 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 13:23:11.585710 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542554 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 13:23:11.585710 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542558 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 13:23:11.585710 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542561 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 13:23:11.585710 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542564 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 13:23:11.586342 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542566 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 13:23:11.586342 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542569 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 13:23:11.586342 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542571 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 13:23:11.586342 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542574 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 13:23:11.586342 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542576 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 13:23:11.586342 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542579 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 13:23:11.586342 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542582 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 13:23:11.586342 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542584 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 13:23:11.586342 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542586 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 13:23:11.586342 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542589 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 13:23:11.586342 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542592 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 13:23:11.586342 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542594 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 13:23:11.586342 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542597 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 13:23:11.586342 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542599 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 13:23:11.586342 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542602 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 13:23:11.586342 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542604 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 13:23:11.586342 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542607 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 13:23:11.586342 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542609 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 13:23:11.586342 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542613 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 13:23:11.586342 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542616 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 13:23:11.586342 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542618 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 13:23:11.586927 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542621 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 13:23:11.586927 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542623 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 13:23:11.586927 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542626 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 13:23:11.586927 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542629 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 13:23:11.586927 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542631 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 13:23:11.586927 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542634 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 13:23:11.586927 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542636 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 13:23:11.586927 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542639 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 13:23:11.586927 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542642 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 13:23:11.586927 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542645 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 13:23:11.586927 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542648 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 13:23:11.586927 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542650 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 13:23:11.586927 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542653 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 13:23:11.586927 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542655 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 13:23:11.586927 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542658 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 13:23:11.586927 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542660 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 13:23:11.586927 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542663 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 13:23:11.586927 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542666 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 08 13:23:11.586927 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542668 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 13:23:11.587530 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542671 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 13:23:11.587530 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542673 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 13:23:11.587530 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542676 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 13:23:11.587530 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542679 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 13:23:11.587530 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542681 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 13:23:11.587530 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542684 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 13:23:11.587530 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542686 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 13:23:11.587530 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542689 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 13:23:11.587530 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542692 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 13:23:11.587530 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542694 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 13:23:11.587530 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542698 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 13:23:11.587530 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542700 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 13:23:11.587530 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542702 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 13:23:11.587530 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542705 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 13:23:11.587530 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542707 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 13:23:11.587530 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542710 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 13:23:11.587530 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542712 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 13:23:11.587530 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542715 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 13:23:11.587530 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542718 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 13:23:11.589216 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542720 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 13:23:11.589216 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542723 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 13:23:11.589216 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542726 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 13:23:11.589216 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542729 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 13:23:11.589216 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542731 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 13:23:11.589216 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542734 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 13:23:11.589216 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542737 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 13:23:11.589216 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542739 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 13:23:11.589216 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542741 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 13:23:11.589216 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542744 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 13:23:11.589216 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542746 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 13:23:11.589216 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542749 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 13:23:11.589216 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542753 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 13:23:11.589216 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542756 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 13:23:11.589216 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542759 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 13:23:11.589216 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542762 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 13:23:11.589216 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542766 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 13:23:11.589216 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542770 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 13:23:11.589216 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.542773 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 13:23:11.590873 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.543459 2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 13:23:11.590873 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.550291 2567 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 08 13:23:11.590873 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.550307 2567 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 08 13:23:11.590873 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.550321 2567 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 08 13:23:11.590873 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.550326 2567 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 08 13:23:11.590873 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550384 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 13:23:11.590873 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550388 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 13:23:11.590873 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550391 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 13:23:11.590873 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550395 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 13:23:11.590873 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550397 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 13:23:11.590873 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550402 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 13:23:11.590873 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550406 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 13:23:11.590873 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550409 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 13:23:11.590873 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550413 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 13:23:11.591299 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550415 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 13:23:11.591299 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550418 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 13:23:11.591299 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550421 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 13:23:11.591299 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550424 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 13:23:11.591299 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550427 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 13:23:11.591299 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550429 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 13:23:11.591299 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550432 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 13:23:11.591299 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550434 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 13:23:11.591299 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550437 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 13:23:11.591299 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550440 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 13:23:11.591299 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550442 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 13:23:11.591299 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550445 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 13:23:11.591299 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550447 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 13:23:11.591299 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550449 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 13:23:11.591299 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550452 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 13:23:11.591299 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550455 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 13:23:11.591299 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550458 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 13:23:11.591299 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550461 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 13:23:11.591299 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550464 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 13:23:11.591299 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550467 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 13:23:11.591935 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550470 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 13:23:11.591935 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550472 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 13:23:11.591935 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550475 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 13:23:11.591935 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550478 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 13:23:11.591935 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550480 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 13:23:11.591935 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550483 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 13:23:11.591935 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550485 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 13:23:11.591935 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550488 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 13:23:11.591935 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550490 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 13:23:11.591935 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550493 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 13:23:11.591935 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550495 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 13:23:11.591935 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550498 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 13:23:11.591935 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550500 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 13:23:11.591935 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550504 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 08 13:23:11.591935 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550506 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 13:23:11.591935 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550509 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 13:23:11.591935 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550511 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 13:23:11.591935 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550513 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 13:23:11.591935 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550516 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 13:23:11.591935 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550519 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 13:23:11.594499 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550521 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 13:23:11.594499 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550524 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 13:23:11.594499 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550526 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 13:23:11.594499 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550529 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 13:23:11.594499 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550532 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 13:23:11.594499 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550535 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 13:23:11.594499 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550538 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 13:23:11.594499 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550540 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 13:23:11.594499 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550544 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 13:23:11.594499 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550546 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 13:23:11.594499 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550549 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 13:23:11.594499 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550554 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 13:23:11.594499 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550557 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 13:23:11.594499 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550559 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 13:23:11.594499 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550562 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 13:23:11.594499 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550565 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 13:23:11.594499 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550567 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 13:23:11.594499 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550570 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 13:23:11.594499 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550572 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 13:23:11.594499 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550575 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 13:23:11.595333 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550577 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 13:23:11.595333 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550580 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 13:23:11.595333 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550582 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 13:23:11.595333 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550585 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 13:23:11.595333 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550588 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 13:23:11.595333 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550591 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 13:23:11.595333 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550593 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 13:23:11.595333 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550602 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 13:23:11.595333 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550605 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 13:23:11.595333 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550607 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 13:23:11.595333 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550610 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 13:23:11.595333 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550613 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 13:23:11.595333 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550616 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 13:23:11.595333 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550619 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 13:23:11.595333 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550623 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 13:23:11.595333 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550627 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 13:23:11.595333 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550631 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 13:23:11.595333 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550634 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 13:23:11.595333 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550637 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 13:23:11.595983 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550640 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 13:23:11.595983 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550643 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 13:23:11.595983 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550646 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 13:23:11.595983 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550649 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 13:23:11.595983 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550653 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 13:23:11.595983 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550656 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 13:23:11.595983 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550659 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 13:23:11.595983 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550662 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 13:23:11.595983 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550665 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 13:23:11.595983 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550667 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 13:23:11.595983 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550670 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 13:23:11.595983 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.550675 2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 13:23:11.595983 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550790 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 13:23:11.595983 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550797 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 13:23:11.595983 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550801 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 13:23:11.676287 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550804 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 13:23:11.676287 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550807 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 13:23:11.676287 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550810 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 13:23:11.676287 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550812 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 13:23:11.676287 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550815 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 13:23:11.676287 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550817 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 13:23:11.676287 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550820 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 13:23:11.676287 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550823 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 13:23:11.676287 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550825 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 13:23:11.676287 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550828 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 13:23:11.676287 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550830 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 13:23:11.676287 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550833 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 13:23:11.676287 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550835 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 13:23:11.676287 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550838 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 13:23:11.676287 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550840 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 13:23:11.676287 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550843 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 13:23:11.676287 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550845 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 13:23:11.676287 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550848 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 13:23:11.676287 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550851 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 13:23:11.676287 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550853 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 13:23:11.609091 ip-10-0-137-164 systemd[1]: Started Kubernetes Kubelet. Jul 08 13:23:11.676992 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550856 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 13:23:11.676992 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550858 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 13:23:11.676992 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550862 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 13:23:11.676992 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550865 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 13:23:11.676992 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550868 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 13:23:11.676992 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550870 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 13:23:11.676992 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550873 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 13:23:11.676992 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550875 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 13:23:11.676992 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550878 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 13:23:11.676992 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550880 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 13:23:11.676992 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550885 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 13:23:11.676992 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550887 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 13:23:11.676992 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550890 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 13:23:11.676992 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550892 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 13:23:11.676992 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550895 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 13:23:11.676992 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550897 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 13:23:11.676992 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550900 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 13:23:11.676992 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550902 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 13:23:11.676992 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550905 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 13:23:11.676992 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550907 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 13:23:11.677718 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550910 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 13:23:11.677718 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550912 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 13:23:11.677718 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550915 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 13:23:11.677718 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550918 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 13:23:11.677718 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550920 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 13:23:11.677718 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550923 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 13:23:11.677718 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550925 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 13:23:11.677718 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550928 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 08 13:23:11.677718 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550930 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 13:23:11.677718 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550933 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 13:23:11.677718 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550936 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 13:23:11.677718 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550940 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 13:23:11.677718 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550943 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 13:23:11.677718 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550945 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 13:23:11.677718 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550949 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 13:23:11.677718 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550952 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 13:23:11.677718 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550955 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 13:23:11.677718 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550957 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 13:23:11.677718 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550960 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 13:23:11.678384 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550963 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 13:23:11.678384 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550966 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 13:23:11.678384 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550968 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 13:23:11.678384 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550972 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 13:23:11.678384 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550975 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 13:23:11.678384 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550977 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 13:23:11.678384 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550981 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 13:23:11.678384 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550984 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 13:23:11.678384 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550987 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 13:23:11.678384 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550990 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 13:23:11.678384 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550992 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 13:23:11.678384 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550995 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 13:23:11.678384 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.550997 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 13:23:11.678384 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.551000 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 13:23:11.678384 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.551002 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 13:23:11.678384 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.551005 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 13:23:11.678384 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.551007 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 13:23:11.678384 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.551010 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 13:23:11.678384 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.551012 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 13:23:11.678962 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.551015 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 13:23:11.678962 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.551017 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 13:23:11.678962 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.551020 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 13:23:11.678962 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.551022 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 13:23:11.678962 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.551025 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 13:23:11.678962 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.551027 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 13:23:11.678962 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.551029 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 13:23:11.678962 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.551032 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 13:23:11.678962 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.551037 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 13:23:11.678962 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.551040 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 13:23:11.678962 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.551042 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 13:23:11.678962 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.551045 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 13:23:11.678962 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.551047 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 13:23:11.678962 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.551050 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 13:23:11.678962 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.551052 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 13:23:11.678962 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.551055 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 13:23:11.678962 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.551059 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 13:23:11.678962 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:11.551061 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 13:23:11.679546 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.551066 2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 13:23:11.679546 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.552152 2567 server.go:962] "Client rotation is on, will bootstrap in background" Jul 08 13:23:11.679546 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.555175 2567 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 08 13:23:11.679546 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.556535 2567 server.go:1019] "Starting client certificate rotation" Jul 08 13:23:11.679546 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.556634 2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 13:23:11.679546 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.556666 2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 13:23:11.679546 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.558981 2567 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 13:23:11.679546 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.563906 2567 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 13:23:11.679546 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.573827 2567 log.go:25] "Validated CRI v1 runtime API" Jul 08 13:23:11.679546 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.576038 2567 log.go:25] "Validated CRI v1 image API" Jul 08 13:23:11.679546 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.577643 2567 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 08 13:23:11.679546 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.582250 2567 fs.go:135] Filesystem UUIDs: map[18038e17-3d21-4a93-806a-0b77f24cc163:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 c8e95908-ac73-4b52-955b-3224aedbcab2:/dev/nvme0n1p3] Jul 08 13:23:11.679981 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.582266 2567 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 08 13:23:11.679981 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.586753 2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 13:23:11.680295 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.587210 2567 manager.go:216] Machine: {Timestamp:2026-07-08 13:23:11.58602071 +0000 UTC m=+0.382961358 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101153 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2dcfbe328f2c947dd73ff807c01e44 SystemUUID:ec2dcfbe-328f-2c94-7dd7-3ff807c01e44 BootID:30842a19-e07b-43d3-9572-2096c3a68bbe Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:15:a6:e4:d6:15 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:15:a6:e4:d6:15 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:9e:9a:86:70:f6:40 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 08 13:23:11.680295 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.587314 2567 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 08 13:23:11.680295 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.587393 2567 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 08 13:23:11.680295 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.590428 2567 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 08 13:23:11.680295 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.590455 2567 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-164.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 08 13:23:11.680295 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.591239 2567 topology_manager.go:138] "Creating topology manager with none policy" Jul 08 13:23:11.680295 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.591249 2567 container_manager_linux.go:309] "Creating device plugin manager" Jul 08 13:23:11.680295 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.591256 2567 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 13:23:11.680295 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.591841 2567 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 13:23:11.680295 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.593039 2567 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 08 13:23:11.680295 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.595874 2567 state_mem.go:36] "Initialized new in-memory state store" Jul 08 13:23:11.680295 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.596005 2567 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 08 13:23:11.680295 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.596030 2567 kubelet.go:486] "Attempting to sync node with API server" Jul 08 13:23:11.680295 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.596040 2567 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 08 13:23:11.680295 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.596052 2567 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 08 13:23:11.680295 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.596060 2567 kubelet.go:398] "Adding apiserver pod source" Jul 08 13:23:11.680295 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.596068 2567 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 08 13:23:11.680295 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.596421 2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 13:23:11.680920 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.596436 2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 13:23:11.680920 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.599922 2567 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 08 13:23:11.680920 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.602116 2567 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 08 13:23:11.680920 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.602143 2567 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 08 13:23:11.680920 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.603189 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 08 13:23:11.680920 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.603202 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 08 13:23:11.680920 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.603208 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 08 13:23:11.680920 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.603213 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 08 13:23:11.680920 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.603221 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 08 13:23:11.680920 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.603226 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 08 13:23:11.680920 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.603234 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 08 13:23:11.680920 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.603239 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 08 13:23:11.680920 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.603245 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 08 13:23:11.680920 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.603250 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 08 13:23:11.680920 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.603264 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 08 13:23:11.680920 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.603269 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 08 13:23:11.680920 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.604639 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 08 13:23:11.680920 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.604652 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 08 13:23:11.680920 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.607355 2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-q9fsn" Jul 08 13:23:11.680920 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.608103 2567 server.go:1268] "Started kubelet" Jul 08 13:23:11.680920 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.608179 2567 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 08 13:23:11.680920 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.608660 2567 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 08 13:23:11.680920 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.608739 2567 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 08 13:23:11.680920 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.609963 2567 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 08 13:23:11.680920 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.610474 2567 server.go:310] "Adding debug handlers to kubelet server" Jul 08 13:23:11.680920 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:11.614233 2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-164.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 08 13:23:11.680920 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:11.614413 2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 08 13:23:11.680920 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.614515 2567 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-164.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 08 13:23:11.680920 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:11.614781 2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-164.ec2.internal.18c053411fac00e3 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-164.ec2.internal,UID:ip-10-0-137-164.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-164.ec2.internal,},FirstTimestamp:2026-07-08 13:23:11.608062179 +0000 UTC m=+0.405002827,LastTimestamp:2026-07-08 13:23:11.608062179 +0000 UTC m=+0.405002827,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-164.ec2.internal,}" Jul 08 13:23:11.680920 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.619206 2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-q9fsn" Jul 08 13:23:11.680920 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:11.619641 2567 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 08 13:23:11.680920 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.621926 2567 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 08 13:23:11.680920 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.622380 2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 08 13:23:11.680920 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.623172 2567 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 08 13:23:11.680920 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:11.623203 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-164.ec2.internal\" not found" Jul 08 13:23:11.680920 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.623870 2567 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 08 13:23:11.680920 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.623889 2567 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 08 13:23:11.680920 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.624008 2567 reconstruct.go:97] "Volume reconstruction finished" Jul 08 13:23:11.680920 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.624015 2567 reconciler.go:29] "Reconciler: start to sync state" Jul 08 13:23:11.680920 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.625853 2567 factory.go:55] Registering systemd factory Jul 08 13:23:11.682120 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.625866 2567 factory.go:223] Registration of the systemd container factory successfully Jul 08 13:23:11.682120 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.626121 2567 factory.go:153] Registering CRI-O factory Jul 08 13:23:11.682120 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.626134 2567 factory.go:223] Registration of the crio container factory successfully Jul 08 13:23:11.682120 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.626179 2567 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 08 13:23:11.682120 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.626196 2567 factory.go:103] Registering Raw factory Jul 08 13:23:11.682120 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.626208 2567 manager.go:1196] Started watching for new ooms in manager Jul 08 13:23:11.682120 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.627025 2567 manager.go:349] Starting recovery of all containers Jul 08 13:23:11.682120 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.631265 2567 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 13:23:11.682120 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:11.633606 2567 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-137-164.ec2.internal\" not found" node="ip-10-0-137-164.ec2.internal" Jul 08 13:23:11.682120 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.645915 2567 manager.go:354] Recovery completed Jul 08 13:23:11.682120 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.651024 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 13:23:11.682120 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.653101 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-164.ec2.internal" event="NodeHasSufficientMemory" Jul 08 13:23:11.682120 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.653126 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-164.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 13:23:11.682120 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.653139 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-164.ec2.internal" event="NodeHasSufficientPID" Jul 08 13:23:11.682120 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.653692 2567 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 08 13:23:11.682120 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.653701 2567 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 08 13:23:11.682120 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.653720 2567 state_mem.go:36] "Initialized new in-memory state store" Jul 08 13:23:11.682120 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.654904 2567 policy_none.go:49] "None policy: Start" Jul 08 13:23:11.682120 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.654917 2567 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 08 13:23:11.682120 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.654929 2567 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 08 13:23:11.682120 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.655438 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 13:23:11.682120 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.655907 2567 policy_none.go:47] "Start" Jul 08 13:23:11.682120 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.657249 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-164.ec2.internal" event="NodeHasSufficientMemory" Jul 08 13:23:11.682120 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.657267 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-164.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 13:23:11.682120 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.657277 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-164.ec2.internal" event="NodeHasSufficientPID" Jul 08 13:23:11.798925 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.688843 2567 manager.go:337] "Starting Device Plugin manager" Jul 08 13:23:11.798925 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:11.689100 2567 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 08 13:23:11.798925 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.689116 2567 server.go:87] "Starting device plugin registration server" Jul 08 13:23:11.798925 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.689533 2567 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 08 13:23:11.798925 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.689545 2567 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 08 13:23:11.798925 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.689760 2567 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 08 13:23:11.798925 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.689840 2567 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 08 13:23:11.798925 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.689848 2567 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 08 13:23:11.798925 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:11.690578 2567 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 08 13:23:11.798925 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:11.690636 2567 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-164.ec2.internal\" not found" Jul 08 13:23:11.798925 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.788085 2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 08 13:23:11.798925 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.789438 2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 08 13:23:11.798925 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.789472 2567 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 08 13:23:11.798925 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.789497 2567 kubelet.go:2448] "Starting kubelet main sync loop" Jul 08 13:23:11.798925 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:11.789530 2567 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 08 13:23:11.798925 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.790617 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 13:23:11.798925 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.791670 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-164.ec2.internal" event="NodeHasSufficientMemory" Jul 08 13:23:11.798925 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.791709 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-164.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 13:23:11.798925 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.791731 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-164.ec2.internal" event="NodeHasSufficientPID" Jul 08 13:23:11.798925 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.791766 2567 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-164.ec2.internal" Jul 08 13:23:11.798925 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.792082 2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 13:23:11.814504 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.802356 2567 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-164.ec2.internal" Jul 08 13:23:11.814504 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:11.802377 2567 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-164.ec2.internal\": node \"ip-10-0-137-164.ec2.internal\" not found" Jul 08 13:23:11.818878 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:11.817036 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-164.ec2.internal\" not found" Jul 08 13:23:11.961894 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.893240 2567 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-164.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-164.ec2.internal"] Jul 08 13:23:11.961894 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.893363 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 13:23:11.961894 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.894640 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-164.ec2.internal" event="NodeHasSufficientMemory" Jul 08 13:23:11.961894 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.894669 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-164.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 13:23:11.961894 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.894679 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-164.ec2.internal" event="NodeHasSufficientPID" Jul 08 13:23:11.961894 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.896168 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 13:23:11.961894 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.896282 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-164.ec2.internal" Jul 08 13:23:11.961894 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.896312 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 13:23:11.961894 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.897456 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-164.ec2.internal" event="NodeHasSufficientMemory" Jul 08 13:23:11.961894 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.897487 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-164.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 13:23:11.961894 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.897501 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-164.ec2.internal" event="NodeHasSufficientPID" Jul 08 13:23:11.961894 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.897486 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-164.ec2.internal" event="NodeHasSufficientMemory" Jul 08 13:23:11.961894 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.897571 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-164.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 13:23:11.961894 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.897588 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-164.ec2.internal" event="NodeHasSufficientPID" Jul 08 13:23:11.961894 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.898832 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-164.ec2.internal" Jul 08 13:23:11.961894 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.898864 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 13:23:11.961894 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.899619 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-164.ec2.internal" event="NodeHasSufficientMemory" Jul 08 13:23:11.961894 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.899652 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-164.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 13:23:11.961894 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.899668 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-164.ec2.internal" event="NodeHasSufficientPID" Jul 08 13:23:11.961894 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:11.917757 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-164.ec2.internal\" not found" Jul 08 13:23:11.961894 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.926260 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7264a5fddc17c5fbd72de875bb444234-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-164.ec2.internal\" (UID: \"7264a5fddc17c5fbd72de875bb444234\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-164.ec2.internal" Jul 08 13:23:11.961894 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.926285 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/abd4fe04a7d63176e943bf70844ade69-config\") pod \"kube-apiserver-proxy-ip-10-0-137-164.ec2.internal\" (UID: \"abd4fe04a7d63176e943bf70844ade69\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-164.ec2.internal" Jul 08 13:23:11.961894 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:11.926303 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7264a5fddc17c5fbd72de875bb444234-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-164.ec2.internal\" (UID: \"7264a5fddc17c5fbd72de875bb444234\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-164.ec2.internal" Jul 08 13:23:11.961894 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:11.931315 2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-164.ec2.internal\" not found" node="ip-10-0-137-164.ec2.internal" Jul 08 13:23:11.961894 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:11.936429 2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-164.ec2.internal\" not found" node="ip-10-0-137-164.ec2.internal" Jul 08 13:23:12.089358 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:12.017938 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-164.ec2.internal\" not found" Jul 08 13:23:12.089358 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.026751 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7264a5fddc17c5fbd72de875bb444234-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-164.ec2.internal\" (UID: \"7264a5fddc17c5fbd72de875bb444234\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-164.ec2.internal" Jul 08 13:23:12.089358 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.026777 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7264a5fddc17c5fbd72de875bb444234-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-164.ec2.internal\" (UID: \"7264a5fddc17c5fbd72de875bb444234\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-164.ec2.internal" Jul 08 13:23:12.089358 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.026798 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/abd4fe04a7d63176e943bf70844ade69-config\") pod \"kube-apiserver-proxy-ip-10-0-137-164.ec2.internal\" (UID: \"abd4fe04a7d63176e943bf70844ade69\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-164.ec2.internal" Jul 08 13:23:12.089358 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.026826 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/abd4fe04a7d63176e943bf70844ade69-config\") pod \"kube-apiserver-proxy-ip-10-0-137-164.ec2.internal\" (UID: \"abd4fe04a7d63176e943bf70844ade69\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-164.ec2.internal" Jul 08 13:23:12.089358 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.026864 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7264a5fddc17c5fbd72de875bb444234-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-164.ec2.internal\" (UID: \"7264a5fddc17c5fbd72de875bb444234\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-164.ec2.internal" Jul 08 13:23:12.089358 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.026874 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7264a5fddc17c5fbd72de875bb444234-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-164.ec2.internal\" (UID: \"7264a5fddc17c5fbd72de875bb444234\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-164.ec2.internal" Jul 08 13:23:12.240654 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:12.119125 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-164.ec2.internal\" not found" Jul 08 13:23:12.240654 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:12.226682 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-164.ec2.internal\" not found" Jul 08 13:23:12.240654 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.234434 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-164.ec2.internal" Jul 08 13:23:12.240654 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.240536 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-164.ec2.internal" Jul 08 13:23:12.383077 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:12.327331 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-164.ec2.internal\" not found" Jul 08 13:23:12.430544 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:12.430506 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-164.ec2.internal\" not found" Jul 08 13:23:12.496630 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.496576 2567 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 13:23:12.524246 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.524201 2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-164.ec2.internal" Jul 08 13:23:12.538905 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.538877 2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 13:23:12.541164 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.541143 2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-164.ec2.internal" Jul 08 13:23:12.555539 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.555503 2567 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 08 13:23:12.555738 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.555711 2567 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 13:23:12.555800 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:12.555710 2567 kubelet.go:3250] "Failed creating a mirror pod" err="Post \"https://affb78f60cad14e6fb0028e1ef483bdc-f103e07a3a87b1d0.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/kube-system/pods\": read tcp 10.0.137.164:51346->52.45.32.147:6443: use of closed network connection" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-164.ec2.internal" Jul 08 13:23:12.555800 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.555718 2567 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 13:23:12.555800 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.555734 2567 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 13:23:12.599458 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.599397 2567 apiserver.go:52] "Watching apiserver" Jul 08 13:23:12.618205 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.618176 2567 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 08 13:23:12.618552 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.618527 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-zs2fw","openshift-multus/network-metrics-daemon-cnv6f","openshift-network-diagnostics/network-check-target-rm9f4","openshift-network-operator/iptables-alerter-zfsbx","kube-system/konnectivity-agent-g4lnf","kube-system/kube-apiserver-proxy-ip-10-0-137-164.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpkn4","openshift-image-registry/node-ca-pn22q","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-164.ec2.internal","openshift-multus/multus-85xxk","openshift-multus/multus-additional-cni-plugins-qmwvz","openshift-ovn-kubernetes/ovnkube-node-sqlwq","kube-system/global-pull-secret-syncer-tp2lh","openshift-cluster-node-tuning-operator/tuned-4wjn6"] Jul 08 13:23:12.620338 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.620312 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpkn4" Jul 08 13:23:12.621584 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.621531 2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-07 13:18:11 +0000 UTC" deadline="2027-12-11 08:40:33.921544135 +0000 UTC" Jul 08 13:23:12.621584 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.621576 2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12499h17m21.299971053s" Jul 08 13:23:12.621584 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.621583 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnv6f" Jul 08 13:23:12.621823 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:12.621656 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnv6f" podUID="a3381953-6131-477f-be6d-92ef7dc333a7" Jul 08 13:23:12.622633 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.622612 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 08 13:23:12.622734 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.622674 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 08 13:23:12.622868 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.622853 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 08 13:23:12.622958 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.622943 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-zb85s\"" Jul 08 13:23:12.623297 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.623274 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4wjn6" Jul 08 13:23:12.623384 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.623313 2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 08 13:23:12.624606 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.624578 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qmwvz" Jul 08 13:23:12.625891 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.625658 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 08 13:23:12.625891 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.625668 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 08 13:23:12.625891 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.625721 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-njqq9\"" Jul 08 13:23:12.625891 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.625728 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-g4lnf" Jul 08 13:23:12.626822 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.626807 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 08 13:23:12.626930 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.626915 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-pn22q" Jul 08 13:23:12.627176 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.627135 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 08 13:23:12.627176 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.627137 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-pxf8s\"" Jul 08 13:23:12.627318 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.627252 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 08 13:23:12.627365 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.627322 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 08 13:23:12.627365 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.627348 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 08 13:23:12.628073 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.628054 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 08 13:23:12.628216 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.628073 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-5xcqm\"" Jul 08 13:23:12.628216 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.628054 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 08 13:23:12.628363 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.628349 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-zfsbx" Jul 08 13:23:12.629167 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.629152 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 08 13:23:12.629444 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.629428 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 08 13:23:12.629635 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.629615 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 08 13:23:12.629740 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.629726 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-85xxk" Jul 08 13:23:12.629910 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.629863 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-lrr5t\"" Jul 08 13:23:12.630529 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.630509 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 08 13:23:12.630762 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.630726 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 08 13:23:12.630762 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.630747 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-7qjwb\"" Jul 08 13:23:12.630896 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.630747 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 08 13:23:12.631746 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.631578 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zs2fw" Jul 08 13:23:12.631746 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.631700 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/029e8f8b-d068-4215-882f-0c72f71d5244-etc-sysctl-conf\") pod \"tuned-4wjn6\" (UID: \"029e8f8b-d068-4215-882f-0c72f71d5244\") " pod="openshift-cluster-node-tuning-operator/tuned-4wjn6" Jul 08 13:23:12.632965 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.631752 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ea6986a3-b0b4-4704-a47a-c991094c04f4-cni-binary-copy\") pod \"multus-additional-cni-plugins-qmwvz\" (UID: \"ea6986a3-b0b4-4704-a47a-c991094c04f4\") " pod="openshift-multus/multus-additional-cni-plugins-qmwvz" Jul 08 13:23:12.632965 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.631965 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a3381953-6131-477f-be6d-92ef7dc333a7-metrics-certs\") pod \"network-metrics-daemon-cnv6f\" (UID: \"a3381953-6131-477f-be6d-92ef7dc333a7\") " pod="openshift-multus/network-metrics-daemon-cnv6f" Jul 08 13:23:12.632965 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.632009 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/029e8f8b-d068-4215-882f-0c72f71d5244-etc-sysconfig\") pod \"tuned-4wjn6\" (UID: \"029e8f8b-d068-4215-882f-0c72f71d5244\") " pod="openshift-cluster-node-tuning-operator/tuned-4wjn6" Jul 08 13:23:12.632965 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.632226 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/029e8f8b-d068-4215-882f-0c72f71d5244-host\") pod \"tuned-4wjn6\" (UID: \"029e8f8b-d068-4215-882f-0c72f71d5244\") " pod="openshift-cluster-node-tuning-operator/tuned-4wjn6" Jul 08 13:23:12.632965 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.632285 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fx5fs\" (UniqueName: \"kubernetes.io/projected/029e8f8b-d068-4215-882f-0c72f71d5244-kube-api-access-fx5fs\") pod \"tuned-4wjn6\" (UID: \"029e8f8b-d068-4215-882f-0c72f71d5244\") " pod="openshift-cluster-node-tuning-operator/tuned-4wjn6" Jul 08 13:23:12.632965 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.632330 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4b747c9c-61b0-41b9-ac34-7c8d1e63e6df-registration-dir\") pod \"aws-ebs-csi-driver-node-rpkn4\" (UID: \"4b747c9c-61b0-41b9-ac34-7c8d1e63e6df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpkn4" Jul 08 13:23:12.632965 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.632414 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhdl8\" (UniqueName: \"kubernetes.io/projected/a3381953-6131-477f-be6d-92ef7dc333a7-kube-api-access-nhdl8\") pod \"network-metrics-daemon-cnv6f\" (UID: \"a3381953-6131-477f-be6d-92ef7dc333a7\") " pod="openshift-multus/network-metrics-daemon-cnv6f" Jul 08 13:23:12.632965 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.632457 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/029e8f8b-d068-4215-882f-0c72f71d5244-tmp\") pod \"tuned-4wjn6\" (UID: \"029e8f8b-d068-4215-882f-0c72f71d5244\") " pod="openshift-cluster-node-tuning-operator/tuned-4wjn6" Jul 08 13:23:12.632965 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.632480 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4b747c9c-61b0-41b9-ac34-7c8d1e63e6df-device-dir\") pod \"aws-ebs-csi-driver-node-rpkn4\" (UID: \"4b747c9c-61b0-41b9-ac34-7c8d1e63e6df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpkn4" Jul 08 13:23:12.632965 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.632502 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/029e8f8b-d068-4215-882f-0c72f71d5244-etc-sysctl-d\") pod \"tuned-4wjn6\" (UID: \"029e8f8b-d068-4215-882f-0c72f71d5244\") " pod="openshift-cluster-node-tuning-operator/tuned-4wjn6" Jul 08 13:23:12.632965 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.632523 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/029e8f8b-d068-4215-882f-0c72f71d5244-etc-kubernetes\") pod \"tuned-4wjn6\" (UID: \"029e8f8b-d068-4215-882f-0c72f71d5244\") " pod="openshift-cluster-node-tuning-operator/tuned-4wjn6" Jul 08 13:23:12.632965 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.632544 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ea6986a3-b0b4-4704-a47a-c991094c04f4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qmwvz\" (UID: \"ea6986a3-b0b4-4704-a47a-c991094c04f4\") " pod="openshift-multus/multus-additional-cni-plugins-qmwvz" Jul 08 13:23:12.632965 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.632566 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbp6t\" (UniqueName: \"kubernetes.io/projected/ea6986a3-b0b4-4704-a47a-c991094c04f4-kube-api-access-rbp6t\") pod \"multus-additional-cni-plugins-qmwvz\" (UID: \"ea6986a3-b0b4-4704-a47a-c991094c04f4\") " pod="openshift-multus/multus-additional-cni-plugins-qmwvz" Jul 08 13:23:12.632965 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.632586 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4b747c9c-61b0-41b9-ac34-7c8d1e63e6df-socket-dir\") pod \"aws-ebs-csi-driver-node-rpkn4\" (UID: \"4b747c9c-61b0-41b9-ac34-7c8d1e63e6df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpkn4" Jul 08 13:23:12.632965 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.632761 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbv4p\" (UniqueName: \"kubernetes.io/projected/4b747c9c-61b0-41b9-ac34-7c8d1e63e6df-kube-api-access-kbv4p\") pod \"aws-ebs-csi-driver-node-rpkn4\" (UID: \"4b747c9c-61b0-41b9-ac34-7c8d1e63e6df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpkn4" Jul 08 13:23:12.632965 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.632799 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/029e8f8b-d068-4215-882f-0c72f71d5244-etc-modprobe-d\") pod \"tuned-4wjn6\" (UID: \"029e8f8b-d068-4215-882f-0c72f71d5244\") " pod="openshift-cluster-node-tuning-operator/tuned-4wjn6" Jul 08 13:23:12.633793 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.632853 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/029e8f8b-d068-4215-882f-0c72f71d5244-lib-modules\") pod \"tuned-4wjn6\" (UID: \"029e8f8b-d068-4215-882f-0c72f71d5244\") " pod="openshift-cluster-node-tuning-operator/tuned-4wjn6" Jul 08 13:23:12.633793 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.632884 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/029e8f8b-d068-4215-882f-0c72f71d5244-run\") pod \"tuned-4wjn6\" (UID: \"029e8f8b-d068-4215-882f-0c72f71d5244\") " pod="openshift-cluster-node-tuning-operator/tuned-4wjn6" Jul 08 13:23:12.633793 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.632958 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ea6986a3-b0b4-4704-a47a-c991094c04f4-system-cni-dir\") pod \"multus-additional-cni-plugins-qmwvz\" (UID: \"ea6986a3-b0b4-4704-a47a-c991094c04f4\") " pod="openshift-multus/multus-additional-cni-plugins-qmwvz" Jul 08 13:23:12.633793 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.632997 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ea6986a3-b0b4-4704-a47a-c991094c04f4-cnibin\") pod \"multus-additional-cni-plugins-qmwvz\" (UID: \"ea6986a3-b0b4-4704-a47a-c991094c04f4\") " pod="openshift-multus/multus-additional-cni-plugins-qmwvz" Jul 08 13:23:12.633793 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.633023 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4b747c9c-61b0-41b9-ac34-7c8d1e63e6df-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rpkn4\" (UID: \"4b747c9c-61b0-41b9-ac34-7c8d1e63e6df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpkn4" Jul 08 13:23:12.633793 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.633053 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4b747c9c-61b0-41b9-ac34-7c8d1e63e6df-etc-selinux\") pod \"aws-ebs-csi-driver-node-rpkn4\" (UID: \"4b747c9c-61b0-41b9-ac34-7c8d1e63e6df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpkn4" Jul 08 13:23:12.633793 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.633082 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/029e8f8b-d068-4215-882f-0c72f71d5244-sys\") pod \"tuned-4wjn6\" (UID: \"029e8f8b-d068-4215-882f-0c72f71d5244\") " pod="openshift-cluster-node-tuning-operator/tuned-4wjn6" Jul 08 13:23:12.633793 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.633140 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ea6986a3-b0b4-4704-a47a-c991094c04f4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qmwvz\" (UID: \"ea6986a3-b0b4-4704-a47a-c991094c04f4\") " pod="openshift-multus/multus-additional-cni-plugins-qmwvz" Jul 08 13:23:12.633793 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.633172 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/029e8f8b-d068-4215-882f-0c72f71d5244-etc-systemd\") pod \"tuned-4wjn6\" (UID: \"029e8f8b-d068-4215-882f-0c72f71d5244\") " pod="openshift-cluster-node-tuning-operator/tuned-4wjn6" Jul 08 13:23:12.633793 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.633290 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ea6986a3-b0b4-4704-a47a-c991094c04f4-os-release\") pod \"multus-additional-cni-plugins-qmwvz\" (UID: \"ea6986a3-b0b4-4704-a47a-c991094c04f4\") " pod="openshift-multus/multus-additional-cni-plugins-qmwvz" Jul 08 13:23:12.633793 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.633322 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ea6986a3-b0b4-4704-a47a-c991094c04f4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qmwvz\" (UID: \"ea6986a3-b0b4-4704-a47a-c991094c04f4\") " pod="openshift-multus/multus-additional-cni-plugins-qmwvz" Jul 08 13:23:12.633793 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.633356 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4b747c9c-61b0-41b9-ac34-7c8d1e63e6df-sys-fs\") pod \"aws-ebs-csi-driver-node-rpkn4\" (UID: \"4b747c9c-61b0-41b9-ac34-7c8d1e63e6df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpkn4" Jul 08 13:23:12.633793 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.633385 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/029e8f8b-d068-4215-882f-0c72f71d5244-var-lib-kubelet\") pod \"tuned-4wjn6\" (UID: \"029e8f8b-d068-4215-882f-0c72f71d5244\") " pod="openshift-cluster-node-tuning-operator/tuned-4wjn6" Jul 08 13:23:12.633793 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.633414 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/029e8f8b-d068-4215-882f-0c72f71d5244-etc-tuned\") pod \"tuned-4wjn6\" (UID: \"029e8f8b-d068-4215-882f-0c72f71d5244\") " pod="openshift-cluster-node-tuning-operator/tuned-4wjn6" Jul 08 13:23:12.633793 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.633476 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 08 13:23:12.633793 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.633715 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-rxggx\"" Jul 08 13:23:12.635008 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.634987 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 08 13:23:12.635514 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.635459 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.635603 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.635566 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 08 13:23:12.635745 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.635569 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-kj4bg\"" Jul 08 13:23:12.637939 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.637917 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 08 13:23:12.638017 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.637973 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tp2lh" Jul 08 13:23:12.638081 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.638063 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 08 13:23:12.638117 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:12.638062 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tp2lh" podUID="f93cae56-ca8c-4c37-866c-51c79f237448" Jul 08 13:23:12.638284 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.638258 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 08 13:23:12.638380 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.638259 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 08 13:23:12.638442 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.638428 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-x9qkf\"" Jul 08 13:23:12.638673 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.638660 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 08 13:23:12.639112 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.639097 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 08 13:23:12.639356 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.639339 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rm9f4" Jul 08 13:23:12.639439 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:12.639403 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rm9f4" podUID="b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5" Jul 08 13:23:12.640717 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.640701 2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 13:23:12.660297 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.660271 2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-2kndw" Jul 08 13:23:12.665684 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.665665 2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-2kndw" Jul 08 13:23:12.724456 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.724432 2567 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 08 13:23:12.733719 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.733683 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ea6986a3-b0b4-4704-a47a-c991094c04f4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qmwvz\" (UID: \"ea6986a3-b0b4-4704-a47a-c991094c04f4\") " pod="openshift-multus/multus-additional-cni-plugins-qmwvz" Jul 08 13:23:12.733719 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.733716 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/67e86263-6fde-4f3a-a775-9e29eb8cacef-host\") pod \"node-ca-pn22q\" (UID: \"67e86263-6fde-4f3a-a775-9e29eb8cacef\") " pod="openshift-image-registry/node-ca-pn22q" Jul 08 13:23:12.733981 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.733736 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d492fe73-0e0b-4b21-8ea1-2ab7cfcefe74-hosts-file\") pod \"node-resolver-zs2fw\" (UID: \"d492fe73-0e0b-4b21-8ea1-2ab7cfcefe74\") " pod="openshift-dns/node-resolver-zs2fw" Jul 08 13:23:12.733981 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.733756 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d41d04d6-dc32-43f2-8e7c-24c36daed99f-host-kubelet\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.733981 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.733783 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d41d04d6-dc32-43f2-8e7c-24c36daed99f-ovnkube-config\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.733981 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.733826 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d41d04d6-dc32-43f2-8e7c-24c36daed99f-ovn-node-metrics-cert\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.733981 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.733874 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/029e8f8b-d068-4215-882f-0c72f71d5244-var-lib-kubelet\") pod \"tuned-4wjn6\" (UID: \"029e8f8b-d068-4215-882f-0c72f71d5244\") " pod="openshift-cluster-node-tuning-operator/tuned-4wjn6" Jul 08 13:23:12.733981 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.733901 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/db3380e5-69cf-4c03-b995-6de00f6b269f-host-run-k8s-cni-cncf-io\") pod \"multus-85xxk\" (UID: \"db3380e5-69cf-4c03-b995-6de00f6b269f\") " pod="openshift-multus/multus-85xxk" Jul 08 13:23:12.733981 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.733926 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/db3380e5-69cf-4c03-b995-6de00f6b269f-host-var-lib-kubelet\") pod \"multus-85xxk\" (UID: \"db3380e5-69cf-4c03-b995-6de00f6b269f\") " pod="openshift-multus/multus-85xxk" Jul 08 13:23:12.733981 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.733952 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d41d04d6-dc32-43f2-8e7c-24c36daed99f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.733981 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.733981 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wm2f\" (UniqueName: \"kubernetes.io/projected/b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5-kube-api-access-6wm2f\") pod \"network-check-target-rm9f4\" (UID: \"b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5\") " pod="openshift-network-diagnostics/network-check-target-rm9f4" Jul 08 13:23:12.734379 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.733983 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/029e8f8b-d068-4215-882f-0c72f71d5244-var-lib-kubelet\") pod \"tuned-4wjn6\" (UID: \"029e8f8b-d068-4215-882f-0c72f71d5244\") " pod="openshift-cluster-node-tuning-operator/tuned-4wjn6" Jul 08 13:23:12.734379 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.734013 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/029e8f8b-d068-4215-882f-0c72f71d5244-etc-sysctl-conf\") pod \"tuned-4wjn6\" (UID: \"029e8f8b-d068-4215-882f-0c72f71d5244\") " pod="openshift-cluster-node-tuning-operator/tuned-4wjn6" Jul 08 13:23:12.734379 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.734031 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a3381953-6131-477f-be6d-92ef7dc333a7-metrics-certs\") pod \"network-metrics-daemon-cnv6f\" (UID: \"a3381953-6131-477f-be6d-92ef7dc333a7\") " pod="openshift-multus/network-metrics-daemon-cnv6f" Jul 08 13:23:12.734379 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.734047 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/029e8f8b-d068-4215-882f-0c72f71d5244-host\") pod \"tuned-4wjn6\" (UID: \"029e8f8b-d068-4215-882f-0c72f71d5244\") " pod="openshift-cluster-node-tuning-operator/tuned-4wjn6" Jul 08 13:23:12.734379 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.734111 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/029e8f8b-d068-4215-882f-0c72f71d5244-host\") pod \"tuned-4wjn6\" (UID: \"029e8f8b-d068-4215-882f-0c72f71d5244\") " pod="openshift-cluster-node-tuning-operator/tuned-4wjn6" Jul 08 13:23:12.734379 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.734121 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/db3380e5-69cf-4c03-b995-6de00f6b269f-etc-kubernetes\") pod \"multus-85xxk\" (UID: \"db3380e5-69cf-4c03-b995-6de00f6b269f\") " pod="openshift-multus/multus-85xxk" Jul 08 13:23:12.734379 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:12.734153 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 13:23:12.734379 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.734154 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d492fe73-0e0b-4b21-8ea1-2ab7cfcefe74-tmp-dir\") pod \"node-resolver-zs2fw\" (UID: \"d492fe73-0e0b-4b21-8ea1-2ab7cfcefe74\") " pod="openshift-dns/node-resolver-zs2fw" Jul 08 13:23:12.734379 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.734199 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/029e8f8b-d068-4215-882f-0c72f71d5244-etc-sysctl-conf\") pod \"tuned-4wjn6\" (UID: \"029e8f8b-d068-4215-882f-0c72f71d5244\") " pod="openshift-cluster-node-tuning-operator/tuned-4wjn6" Jul 08 13:23:12.734379 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.734203 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7tkp\" (UniqueName: \"kubernetes.io/projected/d492fe73-0e0b-4b21-8ea1-2ab7cfcefe74-kube-api-access-x7tkp\") pod \"node-resolver-zs2fw\" (UID: \"d492fe73-0e0b-4b21-8ea1-2ab7cfcefe74\") " pod="openshift-dns/node-resolver-zs2fw" Jul 08 13:23:12.734379 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:12.734260 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3381953-6131-477f-be6d-92ef7dc333a7-metrics-certs podName:a3381953-6131-477f-be6d-92ef7dc333a7 nodeName:}" failed. No retries permitted until 2026-07-08 13:23:13.234217151 +0000 UTC m=+2.031157789 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a3381953-6131-477f-be6d-92ef7dc333a7-metrics-certs") pod "network-metrics-daemon-cnv6f" (UID: "a3381953-6131-477f-be6d-92ef7dc333a7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 13:23:12.734379 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.734306 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d41d04d6-dc32-43f2-8e7c-24c36daed99f-ovnkube-script-lib\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.734379 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.734318 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ea6986a3-b0b4-4704-a47a-c991094c04f4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qmwvz\" (UID: \"ea6986a3-b0b4-4704-a47a-c991094c04f4\") " pod="openshift-multus/multus-additional-cni-plugins-qmwvz" Jul 08 13:23:12.734379 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.734341 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4b747c9c-61b0-41b9-ac34-7c8d1e63e6df-registration-dir\") pod \"aws-ebs-csi-driver-node-rpkn4\" (UID: \"4b747c9c-61b0-41b9-ac34-7c8d1e63e6df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpkn4" Jul 08 13:23:12.734379 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.734376 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6d99\" (UniqueName: \"kubernetes.io/projected/67e86263-6fde-4f3a-a775-9e29eb8cacef-kube-api-access-v6d99\") pod \"node-ca-pn22q\" (UID: \"67e86263-6fde-4f3a-a775-9e29eb8cacef\") " pod="openshift-image-registry/node-ca-pn22q" Jul 08 13:23:12.735049 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.734392 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4b747c9c-61b0-41b9-ac34-7c8d1e63e6df-registration-dir\") pod \"aws-ebs-csi-driver-node-rpkn4\" (UID: \"4b747c9c-61b0-41b9-ac34-7c8d1e63e6df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpkn4" Jul 08 13:23:12.735049 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.734419 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7bcaa09d-1aac-4ac4-af53-f79bfc73b690-konnectivity-ca\") pod \"konnectivity-agent-g4lnf\" (UID: \"7bcaa09d-1aac-4ac4-af53-f79bfc73b690\") " pod="kube-system/konnectivity-agent-g4lnf" Jul 08 13:23:12.735049 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.734445 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d41d04d6-dc32-43f2-8e7c-24c36daed99f-host-slash\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.735049 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.734468 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d41d04d6-dc32-43f2-8e7c-24c36daed99f-log-socket\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.735049 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.734493 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d41d04d6-dc32-43f2-8e7c-24c36daed99f-env-overrides\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.735049 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.734525 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/db3380e5-69cf-4c03-b995-6de00f6b269f-host-run-multus-certs\") pod \"multus-85xxk\" (UID: \"db3380e5-69cf-4c03-b995-6de00f6b269f\") " pod="openshift-multus/multus-85xxk" Jul 08 13:23:12.735049 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.734553 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4b747c9c-61b0-41b9-ac34-7c8d1e63e6df-device-dir\") pod \"aws-ebs-csi-driver-node-rpkn4\" (UID: \"4b747c9c-61b0-41b9-ac34-7c8d1e63e6df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpkn4" Jul 08 13:23:12.735049 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.734595 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/db3380e5-69cf-4c03-b995-6de00f6b269f-system-cni-dir\") pod \"multus-85xxk\" (UID: \"db3380e5-69cf-4c03-b995-6de00f6b269f\") " pod="openshift-multus/multus-85xxk" Jul 08 13:23:12.735049 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.734634 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/db3380e5-69cf-4c03-b995-6de00f6b269f-host-var-lib-cni-bin\") pod \"multus-85xxk\" (UID: \"db3380e5-69cf-4c03-b995-6de00f6b269f\") " pod="openshift-multus/multus-85xxk" Jul 08 13:23:12.735049 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.734660 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4b747c9c-61b0-41b9-ac34-7c8d1e63e6df-device-dir\") pod \"aws-ebs-csi-driver-node-rpkn4\" (UID: \"4b747c9c-61b0-41b9-ac34-7c8d1e63e6df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpkn4" Jul 08 13:23:12.735049 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.734694 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d41d04d6-dc32-43f2-8e7c-24c36daed99f-systemd-units\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.735049 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.734741 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d41d04d6-dc32-43f2-8e7c-24c36daed99f-host-run-ovn-kubernetes\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.735049 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.734804 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/029e8f8b-d068-4215-882f-0c72f71d5244-etc-kubernetes\") pod \"tuned-4wjn6\" (UID: \"029e8f8b-d068-4215-882f-0c72f71d5244\") " pod="openshift-cluster-node-tuning-operator/tuned-4wjn6" Jul 08 13:23:12.735049 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.734847 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/029e8f8b-d068-4215-882f-0c72f71d5244-etc-kubernetes\") pod \"tuned-4wjn6\" (UID: \"029e8f8b-d068-4215-882f-0c72f71d5244\") " pod="openshift-cluster-node-tuning-operator/tuned-4wjn6" Jul 08 13:23:12.735049 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.734867 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ea6986a3-b0b4-4704-a47a-c991094c04f4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qmwvz\" (UID: \"ea6986a3-b0b4-4704-a47a-c991094c04f4\") " pod="openshift-multus/multus-additional-cni-plugins-qmwvz" Jul 08 13:23:12.735049 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.734962 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbp6t\" (UniqueName: \"kubernetes.io/projected/ea6986a3-b0b4-4704-a47a-c991094c04f4-kube-api-access-rbp6t\") pod \"multus-additional-cni-plugins-qmwvz\" (UID: \"ea6986a3-b0b4-4704-a47a-c991094c04f4\") " pod="openshift-multus/multus-additional-cni-plugins-qmwvz" Jul 08 13:23:12.736178 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.734994 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbv4p\" (UniqueName: \"kubernetes.io/projected/4b747c9c-61b0-41b9-ac34-7c8d1e63e6df-kube-api-access-kbv4p\") pod \"aws-ebs-csi-driver-node-rpkn4\" (UID: \"4b747c9c-61b0-41b9-ac34-7c8d1e63e6df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpkn4" Jul 08 13:23:12.736178 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.734985 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ea6986a3-b0b4-4704-a47a-c991094c04f4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qmwvz\" (UID: \"ea6986a3-b0b4-4704-a47a-c991094c04f4\") " pod="openshift-multus/multus-additional-cni-plugins-qmwvz" Jul 08 13:23:12.736178 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.735023 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/029e8f8b-d068-4215-882f-0c72f71d5244-lib-modules\") pod \"tuned-4wjn6\" (UID: \"029e8f8b-d068-4215-882f-0c72f71d5244\") " pod="openshift-cluster-node-tuning-operator/tuned-4wjn6" Jul 08 13:23:12.736178 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.735051 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/67e86263-6fde-4f3a-a775-9e29eb8cacef-serviceca\") pod \"node-ca-pn22q\" (UID: \"67e86263-6fde-4f3a-a775-9e29eb8cacef\") " pod="openshift-image-registry/node-ca-pn22q" Jul 08 13:23:12.736178 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.735078 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d41d04d6-dc32-43f2-8e7c-24c36daed99f-host-run-netns\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.736178 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.735140 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdst2\" (UniqueName: \"kubernetes.io/projected/d41d04d6-dc32-43f2-8e7c-24c36daed99f-kube-api-access-fdst2\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.736178 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.735164 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/db3380e5-69cf-4c03-b995-6de00f6b269f-hostroot\") pod \"multus-85xxk\" (UID: \"db3380e5-69cf-4c03-b995-6de00f6b269f\") " pod="openshift-multus/multus-85xxk" Jul 08 13:23:12.736178 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.735189 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/029e8f8b-d068-4215-882f-0c72f71d5244-run\") pod \"tuned-4wjn6\" (UID: \"029e8f8b-d068-4215-882f-0c72f71d5244\") " pod="openshift-cluster-node-tuning-operator/tuned-4wjn6" Jul 08 13:23:12.736178 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.735186 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/029e8f8b-d068-4215-882f-0c72f71d5244-lib-modules\") pod \"tuned-4wjn6\" (UID: \"029e8f8b-d068-4215-882f-0c72f71d5244\") " pod="openshift-cluster-node-tuning-operator/tuned-4wjn6" Jul 08 13:23:12.736178 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.735212 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4b747c9c-61b0-41b9-ac34-7c8d1e63e6df-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rpkn4\" (UID: \"4b747c9c-61b0-41b9-ac34-7c8d1e63e6df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpkn4" Jul 08 13:23:12.736178 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.735237 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/36ed19b3-ce4a-466e-8d8f-e6ef13588a83-host-slash\") pod \"iptables-alerter-zfsbx\" (UID: \"36ed19b3-ce4a-466e-8d8f-e6ef13588a83\") " pod="openshift-network-operator/iptables-alerter-zfsbx" Jul 08 13:23:12.736178 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.735263 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d41d04d6-dc32-43f2-8e7c-24c36daed99f-var-lib-openvswitch\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.736178 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.735287 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/029e8f8b-d068-4215-882f-0c72f71d5244-sys\") pod \"tuned-4wjn6\" (UID: \"029e8f8b-d068-4215-882f-0c72f71d5244\") " pod="openshift-cluster-node-tuning-operator/tuned-4wjn6" Jul 08 13:23:12.736178 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.735299 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4b747c9c-61b0-41b9-ac34-7c8d1e63e6df-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rpkn4\" (UID: \"4b747c9c-61b0-41b9-ac34-7c8d1e63e6df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpkn4" Jul 08 13:23:12.736178 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.735311 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/db3380e5-69cf-4c03-b995-6de00f6b269f-multus-socket-dir-parent\") pod \"multus-85xxk\" (UID: \"db3380e5-69cf-4c03-b995-6de00f6b269f\") " pod="openshift-multus/multus-85xxk" Jul 08 13:23:12.736178 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.735339 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/db3380e5-69cf-4c03-b995-6de00f6b269f-host-var-lib-cni-multus\") pod \"multus-85xxk\" (UID: \"db3380e5-69cf-4c03-b995-6de00f6b269f\") " pod="openshift-multus/multus-85xxk" Jul 08 13:23:12.736178 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.735338 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/029e8f8b-d068-4215-882f-0c72f71d5244-run\") pod \"tuned-4wjn6\" (UID: \"029e8f8b-d068-4215-882f-0c72f71d5244\") " pod="openshift-cluster-node-tuning-operator/tuned-4wjn6" Jul 08 13:23:12.737047 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.735362 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d41d04d6-dc32-43f2-8e7c-24c36daed99f-node-log\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.737047 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.735390 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/029e8f8b-d068-4215-882f-0c72f71d5244-etc-systemd\") pod \"tuned-4wjn6\" (UID: \"029e8f8b-d068-4215-882f-0c72f71d5244\") " pod="openshift-cluster-node-tuning-operator/tuned-4wjn6" Jul 08 13:23:12.737047 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.735392 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/029e8f8b-d068-4215-882f-0c72f71d5244-sys\") pod \"tuned-4wjn6\" (UID: \"029e8f8b-d068-4215-882f-0c72f71d5244\") " pod="openshift-cluster-node-tuning-operator/tuned-4wjn6" Jul 08 13:23:12.737047 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.735416 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d41d04d6-dc32-43f2-8e7c-24c36daed99f-run-systemd\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.737047 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.735446 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d41d04d6-dc32-43f2-8e7c-24c36daed99f-etc-openvswitch\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.737047 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.735472 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ea6986a3-b0b4-4704-a47a-c991094c04f4-os-release\") pod \"multus-additional-cni-plugins-qmwvz\" (UID: \"ea6986a3-b0b4-4704-a47a-c991094c04f4\") " pod="openshift-multus/multus-additional-cni-plugins-qmwvz" Jul 08 13:23:12.737047 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.735500 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ea6986a3-b0b4-4704-a47a-c991094c04f4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qmwvz\" (UID: \"ea6986a3-b0b4-4704-a47a-c991094c04f4\") " pod="openshift-multus/multus-additional-cni-plugins-qmwvz" Jul 08 13:23:12.737047 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.735525 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4b747c9c-61b0-41b9-ac34-7c8d1e63e6df-sys-fs\") pod \"aws-ebs-csi-driver-node-rpkn4\" (UID: \"4b747c9c-61b0-41b9-ac34-7c8d1e63e6df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpkn4" Jul 08 13:23:12.737047 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.735554 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/029e8f8b-d068-4215-882f-0c72f71d5244-etc-tuned\") pod \"tuned-4wjn6\" (UID: \"029e8f8b-d068-4215-882f-0c72f71d5244\") " pod="openshift-cluster-node-tuning-operator/tuned-4wjn6" Jul 08 13:23:12.737047 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.735576 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ea6986a3-b0b4-4704-a47a-c991094c04f4-cni-binary-copy\") pod \"multus-additional-cni-plugins-qmwvz\" (UID: \"ea6986a3-b0b4-4704-a47a-c991094c04f4\") " pod="openshift-multus/multus-additional-cni-plugins-qmwvz" Jul 08 13:23:12.737047 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.735601 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/029e8f8b-d068-4215-882f-0c72f71d5244-etc-sysconfig\") pod \"tuned-4wjn6\" (UID: \"029e8f8b-d068-4215-882f-0c72f71d5244\") " pod="openshift-cluster-node-tuning-operator/tuned-4wjn6" Jul 08 13:23:12.737047 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.735624 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fx5fs\" (UniqueName: \"kubernetes.io/projected/029e8f8b-d068-4215-882f-0c72f71d5244-kube-api-access-fx5fs\") pod \"tuned-4wjn6\" (UID: \"029e8f8b-d068-4215-882f-0c72f71d5244\") " pod="openshift-cluster-node-tuning-operator/tuned-4wjn6" Jul 08 13:23:12.737047 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.735641 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4b747c9c-61b0-41b9-ac34-7c8d1e63e6df-sys-fs\") pod \"aws-ebs-csi-driver-node-rpkn4\" (UID: \"4b747c9c-61b0-41b9-ac34-7c8d1e63e6df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpkn4" Jul 08 13:23:12.737047 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.735710 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ea6986a3-b0b4-4704-a47a-c991094c04f4-os-release\") pod \"multus-additional-cni-plugins-qmwvz\" (UID: \"ea6986a3-b0b4-4704-a47a-c991094c04f4\") " pod="openshift-multus/multus-additional-cni-plugins-qmwvz" Jul 08 13:23:12.737047 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.735747 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/029e8f8b-d068-4215-882f-0c72f71d5244-etc-systemd\") pod \"tuned-4wjn6\" (UID: \"029e8f8b-d068-4215-882f-0c72f71d5244\") " pod="openshift-cluster-node-tuning-operator/tuned-4wjn6" Jul 08 13:23:12.737047 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.735821 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/029e8f8b-d068-4215-882f-0c72f71d5244-etc-sysconfig\") pod \"tuned-4wjn6\" (UID: \"029e8f8b-d068-4215-882f-0c72f71d5244\") " pod="openshift-cluster-node-tuning-operator/tuned-4wjn6" Jul 08 13:23:12.737047 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.735648 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49f9w\" (UniqueName: \"kubernetes.io/projected/db3380e5-69cf-4c03-b995-6de00f6b269f-kube-api-access-49f9w\") pod \"multus-85xxk\" (UID: \"db3380e5-69cf-4c03-b995-6de00f6b269f\") " pod="openshift-multus/multus-85xxk" Jul 08 13:23:12.737762 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.735898 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d41d04d6-dc32-43f2-8e7c-24c36daed99f-run-ovn\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.737762 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.735924 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d41d04d6-dc32-43f2-8e7c-24c36daed99f-host-cni-bin\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.737762 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.735953 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhdl8\" (UniqueName: \"kubernetes.io/projected/a3381953-6131-477f-be6d-92ef7dc333a7-kube-api-access-nhdl8\") pod \"network-metrics-daemon-cnv6f\" (UID: \"a3381953-6131-477f-be6d-92ef7dc333a7\") " pod="openshift-multus/network-metrics-daemon-cnv6f" Jul 08 13:23:12.737762 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.735977 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/029e8f8b-d068-4215-882f-0c72f71d5244-tmp\") pod \"tuned-4wjn6\" (UID: \"029e8f8b-d068-4215-882f-0c72f71d5244\") " pod="openshift-cluster-node-tuning-operator/tuned-4wjn6" Jul 08 13:23:12.737762 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.736006 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d41d04d6-dc32-43f2-8e7c-24c36daed99f-run-openvswitch\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.737762 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.736033 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f93cae56-ca8c-4c37-866c-51c79f237448-original-pull-secret\") pod \"global-pull-secret-syncer-tp2lh\" (UID: \"f93cae56-ca8c-4c37-866c-51c79f237448\") " pod="kube-system/global-pull-secret-syncer-tp2lh" Jul 08 13:23:12.737762 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.736057 2567 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 08 13:23:12.737762 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.736201 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/029e8f8b-d068-4215-882f-0c72f71d5244-etc-sysctl-d\") pod \"tuned-4wjn6\" (UID: \"029e8f8b-d068-4215-882f-0c72f71d5244\") " pod="openshift-cluster-node-tuning-operator/tuned-4wjn6" Jul 08 13:23:12.737762 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.736220 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ea6986a3-b0b4-4704-a47a-c991094c04f4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qmwvz\" (UID: \"ea6986a3-b0b4-4704-a47a-c991094c04f4\") " pod="openshift-multus/multus-additional-cni-plugins-qmwvz" Jul 08 13:23:12.737762 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.736061 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/029e8f8b-d068-4215-882f-0c72f71d5244-etc-sysctl-d\") pod \"tuned-4wjn6\" (UID: \"029e8f8b-d068-4215-882f-0c72f71d5244\") " pod="openshift-cluster-node-tuning-operator/tuned-4wjn6" Jul 08 13:23:12.737762 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.736560 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/db3380e5-69cf-4c03-b995-6de00f6b269f-cni-binary-copy\") pod \"multus-85xxk\" (UID: \"db3380e5-69cf-4c03-b995-6de00f6b269f\") " pod="openshift-multus/multus-85xxk" Jul 08 13:23:12.737762 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.736583 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/db3380e5-69cf-4c03-b995-6de00f6b269f-host-run-netns\") pod \"multus-85xxk\" (UID: \"db3380e5-69cf-4c03-b995-6de00f6b269f\") " pod="openshift-multus/multus-85xxk" Jul 08 13:23:12.737762 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.736655 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/db3380e5-69cf-4c03-b995-6de00f6b269f-multus-conf-dir\") pod \"multus-85xxk\" (UID: \"db3380e5-69cf-4c03-b995-6de00f6b269f\") " pod="openshift-multus/multus-85xxk" Jul 08 13:23:12.737762 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.736677 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ea6986a3-b0b4-4704-a47a-c991094c04f4-cni-binary-copy\") pod \"multus-additional-cni-plugins-qmwvz\" (UID: \"ea6986a3-b0b4-4704-a47a-c991094c04f4\") " pod="openshift-multus/multus-additional-cni-plugins-qmwvz" Jul 08 13:23:12.737762 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.736683 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d41d04d6-dc32-43f2-8e7c-24c36daed99f-host-cni-netd\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.737762 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.736734 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f93cae56-ca8c-4c37-866c-51c79f237448-dbus\") pod \"global-pull-secret-syncer-tp2lh\" (UID: \"f93cae56-ca8c-4c37-866c-51c79f237448\") " pod="kube-system/global-pull-secret-syncer-tp2lh" Jul 08 13:23:12.737762 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.736761 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvwtn\" (UniqueName: \"kubernetes.io/projected/36ed19b3-ce4a-466e-8d8f-e6ef13588a83-kube-api-access-rvwtn\") pod \"iptables-alerter-zfsbx\" (UID: \"36ed19b3-ce4a-466e-8d8f-e6ef13588a83\") " pod="openshift-network-operator/iptables-alerter-zfsbx" Jul 08 13:23:12.738359 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.736789 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4b747c9c-61b0-41b9-ac34-7c8d1e63e6df-socket-dir\") pod \"aws-ebs-csi-driver-node-rpkn4\" (UID: \"4b747c9c-61b0-41b9-ac34-7c8d1e63e6df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpkn4" Jul 08 13:23:12.738359 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.736814 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/029e8f8b-d068-4215-882f-0c72f71d5244-etc-modprobe-d\") pod \"tuned-4wjn6\" (UID: \"029e8f8b-d068-4215-882f-0c72f71d5244\") " pod="openshift-cluster-node-tuning-operator/tuned-4wjn6" Jul 08 13:23:12.738359 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.736845 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f93cae56-ca8c-4c37-866c-51c79f237448-kubelet-config\") pod \"global-pull-secret-syncer-tp2lh\" (UID: \"f93cae56-ca8c-4c37-866c-51c79f237448\") " pod="kube-system/global-pull-secret-syncer-tp2lh" Jul 08 13:23:12.738359 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.736912 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/029e8f8b-d068-4215-882f-0c72f71d5244-etc-modprobe-d\") pod \"tuned-4wjn6\" (UID: \"029e8f8b-d068-4215-882f-0c72f71d5244\") " pod="openshift-cluster-node-tuning-operator/tuned-4wjn6" Jul 08 13:23:12.738359 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.736910 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/db3380e5-69cf-4c03-b995-6de00f6b269f-cnibin\") pod \"multus-85xxk\" (UID: \"db3380e5-69cf-4c03-b995-6de00f6b269f\") " pod="openshift-multus/multus-85xxk" Jul 08 13:23:12.738359 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.736974 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4b747c9c-61b0-41b9-ac34-7c8d1e63e6df-socket-dir\") pod \"aws-ebs-csi-driver-node-rpkn4\" (UID: \"4b747c9c-61b0-41b9-ac34-7c8d1e63e6df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpkn4" Jul 08 13:23:12.738359 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.737013 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ea6986a3-b0b4-4704-a47a-c991094c04f4-system-cni-dir\") pod \"multus-additional-cni-plugins-qmwvz\" (UID: \"ea6986a3-b0b4-4704-a47a-c991094c04f4\") " pod="openshift-multus/multus-additional-cni-plugins-qmwvz" Jul 08 13:23:12.738359 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.737047 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ea6986a3-b0b4-4704-a47a-c991094c04f4-cnibin\") pod \"multus-additional-cni-plugins-qmwvz\" (UID: \"ea6986a3-b0b4-4704-a47a-c991094c04f4\") " pod="openshift-multus/multus-additional-cni-plugins-qmwvz" Jul 08 13:23:12.738359 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.737074 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4b747c9c-61b0-41b9-ac34-7c8d1e63e6df-etc-selinux\") pod \"aws-ebs-csi-driver-node-rpkn4\" (UID: \"4b747c9c-61b0-41b9-ac34-7c8d1e63e6df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpkn4" Jul 08 13:23:12.738359 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.737119 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/36ed19b3-ce4a-466e-8d8f-e6ef13588a83-iptables-alerter-script\") pod \"iptables-alerter-zfsbx\" (UID: \"36ed19b3-ce4a-466e-8d8f-e6ef13588a83\") " pod="openshift-network-operator/iptables-alerter-zfsbx" Jul 08 13:23:12.738359 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.737161 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7bcaa09d-1aac-4ac4-af53-f79bfc73b690-agent-certs\") pod \"konnectivity-agent-g4lnf\" (UID: \"7bcaa09d-1aac-4ac4-af53-f79bfc73b690\") " pod="kube-system/konnectivity-agent-g4lnf" Jul 08 13:23:12.738359 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.737188 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/db3380e5-69cf-4c03-b995-6de00f6b269f-multus-cni-dir\") pod \"multus-85xxk\" (UID: \"db3380e5-69cf-4c03-b995-6de00f6b269f\") " pod="openshift-multus/multus-85xxk" Jul 08 13:23:12.738359 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.737213 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/db3380e5-69cf-4c03-b995-6de00f6b269f-os-release\") pod \"multus-85xxk\" (UID: \"db3380e5-69cf-4c03-b995-6de00f6b269f\") " pod="openshift-multus/multus-85xxk" Jul 08 13:23:12.738359 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.737246 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/db3380e5-69cf-4c03-b995-6de00f6b269f-multus-daemon-config\") pod \"multus-85xxk\" (UID: \"db3380e5-69cf-4c03-b995-6de00f6b269f\") " pod="openshift-multus/multus-85xxk" Jul 08 13:23:12.738359 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.737348 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ea6986a3-b0b4-4704-a47a-c991094c04f4-system-cni-dir\") pod \"multus-additional-cni-plugins-qmwvz\" (UID: \"ea6986a3-b0b4-4704-a47a-c991094c04f4\") " pod="openshift-multus/multus-additional-cni-plugins-qmwvz" Jul 08 13:23:12.738359 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.737355 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ea6986a3-b0b4-4704-a47a-c991094c04f4-cnibin\") pod \"multus-additional-cni-plugins-qmwvz\" (UID: \"ea6986a3-b0b4-4704-a47a-c991094c04f4\") " pod="openshift-multus/multus-additional-cni-plugins-qmwvz" Jul 08 13:23:12.738359 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.737454 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4b747c9c-61b0-41b9-ac34-7c8d1e63e6df-etc-selinux\") pod \"aws-ebs-csi-driver-node-rpkn4\" (UID: \"4b747c9c-61b0-41b9-ac34-7c8d1e63e6df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpkn4" Jul 08 13:23:12.739969 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.739941 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/029e8f8b-d068-4215-882f-0c72f71d5244-tmp\") pod \"tuned-4wjn6\" (UID: \"029e8f8b-d068-4215-882f-0c72f71d5244\") " pod="openshift-cluster-node-tuning-operator/tuned-4wjn6" Jul 08 13:23:12.740066 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.739972 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/029e8f8b-d068-4215-882f-0c72f71d5244-etc-tuned\") pod \"tuned-4wjn6\" (UID: \"029e8f8b-d068-4215-882f-0c72f71d5244\") " pod="openshift-cluster-node-tuning-operator/tuned-4wjn6" Jul 08 13:23:12.745448 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.745418 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fx5fs\" (UniqueName: \"kubernetes.io/projected/029e8f8b-d068-4215-882f-0c72f71d5244-kube-api-access-fx5fs\") pod \"tuned-4wjn6\" (UID: \"029e8f8b-d068-4215-882f-0c72f71d5244\") " pod="openshift-cluster-node-tuning-operator/tuned-4wjn6" Jul 08 13:23:12.745562 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.745470 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhdl8\" (UniqueName: \"kubernetes.io/projected/a3381953-6131-477f-be6d-92ef7dc333a7-kube-api-access-nhdl8\") pod \"network-metrics-daemon-cnv6f\" (UID: \"a3381953-6131-477f-be6d-92ef7dc333a7\") " pod="openshift-multus/network-metrics-daemon-cnv6f" Jul 08 13:23:12.745562 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.745471 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbp6t\" (UniqueName: \"kubernetes.io/projected/ea6986a3-b0b4-4704-a47a-c991094c04f4-kube-api-access-rbp6t\") pod \"multus-additional-cni-plugins-qmwvz\" (UID: \"ea6986a3-b0b4-4704-a47a-c991094c04f4\") " pod="openshift-multus/multus-additional-cni-plugins-qmwvz" Jul 08 13:23:12.745867 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.745845 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbv4p\" (UniqueName: \"kubernetes.io/projected/4b747c9c-61b0-41b9-ac34-7c8d1e63e6df-kube-api-access-kbv4p\") pod \"aws-ebs-csi-driver-node-rpkn4\" (UID: \"4b747c9c-61b0-41b9-ac34-7c8d1e63e6df\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpkn4" Jul 08 13:23:12.821013 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.820980 2567 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 13:23:12.837667 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.837636 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49f9w\" (UniqueName: \"kubernetes.io/projected/db3380e5-69cf-4c03-b995-6de00f6b269f-kube-api-access-49f9w\") pod \"multus-85xxk\" (UID: \"db3380e5-69cf-4c03-b995-6de00f6b269f\") " pod="openshift-multus/multus-85xxk" Jul 08 13:23:12.837762 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.837683 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d41d04d6-dc32-43f2-8e7c-24c36daed99f-run-ovn\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.837762 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.837713 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d41d04d6-dc32-43f2-8e7c-24c36daed99f-host-cni-bin\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.837762 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.837738 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d41d04d6-dc32-43f2-8e7c-24c36daed99f-run-openvswitch\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.837947 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.837786 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d41d04d6-dc32-43f2-8e7c-24c36daed99f-run-ovn\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.837947 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.837793 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f93cae56-ca8c-4c37-866c-51c79f237448-original-pull-secret\") pod \"global-pull-secret-syncer-tp2lh\" (UID: \"f93cae56-ca8c-4c37-866c-51c79f237448\") " pod="kube-system/global-pull-secret-syncer-tp2lh" Jul 08 13:23:12.837947 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.837822 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/db3380e5-69cf-4c03-b995-6de00f6b269f-cni-binary-copy\") pod \"multus-85xxk\" (UID: \"db3380e5-69cf-4c03-b995-6de00f6b269f\") " pod="openshift-multus/multus-85xxk" Jul 08 13:23:12.837947 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.837844 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d41d04d6-dc32-43f2-8e7c-24c36daed99f-run-openvswitch\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.837947 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.837888 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/db3380e5-69cf-4c03-b995-6de00f6b269f-host-run-netns\") pod \"multus-85xxk\" (UID: \"db3380e5-69cf-4c03-b995-6de00f6b269f\") " pod="openshift-multus/multus-85xxk" Jul 08 13:23:12.837947 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.837939 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d41d04d6-dc32-43f2-8e7c-24c36daed99f-host-cni-bin\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.837947 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.837847 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/db3380e5-69cf-4c03-b995-6de00f6b269f-host-run-netns\") pod \"multus-85xxk\" (UID: \"db3380e5-69cf-4c03-b995-6de00f6b269f\") " pod="openshift-multus/multus-85xxk" Jul 08 13:23:12.837947 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:12.837941 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 13:23:12.838389 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.837977 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/db3380e5-69cf-4c03-b995-6de00f6b269f-multus-conf-dir\") pod \"multus-85xxk\" (UID: \"db3380e5-69cf-4c03-b995-6de00f6b269f\") " pod="openshift-multus/multus-85xxk" Jul 08 13:23:12.838389 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.838005 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d41d04d6-dc32-43f2-8e7c-24c36daed99f-host-cni-netd\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.838389 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:12.838033 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f93cae56-ca8c-4c37-866c-51c79f237448-original-pull-secret podName:f93cae56-ca8c-4c37-866c-51c79f237448 nodeName:}" failed. No retries permitted until 2026-07-08 13:23:13.338010254 +0000 UTC m=+2.134950907 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f93cae56-ca8c-4c37-866c-51c79f237448-original-pull-secret") pod "global-pull-secret-syncer-tp2lh" (UID: "f93cae56-ca8c-4c37-866c-51c79f237448") : object "kube-system"/"original-pull-secret" not registered Jul 08 13:23:12.838389 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.838052 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d41d04d6-dc32-43f2-8e7c-24c36daed99f-host-cni-netd\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.838389 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.838065 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f93cae56-ca8c-4c37-866c-51c79f237448-dbus\") pod \"global-pull-secret-syncer-tp2lh\" (UID: \"f93cae56-ca8c-4c37-866c-51c79f237448\") " pod="kube-system/global-pull-secret-syncer-tp2lh" Jul 08 13:23:12.838389 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.838072 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/db3380e5-69cf-4c03-b995-6de00f6b269f-multus-conf-dir\") pod \"multus-85xxk\" (UID: \"db3380e5-69cf-4c03-b995-6de00f6b269f\") " pod="openshift-multus/multus-85xxk" Jul 08 13:23:12.838389 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.838094 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvwtn\" (UniqueName: \"kubernetes.io/projected/36ed19b3-ce4a-466e-8d8f-e6ef13588a83-kube-api-access-rvwtn\") pod \"iptables-alerter-zfsbx\" (UID: \"36ed19b3-ce4a-466e-8d8f-e6ef13588a83\") " pod="openshift-network-operator/iptables-alerter-zfsbx" Jul 08 13:23:12.838389 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.838144 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f93cae56-ca8c-4c37-866c-51c79f237448-kubelet-config\") pod \"global-pull-secret-syncer-tp2lh\" (UID: \"f93cae56-ca8c-4c37-866c-51c79f237448\") " pod="kube-system/global-pull-secret-syncer-tp2lh" Jul 08 13:23:12.838389 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.838171 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/db3380e5-69cf-4c03-b995-6de00f6b269f-cnibin\") pod \"multus-85xxk\" (UID: \"db3380e5-69cf-4c03-b995-6de00f6b269f\") " pod="openshift-multus/multus-85xxk" Jul 08 13:23:12.838389 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.838201 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/36ed19b3-ce4a-466e-8d8f-e6ef13588a83-iptables-alerter-script\") pod \"iptables-alerter-zfsbx\" (UID: \"36ed19b3-ce4a-466e-8d8f-e6ef13588a83\") " pod="openshift-network-operator/iptables-alerter-zfsbx" Jul 08 13:23:12.838389 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.838269 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f93cae56-ca8c-4c37-866c-51c79f237448-kubelet-config\") pod \"global-pull-secret-syncer-tp2lh\" (UID: \"f93cae56-ca8c-4c37-866c-51c79f237448\") " pod="kube-system/global-pull-secret-syncer-tp2lh" Jul 08 13:23:12.838389 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.838277 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/db3380e5-69cf-4c03-b995-6de00f6b269f-cnibin\") pod \"multus-85xxk\" (UID: \"db3380e5-69cf-4c03-b995-6de00f6b269f\") " pod="openshift-multus/multus-85xxk" Jul 08 13:23:12.838389 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.838292 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f93cae56-ca8c-4c37-866c-51c79f237448-dbus\") pod \"global-pull-secret-syncer-tp2lh\" (UID: \"f93cae56-ca8c-4c37-866c-51c79f237448\") " pod="kube-system/global-pull-secret-syncer-tp2lh" Jul 08 13:23:12.838389 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.838328 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7bcaa09d-1aac-4ac4-af53-f79bfc73b690-agent-certs\") pod \"konnectivity-agent-g4lnf\" (UID: \"7bcaa09d-1aac-4ac4-af53-f79bfc73b690\") " pod="kube-system/konnectivity-agent-g4lnf" Jul 08 13:23:12.838389 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.838360 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/db3380e5-69cf-4c03-b995-6de00f6b269f-multus-cni-dir\") pod \"multus-85xxk\" (UID: \"db3380e5-69cf-4c03-b995-6de00f6b269f\") " pod="openshift-multus/multus-85xxk" Jul 08 13:23:12.838389 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.838385 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/db3380e5-69cf-4c03-b995-6de00f6b269f-os-release\") pod \"multus-85xxk\" (UID: \"db3380e5-69cf-4c03-b995-6de00f6b269f\") " pod="openshift-multus/multus-85xxk" Jul 08 13:23:12.839406 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.838415 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/db3380e5-69cf-4c03-b995-6de00f6b269f-multus-daemon-config\") pod \"multus-85xxk\" (UID: \"db3380e5-69cf-4c03-b995-6de00f6b269f\") " pod="openshift-multus/multus-85xxk" Jul 08 13:23:12.839406 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.838441 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/67e86263-6fde-4f3a-a775-9e29eb8cacef-host\") pod \"node-ca-pn22q\" (UID: \"67e86263-6fde-4f3a-a775-9e29eb8cacef\") " pod="openshift-image-registry/node-ca-pn22q" Jul 08 13:23:12.839406 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.838467 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d492fe73-0e0b-4b21-8ea1-2ab7cfcefe74-hosts-file\") pod \"node-resolver-zs2fw\" (UID: \"d492fe73-0e0b-4b21-8ea1-2ab7cfcefe74\") " pod="openshift-dns/node-resolver-zs2fw" Jul 08 13:23:12.839406 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.838478 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/db3380e5-69cf-4c03-b995-6de00f6b269f-multus-cni-dir\") pod \"multus-85xxk\" (UID: \"db3380e5-69cf-4c03-b995-6de00f6b269f\") " pod="openshift-multus/multus-85xxk" Jul 08 13:23:12.839406 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.838491 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d41d04d6-dc32-43f2-8e7c-24c36daed99f-host-kubelet\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.839406 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.838518 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d41d04d6-dc32-43f2-8e7c-24c36daed99f-ovnkube-config\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.839406 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.838524 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/db3380e5-69cf-4c03-b995-6de00f6b269f-cni-binary-copy\") pod \"multus-85xxk\" (UID: \"db3380e5-69cf-4c03-b995-6de00f6b269f\") " pod="openshift-multus/multus-85xxk" Jul 08 13:23:12.839406 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.838565 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/db3380e5-69cf-4c03-b995-6de00f6b269f-os-release\") pod \"multus-85xxk\" (UID: \"db3380e5-69cf-4c03-b995-6de00f6b269f\") " pod="openshift-multus/multus-85xxk" Jul 08 13:23:12.839406 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.838566 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d41d04d6-dc32-43f2-8e7c-24c36daed99f-ovn-node-metrics-cert\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.839406 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.838612 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/db3380e5-69cf-4c03-b995-6de00f6b269f-host-run-k8s-cni-cncf-io\") pod \"multus-85xxk\" (UID: \"db3380e5-69cf-4c03-b995-6de00f6b269f\") " pod="openshift-multus/multus-85xxk" Jul 08 13:23:12.839406 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.838615 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d41d04d6-dc32-43f2-8e7c-24c36daed99f-host-kubelet\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.839406 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.838641 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/db3380e5-69cf-4c03-b995-6de00f6b269f-host-var-lib-kubelet\") pod \"multus-85xxk\" (UID: \"db3380e5-69cf-4c03-b995-6de00f6b269f\") " pod="openshift-multus/multus-85xxk" Jul 08 13:23:12.839406 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.838677 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/67e86263-6fde-4f3a-a775-9e29eb8cacef-host\") pod \"node-ca-pn22q\" (UID: \"67e86263-6fde-4f3a-a775-9e29eb8cacef\") " pod="openshift-image-registry/node-ca-pn22q" Jul 08 13:23:12.839406 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.838675 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d41d04d6-dc32-43f2-8e7c-24c36daed99f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.839406 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.838716 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d41d04d6-dc32-43f2-8e7c-24c36daed99f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.839406 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.838717 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wm2f\" (UniqueName: \"kubernetes.io/projected/b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5-kube-api-access-6wm2f\") pod \"network-check-target-rm9f4\" (UID: \"b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5\") " pod="openshift-network-diagnostics/network-check-target-rm9f4" Jul 08 13:23:12.839406 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.838771 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/db3380e5-69cf-4c03-b995-6de00f6b269f-etc-kubernetes\") pod \"multus-85xxk\" (UID: \"db3380e5-69cf-4c03-b995-6de00f6b269f\") " pod="openshift-multus/multus-85xxk" Jul 08 13:23:12.839406 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.838793 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d492fe73-0e0b-4b21-8ea1-2ab7cfcefe74-tmp-dir\") pod \"node-resolver-zs2fw\" (UID: \"d492fe73-0e0b-4b21-8ea1-2ab7cfcefe74\") " pod="openshift-dns/node-resolver-zs2fw" Jul 08 13:23:12.840332 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.838817 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7tkp\" (UniqueName: \"kubernetes.io/projected/d492fe73-0e0b-4b21-8ea1-2ab7cfcefe74-kube-api-access-x7tkp\") pod \"node-resolver-zs2fw\" (UID: \"d492fe73-0e0b-4b21-8ea1-2ab7cfcefe74\") " pod="openshift-dns/node-resolver-zs2fw" Jul 08 13:23:12.840332 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.838844 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d41d04d6-dc32-43f2-8e7c-24c36daed99f-ovnkube-script-lib\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.840332 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.838876 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6d99\" (UniqueName: \"kubernetes.io/projected/67e86263-6fde-4f3a-a775-9e29eb8cacef-kube-api-access-v6d99\") pod \"node-ca-pn22q\" (UID: \"67e86263-6fde-4f3a-a775-9e29eb8cacef\") " pod="openshift-image-registry/node-ca-pn22q" Jul 08 13:23:12.840332 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.838904 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7bcaa09d-1aac-4ac4-af53-f79bfc73b690-konnectivity-ca\") pod \"konnectivity-agent-g4lnf\" (UID: \"7bcaa09d-1aac-4ac4-af53-f79bfc73b690\") " pod="kube-system/konnectivity-agent-g4lnf" Jul 08 13:23:12.840332 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.838913 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/36ed19b3-ce4a-466e-8d8f-e6ef13588a83-iptables-alerter-script\") pod \"iptables-alerter-zfsbx\" (UID: \"36ed19b3-ce4a-466e-8d8f-e6ef13588a83\") " pod="openshift-network-operator/iptables-alerter-zfsbx" Jul 08 13:23:12.840332 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.838929 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d41d04d6-dc32-43f2-8e7c-24c36daed99f-host-slash\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.840332 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.838971 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d41d04d6-dc32-43f2-8e7c-24c36daed99f-host-slash\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.840332 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.838974 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/db3380e5-69cf-4c03-b995-6de00f6b269f-etc-kubernetes\") pod \"multus-85xxk\" (UID: \"db3380e5-69cf-4c03-b995-6de00f6b269f\") " pod="openshift-multus/multus-85xxk" Jul 08 13:23:12.840332 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.838978 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d41d04d6-dc32-43f2-8e7c-24c36daed99f-log-socket\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.840332 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.839020 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d41d04d6-dc32-43f2-8e7c-24c36daed99f-env-overrides\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.840332 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.839029 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/db3380e5-69cf-4c03-b995-6de00f6b269f-host-var-lib-kubelet\") pod \"multus-85xxk\" (UID: \"db3380e5-69cf-4c03-b995-6de00f6b269f\") " pod="openshift-multus/multus-85xxk" Jul 08 13:23:12.840332 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.838969 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/db3380e5-69cf-4c03-b995-6de00f6b269f-host-run-k8s-cni-cncf-io\") pod \"multus-85xxk\" (UID: \"db3380e5-69cf-4c03-b995-6de00f6b269f\") " pod="openshift-multus/multus-85xxk" Jul 08 13:23:12.840332 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.839061 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/db3380e5-69cf-4c03-b995-6de00f6b269f-host-run-multus-certs\") pod \"multus-85xxk\" (UID: \"db3380e5-69cf-4c03-b995-6de00f6b269f\") " pod="openshift-multus/multus-85xxk" Jul 08 13:23:12.840332 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.839089 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/db3380e5-69cf-4c03-b995-6de00f6b269f-system-cni-dir\") pod \"multus-85xxk\" (UID: \"db3380e5-69cf-4c03-b995-6de00f6b269f\") " pod="openshift-multus/multus-85xxk" Jul 08 13:23:12.840332 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.839114 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/db3380e5-69cf-4c03-b995-6de00f6b269f-host-var-lib-cni-bin\") pod \"multus-85xxk\" (UID: \"db3380e5-69cf-4c03-b995-6de00f6b269f\") " pod="openshift-multus/multus-85xxk" Jul 08 13:23:12.840332 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.839178 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d41d04d6-dc32-43f2-8e7c-24c36daed99f-systemd-units\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.840332 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.839229 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d41d04d6-dc32-43f2-8e7c-24c36daed99f-host-run-ovn-kubernetes\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.840332 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.839262 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/67e86263-6fde-4f3a-a775-9e29eb8cacef-serviceca\") pod \"node-ca-pn22q\" (UID: \"67e86263-6fde-4f3a-a775-9e29eb8cacef\") " pod="openshift-image-registry/node-ca-pn22q" Jul 08 13:23:12.841046 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.839285 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d41d04d6-dc32-43f2-8e7c-24c36daed99f-host-run-netns\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.841046 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.839301 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d492fe73-0e0b-4b21-8ea1-2ab7cfcefe74-tmp-dir\") pod \"node-resolver-zs2fw\" (UID: \"d492fe73-0e0b-4b21-8ea1-2ab7cfcefe74\") " pod="openshift-dns/node-resolver-zs2fw" Jul 08 13:23:12.841046 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.839310 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdst2\" (UniqueName: \"kubernetes.io/projected/d41d04d6-dc32-43f2-8e7c-24c36daed99f-kube-api-access-fdst2\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.841046 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.839334 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/db3380e5-69cf-4c03-b995-6de00f6b269f-hostroot\") pod \"multus-85xxk\" (UID: \"db3380e5-69cf-4c03-b995-6de00f6b269f\") " pod="openshift-multus/multus-85xxk" Jul 08 13:23:12.841046 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.839361 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/36ed19b3-ce4a-466e-8d8f-e6ef13588a83-host-slash\") pod \"iptables-alerter-zfsbx\" (UID: \"36ed19b3-ce4a-466e-8d8f-e6ef13588a83\") " pod="openshift-network-operator/iptables-alerter-zfsbx" Jul 08 13:23:12.841046 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.839391 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d41d04d6-dc32-43f2-8e7c-24c36daed99f-var-lib-openvswitch\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.841046 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.838903 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d492fe73-0e0b-4b21-8ea1-2ab7cfcefe74-hosts-file\") pod \"node-resolver-zs2fw\" (UID: \"d492fe73-0e0b-4b21-8ea1-2ab7cfcefe74\") " pod="openshift-dns/node-resolver-zs2fw" Jul 08 13:23:12.841046 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.839420 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/db3380e5-69cf-4c03-b995-6de00f6b269f-multus-socket-dir-parent\") pod \"multus-85xxk\" (UID: \"db3380e5-69cf-4c03-b995-6de00f6b269f\") " pod="openshift-multus/multus-85xxk" Jul 08 13:23:12.841046 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.839446 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/db3380e5-69cf-4c03-b995-6de00f6b269f-host-var-lib-cni-multus\") pod \"multus-85xxk\" (UID: \"db3380e5-69cf-4c03-b995-6de00f6b269f\") " pod="openshift-multus/multus-85xxk" Jul 08 13:23:12.841046 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.839474 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d41d04d6-dc32-43f2-8e7c-24c36daed99f-node-log\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.841046 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.839517 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d41d04d6-dc32-43f2-8e7c-24c36daed99f-run-systemd\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.841046 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.839545 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d41d04d6-dc32-43f2-8e7c-24c36daed99f-etc-openvswitch\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.841046 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.839642 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d41d04d6-dc32-43f2-8e7c-24c36daed99f-etc-openvswitch\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.841046 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.839722 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d41d04d6-dc32-43f2-8e7c-24c36daed99f-host-run-netns\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.841046 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.839816 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d41d04d6-dc32-43f2-8e7c-24c36daed99f-ovnkube-script-lib\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.841046 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.839984 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d41d04d6-dc32-43f2-8e7c-24c36daed99f-ovnkube-config\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.841046 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.840003 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/db3380e5-69cf-4c03-b995-6de00f6b269f-hostroot\") pod \"multus-85xxk\" (UID: \"db3380e5-69cf-4c03-b995-6de00f6b269f\") " pod="openshift-multus/multus-85xxk" Jul 08 13:23:12.841046 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.840056 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/36ed19b3-ce4a-466e-8d8f-e6ef13588a83-host-slash\") pod \"iptables-alerter-zfsbx\" (UID: \"36ed19b3-ce4a-466e-8d8f-e6ef13588a83\") " pod="openshift-network-operator/iptables-alerter-zfsbx" Jul 08 13:23:12.841681 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.839446 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/db3380e5-69cf-4c03-b995-6de00f6b269f-multus-daemon-config\") pod \"multus-85xxk\" (UID: \"db3380e5-69cf-4c03-b995-6de00f6b269f\") " pod="openshift-multus/multus-85xxk" Jul 08 13:23:12.841681 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.840105 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d41d04d6-dc32-43f2-8e7c-24c36daed99f-log-socket\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.841681 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.840217 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/db3380e5-69cf-4c03-b995-6de00f6b269f-host-var-lib-cni-multus\") pod \"multus-85xxk\" (UID: \"db3380e5-69cf-4c03-b995-6de00f6b269f\") " pod="openshift-multus/multus-85xxk" Jul 08 13:23:12.841681 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.840281 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d41d04d6-dc32-43f2-8e7c-24c36daed99f-env-overrides\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.841681 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.840294 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d41d04d6-dc32-43f2-8e7c-24c36daed99f-node-log\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.841681 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.840321 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7bcaa09d-1aac-4ac4-af53-f79bfc73b690-konnectivity-ca\") pod \"konnectivity-agent-g4lnf\" (UID: \"7bcaa09d-1aac-4ac4-af53-f79bfc73b690\") " pod="kube-system/konnectivity-agent-g4lnf" Jul 08 13:23:12.841681 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.840510 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d41d04d6-dc32-43f2-8e7c-24c36daed99f-run-systemd\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.841681 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.840488 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/db3380e5-69cf-4c03-b995-6de00f6b269f-host-run-multus-certs\") pod \"multus-85xxk\" (UID: \"db3380e5-69cf-4c03-b995-6de00f6b269f\") " pod="openshift-multus/multus-85xxk" Jul 08 13:23:12.841681 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.840535 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/67e86263-6fde-4f3a-a775-9e29eb8cacef-serviceca\") pod \"node-ca-pn22q\" (UID: \"67e86263-6fde-4f3a-a775-9e29eb8cacef\") " pod="openshift-image-registry/node-ca-pn22q" Jul 08 13:23:12.841681 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.840569 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/db3380e5-69cf-4c03-b995-6de00f6b269f-system-cni-dir\") pod \"multus-85xxk\" (UID: \"db3380e5-69cf-4c03-b995-6de00f6b269f\") " pod="openshift-multus/multus-85xxk" Jul 08 13:23:12.841681 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.840628 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/db3380e5-69cf-4c03-b995-6de00f6b269f-host-var-lib-cni-bin\") pod \"multus-85xxk\" (UID: \"db3380e5-69cf-4c03-b995-6de00f6b269f\") " pod="openshift-multus/multus-85xxk" Jul 08 13:23:12.841681 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.840690 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d41d04d6-dc32-43f2-8e7c-24c36daed99f-systemd-units\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.841681 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.840735 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d41d04d6-dc32-43f2-8e7c-24c36daed99f-host-run-ovn-kubernetes\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.841681 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.840777 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d41d04d6-dc32-43f2-8e7c-24c36daed99f-var-lib-openvswitch\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.841681 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.840830 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/db3380e5-69cf-4c03-b995-6de00f6b269f-multus-socket-dir-parent\") pod \"multus-85xxk\" (UID: \"db3380e5-69cf-4c03-b995-6de00f6b269f\") " pod="openshift-multus/multus-85xxk" Jul 08 13:23:12.841681 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.841583 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d41d04d6-dc32-43f2-8e7c-24c36daed99f-ovn-node-metrics-cert\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.842263 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.842244 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7bcaa09d-1aac-4ac4-af53-f79bfc73b690-agent-certs\") pod \"konnectivity-agent-g4lnf\" (UID: \"7bcaa09d-1aac-4ac4-af53-f79bfc73b690\") " pod="kube-system/konnectivity-agent-g4lnf" Jul 08 13:23:12.845327 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:12.845306 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 13:23:12.845438 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:12.845334 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 13:23:12.845438 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:12.845357 2567 projected.go:196] Error preparing data for projected volume kube-api-access-6wm2f for pod openshift-network-diagnostics/network-check-target-rm9f4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 13:23:12.845554 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:12.845439 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5-kube-api-access-6wm2f podName:b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5 nodeName:}" failed. No retries permitted until 2026-07-08 13:23:13.345418063 +0000 UTC m=+2.142358721 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-6wm2f" (UniqueName: "kubernetes.io/projected/b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5-kube-api-access-6wm2f") pod "network-check-target-rm9f4" (UID: "b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 13:23:12.846708 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.846688 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvwtn\" (UniqueName: \"kubernetes.io/projected/36ed19b3-ce4a-466e-8d8f-e6ef13588a83-kube-api-access-rvwtn\") pod \"iptables-alerter-zfsbx\" (UID: \"36ed19b3-ce4a-466e-8d8f-e6ef13588a83\") " pod="openshift-network-operator/iptables-alerter-zfsbx" Jul 08 13:23:12.847114 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.847073 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-49f9w\" (UniqueName: \"kubernetes.io/projected/db3380e5-69cf-4c03-b995-6de00f6b269f-kube-api-access-49f9w\") pod \"multus-85xxk\" (UID: \"db3380e5-69cf-4c03-b995-6de00f6b269f\") " pod="openshift-multus/multus-85xxk" Jul 08 13:23:12.847686 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.847669 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6d99\" (UniqueName: \"kubernetes.io/projected/67e86263-6fde-4f3a-a775-9e29eb8cacef-kube-api-access-v6d99\") pod \"node-ca-pn22q\" (UID: \"67e86263-6fde-4f3a-a775-9e29eb8cacef\") " pod="openshift-image-registry/node-ca-pn22q" Jul 08 13:23:12.847905 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.847888 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7tkp\" (UniqueName: \"kubernetes.io/projected/d492fe73-0e0b-4b21-8ea1-2ab7cfcefe74-kube-api-access-x7tkp\") pod \"node-resolver-zs2fw\" (UID: \"d492fe73-0e0b-4b21-8ea1-2ab7cfcefe74\") " pod="openshift-dns/node-resolver-zs2fw" Jul 08 13:23:12.848013 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.847998 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdst2\" (UniqueName: \"kubernetes.io/projected/d41d04d6-dc32-43f2-8e7c-24c36daed99f-kube-api-access-fdst2\") pod \"ovnkube-node-sqlwq\" (UID: \"d41d04d6-dc32-43f2-8e7c-24c36daed99f\") " pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:12.852099 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:12.852049 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabd4fe04a7d63176e943bf70844ade69.slice/crio-07fc01d6c6f73a3736a9e36a921706844b19b527e8d18aed7fee36b8e622311c WatchSource:0}: Error finding container 07fc01d6c6f73a3736a9e36a921706844b19b527e8d18aed7fee36b8e622311c: Status 404 returned error can't find the container with id 07fc01d6c6f73a3736a9e36a921706844b19b527e8d18aed7fee36b8e622311c Jul 08 13:23:12.852657 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:12.852638 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7264a5fddc17c5fbd72de875bb444234.slice/crio-e68608592a494a90a168b8ffd945f79d1d37ef2fb5da1f59c8063074c8566ea9 WatchSource:0}: Error finding container e68608592a494a90a168b8ffd945f79d1d37ef2fb5da1f59c8063074c8566ea9: Status 404 returned error can't find the container with id e68608592a494a90a168b8ffd945f79d1d37ef2fb5da1f59c8063074c8566ea9 Jul 08 13:23:12.858957 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.858937 2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 13:23:12.950072 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.949989 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpkn4" Jul 08 13:23:12.960638 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:12.960610 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b747c9c_61b0_41b9_ac34_7c8d1e63e6df.slice/crio-0899188ff8db0aeb373a2c7c989e6b62a4deb1b3c017ffd165266660a8b9abd8 WatchSource:0}: Error finding container 0899188ff8db0aeb373a2c7c989e6b62a4deb1b3c017ffd165266660a8b9abd8: Status 404 returned error can't find the container with id 0899188ff8db0aeb373a2c7c989e6b62a4deb1b3c017ffd165266660a8b9abd8 Jul 08 13:23:12.968719 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.968700 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4wjn6" Jul 08 13:23:12.976937 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:12.976913 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod029e8f8b_d068_4215_882f_0c72f71d5244.slice/crio-68323fbece0f6186a881bb0d36126cf2bf7d0b49b086e6bc42a7318785773d09 WatchSource:0}: Error finding container 68323fbece0f6186a881bb0d36126cf2bf7d0b49b086e6bc42a7318785773d09: Status 404 returned error can't find the container with id 68323fbece0f6186a881bb0d36126cf2bf7d0b49b086e6bc42a7318785773d09 Jul 08 13:23:12.987414 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:12.987384 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qmwvz" Jul 08 13:23:12.996971 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:12.996948 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea6986a3_b0b4_4704_a47a_c991094c04f4.slice/crio-39c3978079da33d9e6557e25cee8db2d8afba3a99753be3193acc9a4c7041468 WatchSource:0}: Error finding container 39c3978079da33d9e6557e25cee8db2d8afba3a99753be3193acc9a4c7041468: Status 404 returned error can't find the container with id 39c3978079da33d9e6557e25cee8db2d8afba3a99753be3193acc9a4c7041468 Jul 08 13:23:13.003313 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:13.003295 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-g4lnf" Jul 08 13:23:13.012664 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:13.012626 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bcaa09d_1aac_4ac4_af53_f79bfc73b690.slice/crio-42b4eb73333a0316dfbcbe5bfb554a7c3f15f5fdfe9dd6132412ef1f42711d67 WatchSource:0}: Error finding container 42b4eb73333a0316dfbcbe5bfb554a7c3f15f5fdfe9dd6132412ef1f42711d67: Status 404 returned error can't find the container with id 42b4eb73333a0316dfbcbe5bfb554a7c3f15f5fdfe9dd6132412ef1f42711d67 Jul 08 13:23:13.016016 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:13.015998 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-pn22q" Jul 08 13:23:13.024711 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:13.024685 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67e86263_6fde_4f3a_a775_9e29eb8cacef.slice/crio-cc483a5859dc06e2e886f2a9c3c1998f4973328e40884408f5018977f508d395 WatchSource:0}: Error finding container cc483a5859dc06e2e886f2a9c3c1998f4973328e40884408f5018977f508d395: Status 404 returned error can't find the container with id cc483a5859dc06e2e886f2a9c3c1998f4973328e40884408f5018977f508d395 Jul 08 13:23:13.041668 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:13.041397 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-zfsbx" Jul 08 13:23:13.050602 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:13.050569 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36ed19b3_ce4a_466e_8d8f_e6ef13588a83.slice/crio-c999d742057f8186fea08aff0cf243ec3bba3ffbde2c4d73cb3bc6369ed9b45c WatchSource:0}: Error finding container c999d742057f8186fea08aff0cf243ec3bba3ffbde2c4d73cb3bc6369ed9b45c: Status 404 returned error can't find the container with id c999d742057f8186fea08aff0cf243ec3bba3ffbde2c4d73cb3bc6369ed9b45c Jul 08 13:23:13.071316 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:13.071279 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-85xxk" Jul 08 13:23:13.080641 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:13.080616 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb3380e5_69cf_4c03_b995_6de00f6b269f.slice/crio-391402ea376989ff1adffd1317c77413bca398951d502c20e6f358abc2773515 WatchSource:0}: Error finding container 391402ea376989ff1adffd1317c77413bca398951d502c20e6f358abc2773515: Status 404 returned error can't find the container with id 391402ea376989ff1adffd1317c77413bca398951d502c20e6f358abc2773515 Jul 08 13:23:13.088557 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:13.088540 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zs2fw" Jul 08 13:23:13.094706 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:13.094689 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:13.099731 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:13.099260 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd492fe73_0e0b_4b21_8ea1_2ab7cfcefe74.slice/crio-4b96f0d9aebe43f62fe91421019379bdc26b37e7c10da1a75c749e66e44bf6f9 WatchSource:0}: Error finding container 4b96f0d9aebe43f62fe91421019379bdc26b37e7c10da1a75c749e66e44bf6f9: Status 404 returned error can't find the container with id 4b96f0d9aebe43f62fe91421019379bdc26b37e7c10da1a75c749e66e44bf6f9 Jul 08 13:23:13.106524 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:13.106502 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd41d04d6_dc32_43f2_8e7c_24c36daed99f.slice/crio-a4ddb4cd40f18d81e7999384d4318278d7a2ab100361bc3924f32f0b90779ce4 WatchSource:0}: Error finding container a4ddb4cd40f18d81e7999384d4318278d7a2ab100361bc3924f32f0b90779ce4: Status 404 returned error can't find the container with id a4ddb4cd40f18d81e7999384d4318278d7a2ab100361bc3924f32f0b90779ce4 Jul 08 13:23:13.242778 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:13.242673 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a3381953-6131-477f-be6d-92ef7dc333a7-metrics-certs\") pod \"network-metrics-daemon-cnv6f\" (UID: \"a3381953-6131-477f-be6d-92ef7dc333a7\") " pod="openshift-multus/network-metrics-daemon-cnv6f" Jul 08 13:23:13.242944 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:13.242806 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 13:23:13.242944 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:13.242862 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3381953-6131-477f-be6d-92ef7dc333a7-metrics-certs podName:a3381953-6131-477f-be6d-92ef7dc333a7 nodeName:}" failed. No retries permitted until 2026-07-08 13:23:14.242846556 +0000 UTC m=+3.039787195 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a3381953-6131-477f-be6d-92ef7dc333a7-metrics-certs") pod "network-metrics-daemon-cnv6f" (UID: "a3381953-6131-477f-be6d-92ef7dc333a7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 13:23:13.343798 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:13.343766 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f93cae56-ca8c-4c37-866c-51c79f237448-original-pull-secret\") pod \"global-pull-secret-syncer-tp2lh\" (UID: \"f93cae56-ca8c-4c37-866c-51c79f237448\") " pod="kube-system/global-pull-secret-syncer-tp2lh" Jul 08 13:23:13.343969 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:13.343887 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 13:23:13.343969 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:13.343949 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f93cae56-ca8c-4c37-866c-51c79f237448-original-pull-secret podName:f93cae56-ca8c-4c37-866c-51c79f237448 nodeName:}" failed. No retries permitted until 2026-07-08 13:23:14.343931048 +0000 UTC m=+3.140871684 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f93cae56-ca8c-4c37-866c-51c79f237448-original-pull-secret") pod "global-pull-secret-syncer-tp2lh" (UID: "f93cae56-ca8c-4c37-866c-51c79f237448") : object "kube-system"/"original-pull-secret" not registered Jul 08 13:23:13.444064 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:13.444022 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wm2f\" (UniqueName: \"kubernetes.io/projected/b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5-kube-api-access-6wm2f\") pod \"network-check-target-rm9f4\" (UID: \"b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5\") " pod="openshift-network-diagnostics/network-check-target-rm9f4" Jul 08 13:23:13.444281 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:13.444229 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 13:23:13.444281 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:13.444255 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 13:23:13.444281 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:13.444268 2567 projected.go:196] Error preparing data for projected volume kube-api-access-6wm2f for pod openshift-network-diagnostics/network-check-target-rm9f4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 13:23:13.444446 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:13.444338 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5-kube-api-access-6wm2f podName:b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5 nodeName:}" failed. No retries permitted until 2026-07-08 13:23:14.444317481 +0000 UTC m=+3.241258146 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-6wm2f" (UniqueName: "kubernetes.io/projected/b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5-kube-api-access-6wm2f") pod "network-check-target-rm9f4" (UID: "b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 13:23:13.474968 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:13.474943 2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 13:23:13.527102 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:13.527007 2567 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 13:23:13.668372 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:13.668310 2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 13:18:12 +0000 UTC" deadline="2028-02-22 15:01:46.968611402 +0000 UTC" Jul 08 13:23:13.668372 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:13.668369 2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14257h38m33.300247181s" Jul 08 13:23:13.792861 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:13.792782 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rm9f4" Jul 08 13:23:13.793017 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:13.792912 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rm9f4" podUID="b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5" Jul 08 13:23:13.803406 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:13.803277 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" event={"ID":"d41d04d6-dc32-43f2-8e7c-24c36daed99f","Type":"ContainerStarted","Data":"a4ddb4cd40f18d81e7999384d4318278d7a2ab100361bc3924f32f0b90779ce4"} Jul 08 13:23:13.808290 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:13.808242 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zs2fw" event={"ID":"d492fe73-0e0b-4b21-8ea1-2ab7cfcefe74","Type":"ContainerStarted","Data":"4b96f0d9aebe43f62fe91421019379bdc26b37e7c10da1a75c749e66e44bf6f9"} Jul 08 13:23:13.814100 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:13.814056 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-g4lnf" event={"ID":"7bcaa09d-1aac-4ac4-af53-f79bfc73b690","Type":"ContainerStarted","Data":"42b4eb73333a0316dfbcbe5bfb554a7c3f15f5fdfe9dd6132412ef1f42711d67"} Jul 08 13:23:13.818717 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:13.818688 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-164.ec2.internal" event={"ID":"abd4fe04a7d63176e943bf70844ade69","Type":"ContainerStarted","Data":"07fc01d6c6f73a3736a9e36a921706844b19b527e8d18aed7fee36b8e622311c"} Jul 08 13:23:13.825949 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:13.825923 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-164.ec2.internal" event={"ID":"7264a5fddc17c5fbd72de875bb444234","Type":"ContainerStarted","Data":"e68608592a494a90a168b8ffd945f79d1d37ef2fb5da1f59c8063074c8566ea9"} Jul 08 13:23:13.843807 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:13.843776 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-85xxk" event={"ID":"db3380e5-69cf-4c03-b995-6de00f6b269f","Type":"ContainerStarted","Data":"391402ea376989ff1adffd1317c77413bca398951d502c20e6f358abc2773515"} Jul 08 13:23:13.847294 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:13.847268 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-zfsbx" event={"ID":"36ed19b3-ce4a-466e-8d8f-e6ef13588a83","Type":"ContainerStarted","Data":"c999d742057f8186fea08aff0cf243ec3bba3ffbde2c4d73cb3bc6369ed9b45c"} Jul 08 13:23:13.853806 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:13.853775 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-pn22q" event={"ID":"67e86263-6fde-4f3a-a775-9e29eb8cacef","Type":"ContainerStarted","Data":"cc483a5859dc06e2e886f2a9c3c1998f4973328e40884408f5018977f508d395"} Jul 08 13:23:13.863764 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:13.863738 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qmwvz" event={"ID":"ea6986a3-b0b4-4704-a47a-c991094c04f4","Type":"ContainerStarted","Data":"39c3978079da33d9e6557e25cee8db2d8afba3a99753be3193acc9a4c7041468"} Jul 08 13:23:13.872021 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:13.871992 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4wjn6" event={"ID":"029e8f8b-d068-4215-882f-0c72f71d5244","Type":"ContainerStarted","Data":"68323fbece0f6186a881bb0d36126cf2bf7d0b49b086e6bc42a7318785773d09"} Jul 08 13:23:13.880628 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:13.880562 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpkn4" event={"ID":"4b747c9c-61b0-41b9-ac34-7c8d1e63e6df","Type":"ContainerStarted","Data":"0899188ff8db0aeb373a2c7c989e6b62a4deb1b3c017ffd165266660a8b9abd8"} Jul 08 13:23:14.250520 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:14.250435 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a3381953-6131-477f-be6d-92ef7dc333a7-metrics-certs\") pod \"network-metrics-daemon-cnv6f\" (UID: \"a3381953-6131-477f-be6d-92ef7dc333a7\") " pod="openshift-multus/network-metrics-daemon-cnv6f" Jul 08 13:23:14.250689 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:14.250591 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 13:23:14.250689 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:14.250654 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3381953-6131-477f-be6d-92ef7dc333a7-metrics-certs podName:a3381953-6131-477f-be6d-92ef7dc333a7 nodeName:}" failed. No retries permitted until 2026-07-08 13:23:16.250634669 +0000 UTC m=+5.047575317 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a3381953-6131-477f-be6d-92ef7dc333a7-metrics-certs") pod "network-metrics-daemon-cnv6f" (UID: "a3381953-6131-477f-be6d-92ef7dc333a7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 13:23:14.351582 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:14.351014 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f93cae56-ca8c-4c37-866c-51c79f237448-original-pull-secret\") pod \"global-pull-secret-syncer-tp2lh\" (UID: \"f93cae56-ca8c-4c37-866c-51c79f237448\") " pod="kube-system/global-pull-secret-syncer-tp2lh" Jul 08 13:23:14.351582 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:14.351165 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 13:23:14.351582 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:14.351230 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f93cae56-ca8c-4c37-866c-51c79f237448-original-pull-secret podName:f93cae56-ca8c-4c37-866c-51c79f237448 nodeName:}" failed. No retries permitted until 2026-07-08 13:23:16.351212091 +0000 UTC m=+5.148152730 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f93cae56-ca8c-4c37-866c-51c79f237448-original-pull-secret") pod "global-pull-secret-syncer-tp2lh" (UID: "f93cae56-ca8c-4c37-866c-51c79f237448") : object "kube-system"/"original-pull-secret" not registered Jul 08 13:23:14.451817 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:14.451780 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wm2f\" (UniqueName: \"kubernetes.io/projected/b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5-kube-api-access-6wm2f\") pod \"network-check-target-rm9f4\" (UID: \"b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5\") " pod="openshift-network-diagnostics/network-check-target-rm9f4" Jul 08 13:23:14.452052 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:14.452033 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 13:23:14.452162 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:14.452060 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 13:23:14.452162 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:14.452089 2567 projected.go:196] Error preparing data for projected volume kube-api-access-6wm2f for pod openshift-network-diagnostics/network-check-target-rm9f4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 13:23:14.452162 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:14.452148 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5-kube-api-access-6wm2f podName:b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5 nodeName:}" failed. No retries permitted until 2026-07-08 13:23:16.452129819 +0000 UTC m=+5.249070476 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-6wm2f" (UniqueName: "kubernetes.io/projected/b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5-kube-api-access-6wm2f") pod "network-check-target-rm9f4" (UID: "b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 13:23:14.669037 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:14.668878 2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 13:18:12 +0000 UTC" deadline="2027-12-31 03:54:13.069817527 +0000 UTC" Jul 08 13:23:14.669037 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:14.668920 2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12974h30m58.400902157s" Jul 08 13:23:14.790928 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:14.790235 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnv6f" Jul 08 13:23:14.790928 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:14.790374 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnv6f" podUID="a3381953-6131-477f-be6d-92ef7dc333a7" Jul 08 13:23:14.790928 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:14.790769 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tp2lh" Jul 08 13:23:14.790928 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:14.790865 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tp2lh" podUID="f93cae56-ca8c-4c37-866c-51c79f237448" Jul 08 13:23:15.614247 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:15.614218 2567 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 13:23:15.793023 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:15.792939 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rm9f4" Jul 08 13:23:15.793488 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:15.793089 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rm9f4" podUID="b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5" Jul 08 13:23:16.269780 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:16.269697 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a3381953-6131-477f-be6d-92ef7dc333a7-metrics-certs\") pod \"network-metrics-daemon-cnv6f\" (UID: \"a3381953-6131-477f-be6d-92ef7dc333a7\") " pod="openshift-multus/network-metrics-daemon-cnv6f" Jul 08 13:23:16.270009 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:16.269847 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 13:23:16.270009 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:16.269914 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3381953-6131-477f-be6d-92ef7dc333a7-metrics-certs podName:a3381953-6131-477f-be6d-92ef7dc333a7 nodeName:}" failed. No retries permitted until 2026-07-08 13:23:20.269894768 +0000 UTC m=+9.066835406 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a3381953-6131-477f-be6d-92ef7dc333a7-metrics-certs") pod "network-metrics-daemon-cnv6f" (UID: "a3381953-6131-477f-be6d-92ef7dc333a7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 13:23:16.370776 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:16.370650 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f93cae56-ca8c-4c37-866c-51c79f237448-original-pull-secret\") pod \"global-pull-secret-syncer-tp2lh\" (UID: \"f93cae56-ca8c-4c37-866c-51c79f237448\") " pod="kube-system/global-pull-secret-syncer-tp2lh" Jul 08 13:23:16.372670 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:16.372580 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 13:23:16.372802 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:16.372720 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f93cae56-ca8c-4c37-866c-51c79f237448-original-pull-secret podName:f93cae56-ca8c-4c37-866c-51c79f237448 nodeName:}" failed. No retries permitted until 2026-07-08 13:23:20.372693985 +0000 UTC m=+9.169634632 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f93cae56-ca8c-4c37-866c-51c79f237448-original-pull-secret") pod "global-pull-secret-syncer-tp2lh" (UID: "f93cae56-ca8c-4c37-866c-51c79f237448") : object "kube-system"/"original-pull-secret" not registered Jul 08 13:23:16.471370 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:16.471331 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wm2f\" (UniqueName: \"kubernetes.io/projected/b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5-kube-api-access-6wm2f\") pod \"network-check-target-rm9f4\" (UID: \"b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5\") " pod="openshift-network-diagnostics/network-check-target-rm9f4" Jul 08 13:23:16.471524 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:16.471507 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 13:23:16.471596 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:16.471527 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 13:23:16.471596 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:16.471540 2567 projected.go:196] Error preparing data for projected volume kube-api-access-6wm2f for pod openshift-network-diagnostics/network-check-target-rm9f4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 13:23:16.471596 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:16.471594 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5-kube-api-access-6wm2f podName:b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5 nodeName:}" failed. No retries permitted until 2026-07-08 13:23:20.471576867 +0000 UTC m=+9.268517502 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-6wm2f" (UniqueName: "kubernetes.io/projected/b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5-kube-api-access-6wm2f") pod "network-check-target-rm9f4" (UID: "b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 13:23:16.790462 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:16.790426 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnv6f" Jul 08 13:23:16.790638 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:16.790470 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tp2lh" Jul 08 13:23:16.790638 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:16.790561 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnv6f" podUID="a3381953-6131-477f-be6d-92ef7dc333a7" Jul 08 13:23:16.790761 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:16.790719 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tp2lh" podUID="f93cae56-ca8c-4c37-866c-51c79f237448" Jul 08 13:23:17.790281 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:17.790252 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rm9f4" Jul 08 13:23:17.790656 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:17.790360 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rm9f4" podUID="b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5" Jul 08 13:23:18.791139 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:18.790457 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tp2lh" Jul 08 13:23:18.791139 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:18.790596 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tp2lh" podUID="f93cae56-ca8c-4c37-866c-51c79f237448" Jul 08 13:23:18.791139 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:18.790970 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnv6f" Jul 08 13:23:18.791139 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:18.791099 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnv6f" podUID="a3381953-6131-477f-be6d-92ef7dc333a7" Jul 08 13:23:19.790856 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:19.790329 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rm9f4" Jul 08 13:23:19.790856 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:19.790454 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rm9f4" podUID="b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5" Jul 08 13:23:20.302827 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:20.302746 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a3381953-6131-477f-be6d-92ef7dc333a7-metrics-certs\") pod \"network-metrics-daemon-cnv6f\" (UID: \"a3381953-6131-477f-be6d-92ef7dc333a7\") " pod="openshift-multus/network-metrics-daemon-cnv6f" Jul 08 13:23:20.303305 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:20.302948 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 13:23:20.303305 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:20.303027 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3381953-6131-477f-be6d-92ef7dc333a7-metrics-certs podName:a3381953-6131-477f-be6d-92ef7dc333a7 nodeName:}" failed. No retries permitted until 2026-07-08 13:23:28.303006396 +0000 UTC m=+17.099947041 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a3381953-6131-477f-be6d-92ef7dc333a7-metrics-certs") pod "network-metrics-daemon-cnv6f" (UID: "a3381953-6131-477f-be6d-92ef7dc333a7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 13:23:20.403711 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:20.403670 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f93cae56-ca8c-4c37-866c-51c79f237448-original-pull-secret\") pod \"global-pull-secret-syncer-tp2lh\" (UID: \"f93cae56-ca8c-4c37-866c-51c79f237448\") " pod="kube-system/global-pull-secret-syncer-tp2lh" Jul 08 13:23:20.403879 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:20.403780 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 13:23:20.403879 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:20.403844 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f93cae56-ca8c-4c37-866c-51c79f237448-original-pull-secret podName:f93cae56-ca8c-4c37-866c-51c79f237448 nodeName:}" failed. No retries permitted until 2026-07-08 13:23:28.403825843 +0000 UTC m=+17.200766481 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f93cae56-ca8c-4c37-866c-51c79f237448-original-pull-secret") pod "global-pull-secret-syncer-tp2lh" (UID: "f93cae56-ca8c-4c37-866c-51c79f237448") : object "kube-system"/"original-pull-secret" not registered Jul 08 13:23:20.504680 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:20.504631 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wm2f\" (UniqueName: \"kubernetes.io/projected/b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5-kube-api-access-6wm2f\") pod \"network-check-target-rm9f4\" (UID: \"b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5\") " pod="openshift-network-diagnostics/network-check-target-rm9f4" Jul 08 13:23:20.504859 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:20.504848 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 13:23:20.504921 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:20.504870 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 13:23:20.504921 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:20.504883 2567 projected.go:196] Error preparing data for projected volume kube-api-access-6wm2f for pod openshift-network-diagnostics/network-check-target-rm9f4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 13:23:20.505030 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:20.504939 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5-kube-api-access-6wm2f podName:b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5 nodeName:}" failed. No retries permitted until 2026-07-08 13:23:28.504921019 +0000 UTC m=+17.301861668 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-6wm2f" (UniqueName: "kubernetes.io/projected/b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5-kube-api-access-6wm2f") pod "network-check-target-rm9f4" (UID: "b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 13:23:20.790095 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:20.789996 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnv6f" Jul 08 13:23:20.790258 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:20.790167 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnv6f" podUID="a3381953-6131-477f-be6d-92ef7dc333a7" Jul 08 13:23:20.790426 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:20.790407 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tp2lh" Jul 08 13:23:20.790532 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:20.790514 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tp2lh" podUID="f93cae56-ca8c-4c37-866c-51c79f237448" Jul 08 13:23:21.792320 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:21.792260 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rm9f4" Jul 08 13:23:21.793655 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:21.793285 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rm9f4" podUID="b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5" Jul 08 13:23:22.790700 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:22.790668 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnv6f" Jul 08 13:23:22.790899 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:22.790817 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnv6f" podUID="a3381953-6131-477f-be6d-92ef7dc333a7" Jul 08 13:23:22.790975 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:22.790900 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tp2lh" Jul 08 13:23:22.791031 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:22.790995 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tp2lh" podUID="f93cae56-ca8c-4c37-866c-51c79f237448" Jul 08 13:23:23.792593 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:23.792040 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rm9f4" Jul 08 13:23:23.792593 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:23.792199 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rm9f4" podUID="b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5" Jul 08 13:23:24.790423 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:24.790200 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tp2lh" Jul 08 13:23:24.790600 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:24.790229 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnv6f" Jul 08 13:23:24.790600 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:24.790531 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tp2lh" podUID="f93cae56-ca8c-4c37-866c-51c79f237448" Jul 08 13:23:24.790713 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:24.790644 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnv6f" podUID="a3381953-6131-477f-be6d-92ef7dc333a7" Jul 08 13:23:25.793023 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:25.792992 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rm9f4" Jul 08 13:23:25.793458 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:25.793118 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rm9f4" podUID="b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5" Jul 08 13:23:26.790740 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:26.790699 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tp2lh" Jul 08 13:23:26.790740 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:26.790727 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnv6f" Jul 08 13:23:26.790981 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:26.790821 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tp2lh" podUID="f93cae56-ca8c-4c37-866c-51c79f237448" Jul 08 13:23:26.790981 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:26.790941 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnv6f" podUID="a3381953-6131-477f-be6d-92ef7dc333a7" Jul 08 13:23:27.789858 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:27.789824 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rm9f4" Jul 08 13:23:27.790356 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:27.789961 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rm9f4" podUID="b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5" Jul 08 13:23:28.362467 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:28.362427 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a3381953-6131-477f-be6d-92ef7dc333a7-metrics-certs\") pod \"network-metrics-daemon-cnv6f\" (UID: \"a3381953-6131-477f-be6d-92ef7dc333a7\") " pod="openshift-multus/network-metrics-daemon-cnv6f" Jul 08 13:23:28.362669 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:28.362597 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 13:23:28.362718 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:28.362670 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3381953-6131-477f-be6d-92ef7dc333a7-metrics-certs podName:a3381953-6131-477f-be6d-92ef7dc333a7 nodeName:}" failed. No retries permitted until 2026-07-08 13:23:44.362651004 +0000 UTC m=+33.159591638 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a3381953-6131-477f-be6d-92ef7dc333a7-metrics-certs") pod "network-metrics-daemon-cnv6f" (UID: "a3381953-6131-477f-be6d-92ef7dc333a7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 13:23:28.462719 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:28.462695 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f93cae56-ca8c-4c37-866c-51c79f237448-original-pull-secret\") pod \"global-pull-secret-syncer-tp2lh\" (UID: \"f93cae56-ca8c-4c37-866c-51c79f237448\") " pod="kube-system/global-pull-secret-syncer-tp2lh" Jul 08 13:23:28.462842 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:28.462824 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 13:23:28.462911 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:28.462903 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f93cae56-ca8c-4c37-866c-51c79f237448-original-pull-secret podName:f93cae56-ca8c-4c37-866c-51c79f237448 nodeName:}" failed. No retries permitted until 2026-07-08 13:23:44.462889097 +0000 UTC m=+33.259829730 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f93cae56-ca8c-4c37-866c-51c79f237448-original-pull-secret") pod "global-pull-secret-syncer-tp2lh" (UID: "f93cae56-ca8c-4c37-866c-51c79f237448") : object "kube-system"/"original-pull-secret" not registered Jul 08 13:23:28.563206 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:28.563170 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wm2f\" (UniqueName: \"kubernetes.io/projected/b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5-kube-api-access-6wm2f\") pod \"network-check-target-rm9f4\" (UID: \"b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5\") " pod="openshift-network-diagnostics/network-check-target-rm9f4" Jul 08 13:23:28.563375 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:28.563318 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 13:23:28.563375 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:28.563339 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 13:23:28.563375 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:28.563350 2567 projected.go:196] Error preparing data for projected volume kube-api-access-6wm2f for pod openshift-network-diagnostics/network-check-target-rm9f4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 13:23:28.563521 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:28.563414 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5-kube-api-access-6wm2f podName:b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5 nodeName:}" failed. No retries permitted until 2026-07-08 13:23:44.563395912 +0000 UTC m=+33.360336568 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-6wm2f" (UniqueName: "kubernetes.io/projected/b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5-kube-api-access-6wm2f") pod "network-check-target-rm9f4" (UID: "b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 13:23:28.790352 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:28.790317 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tp2lh" Jul 08 13:23:28.790802 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:28.790437 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tp2lh" podUID="f93cae56-ca8c-4c37-866c-51c79f237448" Jul 08 13:23:28.790802 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:28.790485 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnv6f" Jul 08 13:23:28.790802 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:28.790623 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnv6f" podUID="a3381953-6131-477f-be6d-92ef7dc333a7" Jul 08 13:23:29.789834 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:29.789809 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rm9f4" Jul 08 13:23:29.789998 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:29.789914 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rm9f4" podUID="b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5" Jul 08 13:23:30.790349 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:30.790318 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnv6f" Jul 08 13:23:30.790349 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:30.790359 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tp2lh" Jul 08 13:23:30.790878 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:30.790449 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnv6f" podUID="a3381953-6131-477f-be6d-92ef7dc333a7" Jul 08 13:23:30.790878 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:30.790591 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tp2lh" podUID="f93cae56-ca8c-4c37-866c-51c79f237448" Jul 08 13:23:31.791200 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:31.791161 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rm9f4" Jul 08 13:23:31.791651 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:31.791270 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rm9f4" podUID="b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5" Jul 08 13:23:32.790385 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:32.790346 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tp2lh" Jul 08 13:23:32.790543 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:32.790460 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tp2lh" podUID="f93cae56-ca8c-4c37-866c-51c79f237448" Jul 08 13:23:32.790543 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:32.790517 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnv6f" Jul 08 13:23:32.790649 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:32.790628 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnv6f" podUID="a3381953-6131-477f-be6d-92ef7dc333a7" Jul 08 13:23:33.790802 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:33.790572 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rm9f4" Jul 08 13:23:33.791282 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:33.790915 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rm9f4" podUID="b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5" Jul 08 13:23:33.948911 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:33.948873 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-85xxk" event={"ID":"db3380e5-69cf-4c03-b995-6de00f6b269f","Type":"ContainerStarted","Data":"62d47e62336e0cac3228826a28962dd99936f27fb7f8856e29abc71764b85f0a"} Jul 08 13:23:33.950243 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:33.950218 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-pn22q" event={"ID":"67e86263-6fde-4f3a-a775-9e29eb8cacef","Type":"ContainerStarted","Data":"2b62445c520f35c125fffe3bcb2c4b663f37b79fe6f6f4cc9df0d71bda754e6b"} Jul 08 13:23:33.951555 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:33.951530 2567 generic.go:356] "Generic (PLEG): container finished" podID="ea6986a3-b0b4-4704-a47a-c991094c04f4" containerID="cc4a1c0d8b7703a9d52877939ca9c7691ad62355300d9a88e343e1b0f93a9cca" exitCode=0 Jul 08 13:23:33.951667 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:33.951639 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qmwvz" event={"ID":"ea6986a3-b0b4-4704-a47a-c991094c04f4","Type":"ContainerDied","Data":"cc4a1c0d8b7703a9d52877939ca9c7691ad62355300d9a88e343e1b0f93a9cca"} Jul 08 13:23:33.953069 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:33.953047 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4wjn6" event={"ID":"029e8f8b-d068-4215-882f-0c72f71d5244","Type":"ContainerStarted","Data":"6a1ec7234f5938687b43d2f87683e979009fe98000be36814e20d852f4c38d05"} Jul 08 13:23:33.954633 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:33.954608 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpkn4" event={"ID":"4b747c9c-61b0-41b9-ac34-7c8d1e63e6df","Type":"ContainerStarted","Data":"69f46fa21289f7fc14b85bf3371cf72765d627380adda95359e1989817dcc688"} Jul 08 13:23:33.956856 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:33.956836 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" event={"ID":"d41d04d6-dc32-43f2-8e7c-24c36daed99f","Type":"ContainerStarted","Data":"0b13398e35cc65decf0f4c493c36f2e2d4bf18a525703f856e5cee3458271fdc"} Jul 08 13:23:33.956968 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:33.956907 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" event={"ID":"d41d04d6-dc32-43f2-8e7c-24c36daed99f","Type":"ContainerStarted","Data":"2dc0be605c1dec2c3eecbc9f285fec04aed43455d0523c6a2ffaae06bfc9e149"} Jul 08 13:23:33.956968 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:33.956949 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" event={"ID":"d41d04d6-dc32-43f2-8e7c-24c36daed99f","Type":"ContainerStarted","Data":"bafce422d1772a03563cdf9dc45560f977969c73105ac0e1ee3d4bed85b6a9cd"} Jul 08 13:23:33.957049 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:33.956984 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" event={"ID":"d41d04d6-dc32-43f2-8e7c-24c36daed99f","Type":"ContainerStarted","Data":"db8dc79e890cf503caba4fa1b3e79553461c56b0414cf3c6e6a9c13b6fba958d"} Jul 08 13:23:33.958245 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:33.958224 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zs2fw" event={"ID":"d492fe73-0e0b-4b21-8ea1-2ab7cfcefe74","Type":"ContainerStarted","Data":"0abe0570cdaa681b34bfe1e0d5ddd5ddc45affe199de3169c9d568e00d1712d6"} Jul 08 13:23:33.959532 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:33.959508 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-g4lnf" event={"ID":"7bcaa09d-1aac-4ac4-af53-f79bfc73b690","Type":"ContainerStarted","Data":"079bcb72e6185952bbc4030c1f490e5e033deeb0a26dc17a0b79d4c59d917215"} Jul 08 13:23:33.960914 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:33.960893 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-164.ec2.internal" event={"ID":"abd4fe04a7d63176e943bf70844ade69","Type":"ContainerStarted","Data":"b55ecb564ea29a06205524efde99bf2e10a4785d00359ce7349a1beed66d4056"} Jul 08 13:23:33.962256 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:33.962239 2567 generic.go:356] "Generic (PLEG): container finished" podID="7264a5fddc17c5fbd72de875bb444234" containerID="6b22172721d0a7c7112056923f01c86b23578805677e1ef3d006f06e30159014" exitCode=0 Jul 08 13:23:33.962328 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:33.962263 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-164.ec2.internal" event={"ID":"7264a5fddc17c5fbd72de875bb444234","Type":"ContainerDied","Data":"6b22172721d0a7c7112056923f01c86b23578805677e1ef3d006f06e30159014"} Jul 08 13:23:33.970851 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:33.970810 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-85xxk" podStartSLOduration=1.637304272 podStartE2EDuration="21.970798956s" totalImagesPullingTime="20.333494684s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 13:23:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:23:33.970400985 +0000 UTC m=+22.767341640" watchObservedRunningTime="2026-07-08 13:23:33.970798956 +0000 UTC m=+22.767739611" Jul 08 13:23:34.003487 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:34.003321 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-4wjn6" podStartSLOduration=2.732487431 podStartE2EDuration="23.003306873s" totalImagesPullingTime="20.270819442s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 13:23:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:23:34.002774637 +0000 UTC m=+22.799715294" watchObservedRunningTime="2026-07-08 13:23:34.003306873 +0000 UTC m=+22.800247528" Jul 08 13:23:34.055922 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:34.055870 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-g4lnf" podStartSLOduration=11.429405226 podStartE2EDuration="23.055850044s" totalImagesPullingTime="11.626444818s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:23:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:23:34.041376621 +0000 UTC m=+22.838317274" watchObservedRunningTime="2026-07-08 13:23:34.055850044 +0000 UTC m=+22.852790704" Jul 08 13:23:34.056128 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:34.056100 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-pn22q" podStartSLOduration=2.864736803 podStartE2EDuration="23.056065861s" totalImagesPullingTime="20.191329058s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 13:23:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:23:34.055537735 +0000 UTC m=+22.852478391" watchObservedRunningTime="2026-07-08 13:23:34.056065861 +0000 UTC m=+22.853006520" Jul 08 13:23:34.071918 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:34.071883 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-164.ec2.internal" podStartSLOduration=22.07187002 podStartE2EDuration="22.07187002s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 13:23:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:23:34.071455717 +0000 UTC m=+22.868396373" watchObservedRunningTime="2026-07-08 13:23:34.07187002 +0000 UTC m=+22.868810676" Jul 08 13:23:34.087546 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:34.087503 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-zs2fw" podStartSLOduration=2.355224924 podStartE2EDuration="22.087484972s" totalImagesPullingTime="19.732260048s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 13:23:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:23:34.08637875 +0000 UTC m=+22.883319407" watchObservedRunningTime="2026-07-08 13:23:34.087484972 +0000 UTC m=+22.884425630" Jul 08 13:23:34.517272 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:34.517235 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4lnf_7bcaa09d-1aac-4ac4-af53-f79bfc73b690/konnectivity-agent/0.log" limit=1024 Jul 08 13:23:34.789873 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:34.789783 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnv6f" Jul 08 13:23:34.789873 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:34.789831 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tp2lh" Jul 08 13:23:34.790111 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:34.789937 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnv6f" podUID="a3381953-6131-477f-be6d-92ef7dc333a7" Jul 08 13:23:34.790169 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:34.790100 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tp2lh" podUID="f93cae56-ca8c-4c37-866c-51c79f237448" Jul 08 13:23:34.883532 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:34.883506 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-g4lnf" Jul 08 13:23:34.884066 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:34.884035 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-g4lnf" Jul 08 13:23:34.966704 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:34.966673 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-164.ec2.internal" event={"ID":"7264a5fddc17c5fbd72de875bb444234","Type":"ContainerStarted","Data":"ab6bb67f9d8c6353fc0dc53ddd422e4cbb472fa9ce45207a0b4e256620d4960d"} Jul 08 13:23:34.968143 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:34.968116 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-zfsbx" event={"ID":"36ed19b3-ce4a-466e-8d8f-e6ef13588a83","Type":"ContainerStarted","Data":"5017d8ebb542bc2b9bfe7181afed0d7d7f509302442d76cf881168c6338566f7"} Jul 08 13:23:34.970854 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:34.970828 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" event={"ID":"d41d04d6-dc32-43f2-8e7c-24c36daed99f","Type":"ContainerStarted","Data":"3dddd6d67d06a19cd3fc34e43f1e4da543e87210ba8fa9d3438a6dd3dd02ec64"} Jul 08 13:23:34.970943 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:34.970911 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" event={"ID":"d41d04d6-dc32-43f2-8e7c-24c36daed99f","Type":"ContainerStarted","Data":"ac00cc750fb65ef8614688ce439a8136f48645fdb73b36b7fb478eddab33547f"} Jul 08 13:23:34.971598 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:34.971567 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-g4lnf" Jul 08 13:23:34.972014 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:34.972000 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-g4lnf" Jul 08 13:23:34.995323 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:34.995273 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-164.ec2.internal" podStartSLOduration=22.995252208 podStartE2EDuration="22.995252208s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 13:23:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:23:34.980921701 +0000 UTC m=+23.777862356" watchObservedRunningTime="2026-07-08 13:23:34.995252208 +0000 UTC m=+23.792192865" Jul 08 13:23:35.009052 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:35.008989 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-zfsbx" podStartSLOduration=2.813348011 podStartE2EDuration="23.008968855s" totalImagesPullingTime="20.195620844s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 13:23:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:23:35.008569069 +0000 UTC m=+23.805509724" watchObservedRunningTime="2026-07-08 13:23:35.008968855 +0000 UTC m=+23.805909512" Jul 08 13:23:35.272406 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:35.272384 2567 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 08 13:23:35.297591 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:35.297557 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4lnf_7bcaa09d-1aac-4ac4-af53-f79bfc73b690/konnectivity-agent/0.log" limit=1024 Jul 08 13:23:35.689616 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:35.689581 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4lnf_7bcaa09d-1aac-4ac4-af53-f79bfc73b690/konnectivity-agent/0.log" limit=1024 Jul 08 13:23:35.702602 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:35.702522 2567 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-08T13:23:35.272401332Z","UUID":"6c56c949-c60e-4eac-b569-881491645401","Handler":null,"Name":"","Endpoint":""} Jul 08 13:23:35.703893 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:35.703869 2567 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 08 13:23:35.703988 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:35.703900 2567 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 08 13:23:35.792277 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:35.792249 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rm9f4" Jul 08 13:23:35.792437 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:35.792340 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rm9f4" podUID="b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5" Jul 08 13:23:35.889685 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:35.889650 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4lnf_7bcaa09d-1aac-4ac4-af53-f79bfc73b690/konnectivity-agent/0.log" limit=1024 Jul 08 13:23:35.976656 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:35.976622 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpkn4" event={"ID":"4b747c9c-61b0-41b9-ac34-7c8d1e63e6df","Type":"ContainerStarted","Data":"0b63799e862379fc1c65f5ac4bdfb68651c9f8c3ea8c37c35ca235cdc4251ef9"} Jul 08 13:23:36.095287 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:36.095252 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4lnf_7bcaa09d-1aac-4ac4-af53-f79bfc73b690/konnectivity-agent/0.log" limit=1024 Jul 08 13:23:36.498567 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:36.498539 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4lnf_7bcaa09d-1aac-4ac4-af53-f79bfc73b690/konnectivity-agent/0.log" limit=1024 Jul 08 13:23:36.790071 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:36.789997 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnv6f" Jul 08 13:23:36.790071 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:36.790026 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tp2lh" Jul 08 13:23:36.790320 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:36.790132 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnv6f" podUID="a3381953-6131-477f-be6d-92ef7dc333a7" Jul 08 13:23:36.790371 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:36.790307 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tp2lh" podUID="f93cae56-ca8c-4c37-866c-51c79f237448" Jul 08 13:23:36.895210 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:36.895177 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4lnf_7bcaa09d-1aac-4ac4-af53-f79bfc73b690/konnectivity-agent/0.log" limit=1024 Jul 08 13:23:36.980365 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:36.980317 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpkn4" event={"ID":"4b747c9c-61b0-41b9-ac34-7c8d1e63e6df","Type":"ContainerStarted","Data":"1180c31fa202e3145b5ba8a2a7eb0e5620b329b32d038c8e50c1b4a8c527408b"} Jul 08 13:23:36.983930 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:36.983903 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" event={"ID":"d41d04d6-dc32-43f2-8e7c-24c36daed99f","Type":"ContainerStarted","Data":"0df5eac731ffd8f6b0dba0aaf143006d2f9a187574e7c87ace3902b46f2a720a"} Jul 08 13:23:36.998765 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:36.998654 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpkn4" podStartSLOduration=8.060902358 podStartE2EDuration="25.998639443s" totalImagesPullingTime="17.937737085s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 13:23:11 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:23:36.998570877 +0000 UTC m=+25.795511537" watchObservedRunningTime="2026-07-08 13:23:36.998639443 +0000 UTC m=+25.795580098" Jul 08 13:23:37.132762 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:37.132683 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4lnf_7bcaa09d-1aac-4ac4-af53-f79bfc73b690/konnectivity-agent/0.log" limit=1024 Jul 08 13:23:37.298548 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:37.298516 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4lnf_7bcaa09d-1aac-4ac4-af53-f79bfc73b690/konnectivity-agent/0.log" limit=1024 Jul 08 13:23:37.696398 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:37.696361 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4lnf_7bcaa09d-1aac-4ac4-af53-f79bfc73b690/konnectivity-agent/0.log" limit=1024 Jul 08 13:23:37.790419 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:37.790392 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rm9f4" Jul 08 13:23:37.790607 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:37.790524 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rm9f4" podUID="b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5" Jul 08 13:23:37.894857 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:37.894820 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4lnf_7bcaa09d-1aac-4ac4-af53-f79bfc73b690/konnectivity-agent/0.log" limit=1024 Jul 08 13:23:37.944312 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:37.944273 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4lnf_7bcaa09d-1aac-4ac4-af53-f79bfc73b690/konnectivity-agent/0.log" limit=1024 Jul 08 13:23:38.100858 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:38.100812 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4lnf_7bcaa09d-1aac-4ac4-af53-f79bfc73b690/konnectivity-agent/0.log" limit=1024 Jul 08 13:23:38.497542 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:38.497484 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4lnf_7bcaa09d-1aac-4ac4-af53-f79bfc73b690/konnectivity-agent/0.log" limit=1024 Jul 08 13:23:38.790335 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:38.790256 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tp2lh" Jul 08 13:23:38.790494 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:38.790256 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnv6f" Jul 08 13:23:38.790494 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:38.790372 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tp2lh" podUID="f93cae56-ca8c-4c37-866c-51c79f237448" Jul 08 13:23:38.790494 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:38.790439 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnv6f" podUID="a3381953-6131-477f-be6d-92ef7dc333a7" Jul 08 13:23:38.895265 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:38.895236 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4lnf_7bcaa09d-1aac-4ac4-af53-f79bfc73b690/konnectivity-agent/0.log" limit=1024 Jul 08 13:23:39.497519 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:39.497486 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4lnf_7bcaa09d-1aac-4ac4-af53-f79bfc73b690/konnectivity-agent/0.log" limit=1024 Jul 08 13:23:39.789964 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:39.789829 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rm9f4" Jul 08 13:23:39.790066 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:39.790046 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rm9f4" podUID="b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5" Jul 08 13:23:39.892710 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:39.892679 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4lnf_7bcaa09d-1aac-4ac4-af53-f79bfc73b690/konnectivity-agent/0.log" limit=1024 Jul 08 13:23:39.995245 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:39.995208 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" event={"ID":"d41d04d6-dc32-43f2-8e7c-24c36daed99f","Type":"ContainerStarted","Data":"df16808c3e79e6675c441ea2692353a74b787f549c7dbef3b94d977adc1d7229"} Jul 08 13:23:39.995501 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:39.995481 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:39.995740 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:39.995720 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:39.995844 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:39.995830 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:40.013699 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:40.013676 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:40.013829 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:40.013815 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:23:40.023251 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:40.023213 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" podStartSLOduration=7.841814651 podStartE2EDuration="28.023198785s" totalImagesPullingTime="20.181384134s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 13:23:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:23:40.022952846 +0000 UTC m=+28.819893502" watchObservedRunningTime="2026-07-08 13:23:40.023198785 +0000 UTC m=+28.820139441" Jul 08 13:23:40.296304 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:40.296229 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4lnf_7bcaa09d-1aac-4ac4-af53-f79bfc73b690/konnectivity-agent/0.log" limit=1024 Jul 08 13:23:40.491768 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:40.491742 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4lnf_7bcaa09d-1aac-4ac4-af53-f79bfc73b690/konnectivity-agent/0.log" limit=1024 Jul 08 13:23:40.790312 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:40.790282 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tp2lh" Jul 08 13:23:40.791031 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:40.790377 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tp2lh" podUID="f93cae56-ca8c-4c37-866c-51c79f237448" Jul 08 13:23:40.791031 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:40.790439 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnv6f" Jul 08 13:23:40.791031 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:40.790548 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnv6f" podUID="a3381953-6131-477f-be6d-92ef7dc333a7" Jul 08 13:23:40.891378 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:40.891342 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-g4lnf_7bcaa09d-1aac-4ac4-af53-f79bfc73b690/konnectivity-agent/0.log" limit=1024 Jul 08 13:23:40.999163 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:40.999127 2567 generic.go:356] "Generic (PLEG): container finished" podID="ea6986a3-b0b4-4704-a47a-c991094c04f4" containerID="9fc133b3af1b5c1163162f4f57734b05a42d4303313c78adefa593f005c93506" exitCode=0 Jul 08 13:23:40.999294 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:40.999204 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qmwvz" event={"ID":"ea6986a3-b0b4-4704-a47a-c991094c04f4","Type":"ContainerDied","Data":"9fc133b3af1b5c1163162f4f57734b05a42d4303313c78adefa593f005c93506"} Jul 08 13:23:41.791126 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:41.791092 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rm9f4" Jul 08 13:23:41.791569 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:41.791194 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rm9f4" podUID="b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5" Jul 08 13:23:42.790830 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:42.790651 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnv6f" Jul 08 13:23:42.791008 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:42.790681 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tp2lh" Jul 08 13:23:42.791008 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:42.790910 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnv6f" podUID="a3381953-6131-477f-be6d-92ef7dc333a7" Jul 08 13:23:42.791008 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:42.790971 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tp2lh" podUID="f93cae56-ca8c-4c37-866c-51c79f237448" Jul 08 13:23:43.005899 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:43.005863 2567 generic.go:356] "Generic (PLEG): container finished" podID="ea6986a3-b0b4-4704-a47a-c991094c04f4" containerID="7eb175fbfae8dc4789d17853b7ffd49eb3a0f9f263dd14e0f4bd2e29ca0b8bd4" exitCode=0 Jul 08 13:23:43.006375 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:43.005909 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qmwvz" event={"ID":"ea6986a3-b0b4-4704-a47a-c991094c04f4","Type":"ContainerDied","Data":"7eb175fbfae8dc4789d17853b7ffd49eb3a0f9f263dd14e0f4bd2e29ca0b8bd4"} Jul 08 13:23:43.789871 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:43.789833 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rm9f4" Jul 08 13:23:43.790023 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:43.789944 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rm9f4" podUID="b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5" Jul 08 13:23:44.379027 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:44.378997 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a3381953-6131-477f-be6d-92ef7dc333a7-metrics-certs\") pod \"network-metrics-daemon-cnv6f\" (UID: \"a3381953-6131-477f-be6d-92ef7dc333a7\") " pod="openshift-multus/network-metrics-daemon-cnv6f" Jul 08 13:23:44.379363 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:44.379122 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 13:23:44.379363 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:44.379172 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3381953-6131-477f-be6d-92ef7dc333a7-metrics-certs podName:a3381953-6131-477f-be6d-92ef7dc333a7 nodeName:}" failed. No retries permitted until 2026-07-08 13:24:16.379158164 +0000 UTC m=+65.176098797 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a3381953-6131-477f-be6d-92ef7dc333a7-metrics-certs") pod "network-metrics-daemon-cnv6f" (UID: "a3381953-6131-477f-be6d-92ef7dc333a7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 13:23:44.479474 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:44.479446 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f93cae56-ca8c-4c37-866c-51c79f237448-original-pull-secret\") pod \"global-pull-secret-syncer-tp2lh\" (UID: \"f93cae56-ca8c-4c37-866c-51c79f237448\") " pod="kube-system/global-pull-secret-syncer-tp2lh" Jul 08 13:23:44.479616 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:44.479543 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 13:23:44.479616 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:44.479592 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f93cae56-ca8c-4c37-866c-51c79f237448-original-pull-secret podName:f93cae56-ca8c-4c37-866c-51c79f237448 nodeName:}" failed. No retries permitted until 2026-07-08 13:24:16.479579912 +0000 UTC m=+65.276520546 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f93cae56-ca8c-4c37-866c-51c79f237448-original-pull-secret") pod "global-pull-secret-syncer-tp2lh" (UID: "f93cae56-ca8c-4c37-866c-51c79f237448") : object "kube-system"/"original-pull-secret" not registered Jul 08 13:23:44.580194 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:44.580163 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wm2f\" (UniqueName: \"kubernetes.io/projected/b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5-kube-api-access-6wm2f\") pod \"network-check-target-rm9f4\" (UID: \"b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5\") " pod="openshift-network-diagnostics/network-check-target-rm9f4" Jul 08 13:23:44.580336 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:44.580297 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 13:23:44.580336 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:44.580311 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 13:23:44.580336 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:44.580320 2567 projected.go:196] Error preparing data for projected volume kube-api-access-6wm2f for pod openshift-network-diagnostics/network-check-target-rm9f4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 13:23:44.580453 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:44.580366 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5-kube-api-access-6wm2f podName:b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5 nodeName:}" failed. No retries permitted until 2026-07-08 13:24:16.580353129 +0000 UTC m=+65.377293763 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-6wm2f" (UniqueName: "kubernetes.io/projected/b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5-kube-api-access-6wm2f") pod "network-check-target-rm9f4" (UID: "b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 13:23:44.790036 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:44.789955 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tp2lh" Jul 08 13:23:44.790196 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:44.789955 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnv6f" Jul 08 13:23:44.790196 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:44.790053 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tp2lh" podUID="f93cae56-ca8c-4c37-866c-51c79f237448" Jul 08 13:23:44.790196 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:44.790156 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnv6f" podUID="a3381953-6131-477f-be6d-92ef7dc333a7" Jul 08 13:23:45.012764 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:45.012733 2567 generic.go:356] "Generic (PLEG): container finished" podID="ea6986a3-b0b4-4704-a47a-c991094c04f4" containerID="856c67a3a6be87842a0ff581fc3df1a1920506a52c740da8b3ace516a7737db9" exitCode=0 Jul 08 13:23:45.012961 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:45.012791 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qmwvz" event={"ID":"ea6986a3-b0b4-4704-a47a-c991094c04f4","Type":"ContainerDied","Data":"856c67a3a6be87842a0ff581fc3df1a1920506a52c740da8b3ace516a7737db9"} Jul 08 13:23:45.790370 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:45.790334 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rm9f4" Jul 08 13:23:45.790817 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:45.790445 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rm9f4" podUID="b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5" Jul 08 13:23:46.790610 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:46.790579 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnv6f" Jul 08 13:23:46.791227 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:46.790579 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tp2lh" Jul 08 13:23:46.791227 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:46.790715 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnv6f" podUID="a3381953-6131-477f-be6d-92ef7dc333a7" Jul 08 13:23:46.791227 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:46.790790 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tp2lh" podUID="f93cae56-ca8c-4c37-866c-51c79f237448" Jul 08 13:23:47.789981 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:47.789939 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rm9f4" Jul 08 13:23:47.790180 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:47.790066 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rm9f4" podUID="b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5" Jul 08 13:23:48.790370 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:48.790344 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnv6f" Jul 08 13:23:48.790817 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:48.790345 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tp2lh" Jul 08 13:23:48.790817 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:48.790443 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnv6f" podUID="a3381953-6131-477f-be6d-92ef7dc333a7" Jul 08 13:23:48.790817 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:48.790548 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tp2lh" podUID="f93cae56-ca8c-4c37-866c-51c79f237448" Jul 08 13:23:49.790524 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:49.790492 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rm9f4" Jul 08 13:23:49.790896 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:49.790595 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rm9f4" podUID="b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5" Jul 08 13:23:50.757541 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:50.757404 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tp2lh"] Jul 08 13:23:50.757541 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:50.757542 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tp2lh" Jul 08 13:23:50.757809 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:50.757647 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tp2lh" podUID="f93cae56-ca8c-4c37-866c-51c79f237448" Jul 08 13:23:50.763949 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:50.763909 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rm9f4"] Jul 08 13:23:50.764104 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:50.763981 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cnv6f"] Jul 08 13:23:50.764104 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:50.764089 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnv6f" Jul 08 13:23:50.764349 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:50.764201 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnv6f" podUID="a3381953-6131-477f-be6d-92ef7dc333a7" Jul 08 13:23:50.764442 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:50.764374 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rm9f4" Jul 08 13:23:50.764500 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:50.764454 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rm9f4" podUID="b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5" Jul 08 13:23:51.790726 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:51.790651 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnv6f" Jul 08 13:23:51.791197 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:51.790750 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnv6f" podUID="a3381953-6131-477f-be6d-92ef7dc333a7" Jul 08 13:23:52.790315 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:52.790142 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rm9f4" Jul 08 13:23:52.790449 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:52.790195 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tp2lh" Jul 08 13:23:52.790449 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:52.790396 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rm9f4" podUID="b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5" Jul 08 13:23:52.790555 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:52.790490 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tp2lh" podUID="f93cae56-ca8c-4c37-866c-51c79f237448" Jul 08 13:23:53.035682 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:53.035645 2567 generic.go:356] "Generic (PLEG): container finished" podID="ea6986a3-b0b4-4704-a47a-c991094c04f4" containerID="ef6430f30c07947600441b13627801674336fdca148d6a48817d1fb445806146" exitCode=0 Jul 08 13:23:53.036047 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:53.035692 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qmwvz" event={"ID":"ea6986a3-b0b4-4704-a47a-c991094c04f4","Type":"ContainerDied","Data":"ef6430f30c07947600441b13627801674336fdca148d6a48817d1fb445806146"} Jul 08 13:23:53.790095 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:53.790049 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnv6f" Jul 08 13:23:53.790258 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:23:53.790200 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cnv6f" podUID="a3381953-6131-477f-be6d-92ef7dc333a7" Jul 08 13:23:54.040726 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.040648 2567 generic.go:356] "Generic (PLEG): container finished" podID="ea6986a3-b0b4-4704-a47a-c991094c04f4" containerID="84511be0a34a0255c4d97af0fff8ca2268aceeef207def098ace7a89a2efd24a" exitCode=0 Jul 08 13:23:54.040726 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.040707 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qmwvz" event={"ID":"ea6986a3-b0b4-4704-a47a-c991094c04f4","Type":"ContainerDied","Data":"84511be0a34a0255c4d97af0fff8ca2268aceeef207def098ace7a89a2efd24a"} Jul 08 13:23:54.062904 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.062879 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-164.ec2.internal" event="NodeReady" Jul 08 13:23:54.063049 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.063037 2567 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 08 13:23:54.096902 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.096871 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-679776947f-8st28"] Jul 08 13:23:54.117423 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.117355 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-gv29f"] Jul 08 13:23:54.118271 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.118061 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-679776947f-8st28" Jul 08 13:23:54.121146 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.120922 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 08 13:23:54.121146 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.120937 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 08 13:23:54.121146 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.120977 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-dhbls\"" Jul 08 13:23:54.121146 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.120977 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 08 13:23:54.127938 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.127798 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 08 13:23:54.135266 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.135246 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-wgz5x"] Jul 08 13:23:54.135380 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.135366 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gv29f" Jul 08 13:23:54.137669 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.137650 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 08 13:23:54.137776 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.137692 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9rlw5\"" Jul 08 13:23:54.137841 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.137787 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 08 13:23:54.155944 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.155927 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-679776947f-8st28"] Jul 08 13:23:54.156047 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.155955 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gv29f"] Jul 08 13:23:54.156047 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.155977 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wgz5x"] Jul 08 13:23:54.156152 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.156049 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wgz5x" Jul 08 13:23:54.158454 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.158419 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 08 13:23:54.158554 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.158472 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 08 13:23:54.158554 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.158549 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 08 13:23:54.158699 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.158684 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-fpm48\"" Jul 08 13:23:54.255133 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.255069 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/67cefd1d-608b-4db3-801e-4ca6c47aa8ee-trusted-ca\") pod \"image-registry-679776947f-8st28\" (UID: \"67cefd1d-608b-4db3-801e-4ca6c47aa8ee\") " pod="openshift-image-registry/image-registry-679776947f-8st28" Jul 08 13:23:54.255285 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.255141 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/faf0a62e-56a2-449c-9aed-2859b1bfe9bc-config-volume\") pod \"dns-default-gv29f\" (UID: \"faf0a62e-56a2-449c-9aed-2859b1bfe9bc\") " pod="openshift-dns/dns-default-gv29f" Jul 08 13:23:54.255285 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.255179 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/67cefd1d-608b-4db3-801e-4ca6c47aa8ee-installation-pull-secrets\") pod \"image-registry-679776947f-8st28\" (UID: \"67cefd1d-608b-4db3-801e-4ca6c47aa8ee\") " pod="openshift-image-registry/image-registry-679776947f-8st28" Jul 08 13:23:54.255285 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.255199 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsfxf\" (UniqueName: \"kubernetes.io/projected/1111874c-8d49-43f3-a84a-fa214f432735-kube-api-access-nsfxf\") pod \"ingress-canary-wgz5x\" (UID: \"1111874c-8d49-43f3-a84a-fa214f432735\") " pod="openshift-ingress-canary/ingress-canary-wgz5x" Jul 08 13:23:54.255285 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.255242 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/67cefd1d-608b-4db3-801e-4ca6c47aa8ee-registry-certificates\") pod \"image-registry-679776947f-8st28\" (UID: \"67cefd1d-608b-4db3-801e-4ca6c47aa8ee\") " pod="openshift-image-registry/image-registry-679776947f-8st28" Jul 08 13:23:54.255285 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.255263 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/67cefd1d-608b-4db3-801e-4ca6c47aa8ee-image-registry-private-configuration\") pod \"image-registry-679776947f-8st28\" (UID: \"67cefd1d-608b-4db3-801e-4ca6c47aa8ee\") " pod="openshift-image-registry/image-registry-679776947f-8st28" Jul 08 13:23:54.255461 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.255298 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lpp7\" (UniqueName: \"kubernetes.io/projected/67cefd1d-608b-4db3-801e-4ca6c47aa8ee-kube-api-access-9lpp7\") pod \"image-registry-679776947f-8st28\" (UID: \"67cefd1d-608b-4db3-801e-4ca6c47aa8ee\") " pod="openshift-image-registry/image-registry-679776947f-8st28" Jul 08 13:23:54.255461 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.255323 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/67cefd1d-608b-4db3-801e-4ca6c47aa8ee-registry-tls\") pod \"image-registry-679776947f-8st28\" (UID: \"67cefd1d-608b-4db3-801e-4ca6c47aa8ee\") " pod="openshift-image-registry/image-registry-679776947f-8st28" Jul 08 13:23:54.255461 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.255341 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/67cefd1d-608b-4db3-801e-4ca6c47aa8ee-bound-sa-token\") pod \"image-registry-679776947f-8st28\" (UID: \"67cefd1d-608b-4db3-801e-4ca6c47aa8ee\") " pod="openshift-image-registry/image-registry-679776947f-8st28" Jul 08 13:23:54.255461 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.255362 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/faf0a62e-56a2-449c-9aed-2859b1bfe9bc-metrics-tls\") pod \"dns-default-gv29f\" (UID: \"faf0a62e-56a2-449c-9aed-2859b1bfe9bc\") " pod="openshift-dns/dns-default-gv29f" Jul 08 13:23:54.255461 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.255388 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/67cefd1d-608b-4db3-801e-4ca6c47aa8ee-ca-trust-extracted\") pod \"image-registry-679776947f-8st28\" (UID: \"67cefd1d-608b-4db3-801e-4ca6c47aa8ee\") " pod="openshift-image-registry/image-registry-679776947f-8st28" Jul 08 13:23:54.255461 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.255403 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1111874c-8d49-43f3-a84a-fa214f432735-cert\") pod \"ingress-canary-wgz5x\" (UID: \"1111874c-8d49-43f3-a84a-fa214f432735\") " pod="openshift-ingress-canary/ingress-canary-wgz5x" Jul 08 13:23:54.255461 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.255415 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/faf0a62e-56a2-449c-9aed-2859b1bfe9bc-tmp-dir\") pod \"dns-default-gv29f\" (UID: \"faf0a62e-56a2-449c-9aed-2859b1bfe9bc\") " pod="openshift-dns/dns-default-gv29f" Jul 08 13:23:54.255461 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.255440 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjqlx\" (UniqueName: \"kubernetes.io/projected/faf0a62e-56a2-449c-9aed-2859b1bfe9bc-kube-api-access-vjqlx\") pod \"dns-default-gv29f\" (UID: \"faf0a62e-56a2-449c-9aed-2859b1bfe9bc\") " pod="openshift-dns/dns-default-gv29f" Jul 08 13:23:54.356016 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.355941 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjqlx\" (UniqueName: \"kubernetes.io/projected/faf0a62e-56a2-449c-9aed-2859b1bfe9bc-kube-api-access-vjqlx\") pod \"dns-default-gv29f\" (UID: \"faf0a62e-56a2-449c-9aed-2859b1bfe9bc\") " pod="openshift-dns/dns-default-gv29f" Jul 08 13:23:54.356016 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.355972 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/67cefd1d-608b-4db3-801e-4ca6c47aa8ee-trusted-ca\") pod \"image-registry-679776947f-8st28\" (UID: \"67cefd1d-608b-4db3-801e-4ca6c47aa8ee\") " pod="openshift-image-registry/image-registry-679776947f-8st28" Jul 08 13:23:54.356016 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.355994 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/faf0a62e-56a2-449c-9aed-2859b1bfe9bc-config-volume\") pod \"dns-default-gv29f\" (UID: \"faf0a62e-56a2-449c-9aed-2859b1bfe9bc\") " pod="openshift-dns/dns-default-gv29f" Jul 08 13:23:54.356249 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.356032 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/67cefd1d-608b-4db3-801e-4ca6c47aa8ee-installation-pull-secrets\") pod \"image-registry-679776947f-8st28\" (UID: \"67cefd1d-608b-4db3-801e-4ca6c47aa8ee\") " pod="openshift-image-registry/image-registry-679776947f-8st28" Jul 08 13:23:54.356249 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.356148 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsfxf\" (UniqueName: \"kubernetes.io/projected/1111874c-8d49-43f3-a84a-fa214f432735-kube-api-access-nsfxf\") pod \"ingress-canary-wgz5x\" (UID: \"1111874c-8d49-43f3-a84a-fa214f432735\") " pod="openshift-ingress-canary/ingress-canary-wgz5x" Jul 08 13:23:54.356249 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.356181 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/67cefd1d-608b-4db3-801e-4ca6c47aa8ee-registry-certificates\") pod \"image-registry-679776947f-8st28\" (UID: \"67cefd1d-608b-4db3-801e-4ca6c47aa8ee\") " pod="openshift-image-registry/image-registry-679776947f-8st28" Jul 08 13:23:54.356249 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.356220 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/67cefd1d-608b-4db3-801e-4ca6c47aa8ee-image-registry-private-configuration\") pod \"image-registry-679776947f-8st28\" (UID: \"67cefd1d-608b-4db3-801e-4ca6c47aa8ee\") " pod="openshift-image-registry/image-registry-679776947f-8st28" Jul 08 13:23:54.356530 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.356270 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lpp7\" (UniqueName: \"kubernetes.io/projected/67cefd1d-608b-4db3-801e-4ca6c47aa8ee-kube-api-access-9lpp7\") pod \"image-registry-679776947f-8st28\" (UID: \"67cefd1d-608b-4db3-801e-4ca6c47aa8ee\") " pod="openshift-image-registry/image-registry-679776947f-8st28" Jul 08 13:23:54.356530 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.356309 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/67cefd1d-608b-4db3-801e-4ca6c47aa8ee-registry-tls\") pod \"image-registry-679776947f-8st28\" (UID: \"67cefd1d-608b-4db3-801e-4ca6c47aa8ee\") " pod="openshift-image-registry/image-registry-679776947f-8st28" Jul 08 13:23:54.356530 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.356342 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/67cefd1d-608b-4db3-801e-4ca6c47aa8ee-bound-sa-token\") pod \"image-registry-679776947f-8st28\" (UID: \"67cefd1d-608b-4db3-801e-4ca6c47aa8ee\") " pod="openshift-image-registry/image-registry-679776947f-8st28" Jul 08 13:23:54.356530 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.356365 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/faf0a62e-56a2-449c-9aed-2859b1bfe9bc-metrics-tls\") pod \"dns-default-gv29f\" (UID: \"faf0a62e-56a2-449c-9aed-2859b1bfe9bc\") " pod="openshift-dns/dns-default-gv29f" Jul 08 13:23:54.356530 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.356407 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/67cefd1d-608b-4db3-801e-4ca6c47aa8ee-ca-trust-extracted\") pod \"image-registry-679776947f-8st28\" (UID: \"67cefd1d-608b-4db3-801e-4ca6c47aa8ee\") " pod="openshift-image-registry/image-registry-679776947f-8st28" Jul 08 13:23:54.356530 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.356439 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1111874c-8d49-43f3-a84a-fa214f432735-cert\") pod \"ingress-canary-wgz5x\" (UID: \"1111874c-8d49-43f3-a84a-fa214f432735\") " pod="openshift-ingress-canary/ingress-canary-wgz5x" Jul 08 13:23:54.356530 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.356461 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/faf0a62e-56a2-449c-9aed-2859b1bfe9bc-tmp-dir\") pod \"dns-default-gv29f\" (UID: \"faf0a62e-56a2-449c-9aed-2859b1bfe9bc\") " pod="openshift-dns/dns-default-gv29f" Jul 08 13:23:54.356876 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.356665 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/faf0a62e-56a2-449c-9aed-2859b1bfe9bc-config-volume\") pod \"dns-default-gv29f\" (UID: \"faf0a62e-56a2-449c-9aed-2859b1bfe9bc\") " pod="openshift-dns/dns-default-gv29f" Jul 08 13:23:54.356876 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.356758 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/faf0a62e-56a2-449c-9aed-2859b1bfe9bc-tmp-dir\") pod \"dns-default-gv29f\" (UID: \"faf0a62e-56a2-449c-9aed-2859b1bfe9bc\") " pod="openshift-dns/dns-default-gv29f" Jul 08 13:23:54.357029 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.357006 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/67cefd1d-608b-4db3-801e-4ca6c47aa8ee-ca-trust-extracted\") pod \"image-registry-679776947f-8st28\" (UID: \"67cefd1d-608b-4db3-801e-4ca6c47aa8ee\") " pod="openshift-image-registry/image-registry-679776947f-8st28" Jul 08 13:23:54.357109 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.357035 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/67cefd1d-608b-4db3-801e-4ca6c47aa8ee-trusted-ca\") pod \"image-registry-679776947f-8st28\" (UID: \"67cefd1d-608b-4db3-801e-4ca6c47aa8ee\") " pod="openshift-image-registry/image-registry-679776947f-8st28" Jul 08 13:23:54.357109 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.357047 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/67cefd1d-608b-4db3-801e-4ca6c47aa8ee-registry-certificates\") pod \"image-registry-679776947f-8st28\" (UID: \"67cefd1d-608b-4db3-801e-4ca6c47aa8ee\") " pod="openshift-image-registry/image-registry-679776947f-8st28" Jul 08 13:23:54.360379 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.360349 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/faf0a62e-56a2-449c-9aed-2859b1bfe9bc-metrics-tls\") pod \"dns-default-gv29f\" (UID: \"faf0a62e-56a2-449c-9aed-2859b1bfe9bc\") " pod="openshift-dns/dns-default-gv29f" Jul 08 13:23:54.360379 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.360376 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/67cefd1d-608b-4db3-801e-4ca6c47aa8ee-installation-pull-secrets\") pod \"image-registry-679776947f-8st28\" (UID: \"67cefd1d-608b-4db3-801e-4ca6c47aa8ee\") " pod="openshift-image-registry/image-registry-679776947f-8st28" Jul 08 13:23:54.360558 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.360540 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1111874c-8d49-43f3-a84a-fa214f432735-cert\") pod \"ingress-canary-wgz5x\" (UID: \"1111874c-8d49-43f3-a84a-fa214f432735\") " pod="openshift-ingress-canary/ingress-canary-wgz5x" Jul 08 13:23:54.360596 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.360570 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/67cefd1d-608b-4db3-801e-4ca6c47aa8ee-registry-tls\") pod \"image-registry-679776947f-8st28\" (UID: \"67cefd1d-608b-4db3-801e-4ca6c47aa8ee\") " pod="openshift-image-registry/image-registry-679776947f-8st28" Jul 08 13:23:54.360736 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.360720 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/67cefd1d-608b-4db3-801e-4ca6c47aa8ee-image-registry-private-configuration\") pod \"image-registry-679776947f-8st28\" (UID: \"67cefd1d-608b-4db3-801e-4ca6c47aa8ee\") " pod="openshift-image-registry/image-registry-679776947f-8st28" Jul 08 13:23:54.364063 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.364036 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjqlx\" (UniqueName: \"kubernetes.io/projected/faf0a62e-56a2-449c-9aed-2859b1bfe9bc-kube-api-access-vjqlx\") pod \"dns-default-gv29f\" (UID: \"faf0a62e-56a2-449c-9aed-2859b1bfe9bc\") " pod="openshift-dns/dns-default-gv29f" Jul 08 13:23:54.364495 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.364472 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsfxf\" (UniqueName: \"kubernetes.io/projected/1111874c-8d49-43f3-a84a-fa214f432735-kube-api-access-nsfxf\") pod \"ingress-canary-wgz5x\" (UID: \"1111874c-8d49-43f3-a84a-fa214f432735\") " pod="openshift-ingress-canary/ingress-canary-wgz5x" Jul 08 13:23:54.364678 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.364658 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lpp7\" (UniqueName: \"kubernetes.io/projected/67cefd1d-608b-4db3-801e-4ca6c47aa8ee-kube-api-access-9lpp7\") pod \"image-registry-679776947f-8st28\" (UID: \"67cefd1d-608b-4db3-801e-4ca6c47aa8ee\") " pod="openshift-image-registry/image-registry-679776947f-8st28" Jul 08 13:23:54.364728 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.364660 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/67cefd1d-608b-4db3-801e-4ca6c47aa8ee-bound-sa-token\") pod \"image-registry-679776947f-8st28\" (UID: \"67cefd1d-608b-4db3-801e-4ca6c47aa8ee\") " pod="openshift-image-registry/image-registry-679776947f-8st28" Jul 08 13:23:54.431735 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.431703 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-679776947f-8st28" Jul 08 13:23:54.443839 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.443817 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gv29f" Jul 08 13:23:54.463976 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.463954 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wgz5x" Jul 08 13:23:54.641760 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.641455 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gv29f"] Jul 08 13:23:54.644767 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.644741 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-679776947f-8st28"] Jul 08 13:23:54.645601 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:54.645531 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfaf0a62e_56a2_449c_9aed_2859b1bfe9bc.slice/crio-5d97e31797bcf54d1fd4ca2a32de9576fd11af1163ea788f53b340a9d360ea14 WatchSource:0}: Error finding container 5d97e31797bcf54d1fd4ca2a32de9576fd11af1163ea788f53b340a9d360ea14: Status 404 returned error can't find the container with id 5d97e31797bcf54d1fd4ca2a32de9576fd11af1163ea788f53b340a9d360ea14 Jul 08 13:23:54.647523 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:54.647503 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67cefd1d_608b_4db3_801e_4ca6c47aa8ee.slice/crio-753018ff9fea4bab4dbbfc442731cd6e06e1a886436a65bec2cef565287aeb11 WatchSource:0}: Error finding container 753018ff9fea4bab4dbbfc442731cd6e06e1a886436a65bec2cef565287aeb11: Status 404 returned error can't find the container with id 753018ff9fea4bab4dbbfc442731cd6e06e1a886436a65bec2cef565287aeb11 Jul 08 13:23:54.663163 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.663135 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wgz5x"] Jul 08 13:23:54.665347 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:54.665325 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1111874c_8d49_43f3_a84a_fa214f432735.slice/crio-9abf0ef3f0a18238f0dca62cc30454d749aa986eaa2faa8048adbde4e9e0ee36 WatchSource:0}: Error finding container 9abf0ef3f0a18238f0dca62cc30454d749aa986eaa2faa8048adbde4e9e0ee36: Status 404 returned error can't find the container with id 9abf0ef3f0a18238f0dca62cc30454d749aa986eaa2faa8048adbde4e9e0ee36 Jul 08 13:23:54.789731 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.789699 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tp2lh" Jul 08 13:23:54.789871 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.789854 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rm9f4" Jul 08 13:23:54.792632 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.792614 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 13:23:54.792729 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.792691 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 13:23:54.792729 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.792702 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 13:23:54.792729 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:54.792696 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hs9z4\"" Jul 08 13:23:55.047039 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:55.047004 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qmwvz" event={"ID":"ea6986a3-b0b4-4704-a47a-c991094c04f4","Type":"ContainerStarted","Data":"1690b0f09f634de303dc22cbe9cadde1d01d5d53ee9b30229e06bd3155a1e885"} Jul 08 13:23:55.048112 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:55.048070 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wgz5x" event={"ID":"1111874c-8d49-43f3-a84a-fa214f432735","Type":"ContainerStarted","Data":"9abf0ef3f0a18238f0dca62cc30454d749aa986eaa2faa8048adbde4e9e0ee36"} Jul 08 13:23:55.049475 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:55.049452 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-679776947f-8st28" event={"ID":"67cefd1d-608b-4db3-801e-4ca6c47aa8ee","Type":"ContainerStarted","Data":"e1cb35d1af0f3a3c9992f13edf0bbb168820ce612051fc313e46d8e6d68c2d75"} Jul 08 13:23:55.049591 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:55.049487 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-679776947f-8st28" event={"ID":"67cefd1d-608b-4db3-801e-4ca6c47aa8ee","Type":"ContainerStarted","Data":"753018ff9fea4bab4dbbfc442731cd6e06e1a886436a65bec2cef565287aeb11"} Jul 08 13:23:55.049591 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:55.049504 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-679776947f-8st28" Jul 08 13:23:55.050698 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:55.050677 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gv29f" event={"ID":"faf0a62e-56a2-449c-9aed-2859b1bfe9bc","Type":"ContainerStarted","Data":"5d97e31797bcf54d1fd4ca2a32de9576fd11af1163ea788f53b340a9d360ea14"} Jul 08 13:23:55.068068 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:55.068025 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-qmwvz" podStartSLOduration=12.456584016 podStartE2EDuration="44.068011595s" totalImagesPullingTime="31.611427579s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 13:23:11 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:23:55.066933868 +0000 UTC m=+43.863874536" watchObservedRunningTime="2026-07-08 13:23:55.068011595 +0000 UTC m=+43.864952252" Jul 08 13:23:55.092951 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:55.092896 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-679776947f-8st28" podStartSLOduration=66.09287878 podStartE2EDuration="1m6.09287878s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:22:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:23:55.087418934 +0000 UTC m=+43.884359593" watchObservedRunningTime="2026-07-08 13:23:55.09287878 +0000 UTC m=+43.889819437" Jul 08 13:23:55.790539 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:55.790509 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnv6f" Jul 08 13:23:55.794780 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:55.794754 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tktlz\"" Jul 08 13:23:55.795013 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:55.794988 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 13:23:58.064801 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:58.064759 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gv29f" event={"ID":"faf0a62e-56a2-449c-9aed-2859b1bfe9bc","Type":"ContainerStarted","Data":"129a7845ba8a697316994b974629618be1837e24a6b42fe2914c822bb7ef71d7"} Jul 08 13:23:58.065294 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:58.064841 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gv29f" event={"ID":"faf0a62e-56a2-449c-9aed-2859b1bfe9bc","Type":"ContainerStarted","Data":"a5bb8ed8a7c008cbdd188ff8bbae73fdb3305950a5cc49528909d400b5855c22"} Jul 08 13:23:58.065294 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:58.064894 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-gv29f" Jul 08 13:23:58.081609 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:58.081560 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-gv29f" podStartSLOduration=1.795815819 podStartE2EDuration="4.081544068s" totalImagesPullingTime="2.285728249s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:23:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:23:58.080153869 +0000 UTC m=+46.877094527" watchObservedRunningTime="2026-07-08 13:23:58.081544068 +0000 UTC m=+46.878484747" Jul 08 13:23:58.882719 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:58.882550 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-679776947f-8st28"] Jul 08 13:23:58.913829 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:58.913803 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-67596d5cf6-6tq7v"] Jul 08 13:23:58.928702 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:58.928672 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67596d5cf6-6tq7v" Jul 08 13:23:58.929944 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:58.929919 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-67596d5cf6-6tq7v"] Jul 08 13:23:59.068591 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:59.068552 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wgz5x" event={"ID":"1111874c-8d49-43f3-a84a-fa214f432735","Type":"ContainerStarted","Data":"6b51f91b841c78f1318dcf493057d79291657f7e570d326e3009f568fa1ea560"} Jul 08 13:23:59.083845 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:59.083801 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-wgz5x" podStartSLOduration=1.460950708 podStartE2EDuration="5.083785372s" totalImagesPullingTime="3.622834664s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:23:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:23:59.082819923 +0000 UTC m=+47.879760579" watchObservedRunningTime="2026-07-08 13:23:59.083785372 +0000 UTC m=+47.880726039" Jul 08 13:23:59.094404 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:59.094378 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a5eefa2d-4b2e-4d78-9eff-086d7ec4ca2f-bound-sa-token\") pod \"image-registry-67596d5cf6-6tq7v\" (UID: \"a5eefa2d-4b2e-4d78-9eff-086d7ec4ca2f\") " pod="openshift-image-registry/image-registry-67596d5cf6-6tq7v" Jul 08 13:23:59.094530 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:59.094409 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ff2v\" (UniqueName: \"kubernetes.io/projected/a5eefa2d-4b2e-4d78-9eff-086d7ec4ca2f-kube-api-access-5ff2v\") pod \"image-registry-67596d5cf6-6tq7v\" (UID: \"a5eefa2d-4b2e-4d78-9eff-086d7ec4ca2f\") " pod="openshift-image-registry/image-registry-67596d5cf6-6tq7v" Jul 08 13:23:59.094530 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:59.094439 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a5eefa2d-4b2e-4d78-9eff-086d7ec4ca2f-image-registry-private-configuration\") pod \"image-registry-67596d5cf6-6tq7v\" (UID: \"a5eefa2d-4b2e-4d78-9eff-086d7ec4ca2f\") " pod="openshift-image-registry/image-registry-67596d5cf6-6tq7v" Jul 08 13:23:59.094651 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:59.094538 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a5eefa2d-4b2e-4d78-9eff-086d7ec4ca2f-registry-certificates\") pod \"image-registry-67596d5cf6-6tq7v\" (UID: \"a5eefa2d-4b2e-4d78-9eff-086d7ec4ca2f\") " pod="openshift-image-registry/image-registry-67596d5cf6-6tq7v" Jul 08 13:23:59.094651 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:59.094589 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a5eefa2d-4b2e-4d78-9eff-086d7ec4ca2f-ca-trust-extracted\") pod \"image-registry-67596d5cf6-6tq7v\" (UID: \"a5eefa2d-4b2e-4d78-9eff-086d7ec4ca2f\") " pod="openshift-image-registry/image-registry-67596d5cf6-6tq7v" Jul 08 13:23:59.094651 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:59.094615 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a5eefa2d-4b2e-4d78-9eff-086d7ec4ca2f-trusted-ca\") pod \"image-registry-67596d5cf6-6tq7v\" (UID: \"a5eefa2d-4b2e-4d78-9eff-086d7ec4ca2f\") " pod="openshift-image-registry/image-registry-67596d5cf6-6tq7v" Jul 08 13:23:59.094651 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:59.094641 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a5eefa2d-4b2e-4d78-9eff-086d7ec4ca2f-installation-pull-secrets\") pod \"image-registry-67596d5cf6-6tq7v\" (UID: \"a5eefa2d-4b2e-4d78-9eff-086d7ec4ca2f\") " pod="openshift-image-registry/image-registry-67596d5cf6-6tq7v" Jul 08 13:23:59.094840 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:59.094676 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a5eefa2d-4b2e-4d78-9eff-086d7ec4ca2f-registry-tls\") pod \"image-registry-67596d5cf6-6tq7v\" (UID: \"a5eefa2d-4b2e-4d78-9eff-086d7ec4ca2f\") " pod="openshift-image-registry/image-registry-67596d5cf6-6tq7v" Jul 08 13:23:59.195610 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:59.195521 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ff2v\" (UniqueName: \"kubernetes.io/projected/a5eefa2d-4b2e-4d78-9eff-086d7ec4ca2f-kube-api-access-5ff2v\") pod \"image-registry-67596d5cf6-6tq7v\" (UID: \"a5eefa2d-4b2e-4d78-9eff-086d7ec4ca2f\") " pod="openshift-image-registry/image-registry-67596d5cf6-6tq7v" Jul 08 13:23:59.195610 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:59.195572 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a5eefa2d-4b2e-4d78-9eff-086d7ec4ca2f-image-registry-private-configuration\") pod \"image-registry-67596d5cf6-6tq7v\" (UID: \"a5eefa2d-4b2e-4d78-9eff-086d7ec4ca2f\") " pod="openshift-image-registry/image-registry-67596d5cf6-6tq7v" Jul 08 13:23:59.195846 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:59.195634 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a5eefa2d-4b2e-4d78-9eff-086d7ec4ca2f-registry-certificates\") pod \"image-registry-67596d5cf6-6tq7v\" (UID: \"a5eefa2d-4b2e-4d78-9eff-086d7ec4ca2f\") " pod="openshift-image-registry/image-registry-67596d5cf6-6tq7v" Jul 08 13:23:59.195846 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:59.195659 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a5eefa2d-4b2e-4d78-9eff-086d7ec4ca2f-ca-trust-extracted\") pod \"image-registry-67596d5cf6-6tq7v\" (UID: \"a5eefa2d-4b2e-4d78-9eff-086d7ec4ca2f\") " pod="openshift-image-registry/image-registry-67596d5cf6-6tq7v" Jul 08 13:23:59.195846 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:59.195676 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a5eefa2d-4b2e-4d78-9eff-086d7ec4ca2f-trusted-ca\") pod \"image-registry-67596d5cf6-6tq7v\" (UID: \"a5eefa2d-4b2e-4d78-9eff-086d7ec4ca2f\") " pod="openshift-image-registry/image-registry-67596d5cf6-6tq7v" Jul 08 13:23:59.195846 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:59.195700 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a5eefa2d-4b2e-4d78-9eff-086d7ec4ca2f-installation-pull-secrets\") pod \"image-registry-67596d5cf6-6tq7v\" (UID: \"a5eefa2d-4b2e-4d78-9eff-086d7ec4ca2f\") " pod="openshift-image-registry/image-registry-67596d5cf6-6tq7v" Jul 08 13:23:59.195846 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:59.195728 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a5eefa2d-4b2e-4d78-9eff-086d7ec4ca2f-registry-tls\") pod \"image-registry-67596d5cf6-6tq7v\" (UID: \"a5eefa2d-4b2e-4d78-9eff-086d7ec4ca2f\") " pod="openshift-image-registry/image-registry-67596d5cf6-6tq7v" Jul 08 13:23:59.196698 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:59.196187 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a5eefa2d-4b2e-4d78-9eff-086d7ec4ca2f-ca-trust-extracted\") pod \"image-registry-67596d5cf6-6tq7v\" (UID: \"a5eefa2d-4b2e-4d78-9eff-086d7ec4ca2f\") " pod="openshift-image-registry/image-registry-67596d5cf6-6tq7v" Jul 08 13:23:59.196698 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:59.196363 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a5eefa2d-4b2e-4d78-9eff-086d7ec4ca2f-bound-sa-token\") pod \"image-registry-67596d5cf6-6tq7v\" (UID: \"a5eefa2d-4b2e-4d78-9eff-086d7ec4ca2f\") " pod="openshift-image-registry/image-registry-67596d5cf6-6tq7v" Jul 08 13:23:59.196880 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:59.196756 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a5eefa2d-4b2e-4d78-9eff-086d7ec4ca2f-registry-certificates\") pod \"image-registry-67596d5cf6-6tq7v\" (UID: \"a5eefa2d-4b2e-4d78-9eff-086d7ec4ca2f\") " pod="openshift-image-registry/image-registry-67596d5cf6-6tq7v" Jul 08 13:23:59.196938 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:59.196917 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a5eefa2d-4b2e-4d78-9eff-086d7ec4ca2f-trusted-ca\") pod \"image-registry-67596d5cf6-6tq7v\" (UID: \"a5eefa2d-4b2e-4d78-9eff-086d7ec4ca2f\") " pod="openshift-image-registry/image-registry-67596d5cf6-6tq7v" Jul 08 13:23:59.199901 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:59.199874 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a5eefa2d-4b2e-4d78-9eff-086d7ec4ca2f-image-registry-private-configuration\") pod \"image-registry-67596d5cf6-6tq7v\" (UID: \"a5eefa2d-4b2e-4d78-9eff-086d7ec4ca2f\") " pod="openshift-image-registry/image-registry-67596d5cf6-6tq7v" Jul 08 13:23:59.199989 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:59.199895 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a5eefa2d-4b2e-4d78-9eff-086d7ec4ca2f-registry-tls\") pod \"image-registry-67596d5cf6-6tq7v\" (UID: \"a5eefa2d-4b2e-4d78-9eff-086d7ec4ca2f\") " pod="openshift-image-registry/image-registry-67596d5cf6-6tq7v" Jul 08 13:23:59.199989 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:59.199942 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a5eefa2d-4b2e-4d78-9eff-086d7ec4ca2f-installation-pull-secrets\") pod \"image-registry-67596d5cf6-6tq7v\" (UID: \"a5eefa2d-4b2e-4d78-9eff-086d7ec4ca2f\") " pod="openshift-image-registry/image-registry-67596d5cf6-6tq7v" Jul 08 13:23:59.204035 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:59.204013 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ff2v\" (UniqueName: \"kubernetes.io/projected/a5eefa2d-4b2e-4d78-9eff-086d7ec4ca2f-kube-api-access-5ff2v\") pod \"image-registry-67596d5cf6-6tq7v\" (UID: \"a5eefa2d-4b2e-4d78-9eff-086d7ec4ca2f\") " pod="openshift-image-registry/image-registry-67596d5cf6-6tq7v" Jul 08 13:23:59.204144 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:59.204052 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a5eefa2d-4b2e-4d78-9eff-086d7ec4ca2f-bound-sa-token\") pod \"image-registry-67596d5cf6-6tq7v\" (UID: \"a5eefa2d-4b2e-4d78-9eff-086d7ec4ca2f\") " pod="openshift-image-registry/image-registry-67596d5cf6-6tq7v" Jul 08 13:23:59.238035 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:59.238009 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67596d5cf6-6tq7v" Jul 08 13:23:59.401269 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:23:59.400940 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-67596d5cf6-6tq7v"] Jul 08 13:23:59.404176 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:23:59.404147 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5eefa2d_4b2e_4d78_9eff_086d7ec4ca2f.slice/crio-b43521f2042b529b19323e85fcf849505374d698196b62320993b4e18b4ebb1c WatchSource:0}: Error finding container b43521f2042b529b19323e85fcf849505374d698196b62320993b4e18b4ebb1c: Status 404 returned error can't find the container with id b43521f2042b529b19323e85fcf849505374d698196b62320993b4e18b4ebb1c Jul 08 13:24:00.072442 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:00.072400 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67596d5cf6-6tq7v" event={"ID":"a5eefa2d-4b2e-4d78-9eff-086d7ec4ca2f","Type":"ContainerStarted","Data":"30335b808bbb2c8de29c02161f4a651ca254e7e214bb7878243d253abd2f5171"} Jul 08 13:24:00.073070 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:00.072460 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67596d5cf6-6tq7v" event={"ID":"a5eefa2d-4b2e-4d78-9eff-086d7ec4ca2f","Type":"ContainerStarted","Data":"b43521f2042b529b19323e85fcf849505374d698196b62320993b4e18b4ebb1c"} Jul 08 13:24:00.073070 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:00.072576 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-67596d5cf6-6tq7v" Jul 08 13:24:00.090434 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:00.090386 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-67596d5cf6-6tq7v" podStartSLOduration=2.090371578 podStartE2EDuration="2.090371578s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:23:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:24:00.089876586 +0000 UTC m=+48.886817243" watchObservedRunningTime="2026-07-08 13:24:00.090371578 +0000 UTC m=+48.887312238" Jul 08 13:24:02.228551 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:02.228515 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-v8x9r"] Jul 08 13:24:02.239994 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:02.239961 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-v8x9r" Jul 08 13:24:02.243660 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:02.243638 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 08 13:24:02.243771 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:02.243661 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-kvhmk\"" Jul 08 13:24:02.244305 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:02.244286 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 08 13:24:02.244419 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:02.244310 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 08 13:24:02.244419 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:02.244317 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 08 13:24:02.265601 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:02.265577 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-v8x9r"] Jul 08 13:24:02.320491 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:02.320452 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f54b4d08-6fd7-40fc-b9d9-5776291ff5ad-crio-socket\") pod \"insights-runtime-extractor-v8x9r\" (UID: \"f54b4d08-6fd7-40fc-b9d9-5776291ff5ad\") " pod="openshift-insights/insights-runtime-extractor-v8x9r" Jul 08 13:24:02.320491 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:02.320497 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvlkp\" (UniqueName: \"kubernetes.io/projected/f54b4d08-6fd7-40fc-b9d9-5776291ff5ad-kube-api-access-xvlkp\") pod \"insights-runtime-extractor-v8x9r\" (UID: \"f54b4d08-6fd7-40fc-b9d9-5776291ff5ad\") " pod="openshift-insights/insights-runtime-extractor-v8x9r" Jul 08 13:24:02.320732 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:02.320550 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f54b4d08-6fd7-40fc-b9d9-5776291ff5ad-data-volume\") pod \"insights-runtime-extractor-v8x9r\" (UID: \"f54b4d08-6fd7-40fc-b9d9-5776291ff5ad\") " pod="openshift-insights/insights-runtime-extractor-v8x9r" Jul 08 13:24:02.320732 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:02.320582 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f54b4d08-6fd7-40fc-b9d9-5776291ff5ad-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v8x9r\" (UID: \"f54b4d08-6fd7-40fc-b9d9-5776291ff5ad\") " pod="openshift-insights/insights-runtime-extractor-v8x9r" Jul 08 13:24:02.320732 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:02.320637 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f54b4d08-6fd7-40fc-b9d9-5776291ff5ad-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v8x9r\" (UID: \"f54b4d08-6fd7-40fc-b9d9-5776291ff5ad\") " pod="openshift-insights/insights-runtime-extractor-v8x9r" Jul 08 13:24:02.421135 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:02.421067 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f54b4d08-6fd7-40fc-b9d9-5776291ff5ad-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v8x9r\" (UID: \"f54b4d08-6fd7-40fc-b9d9-5776291ff5ad\") " pod="openshift-insights/insights-runtime-extractor-v8x9r" Jul 08 13:24:02.421312 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:02.421148 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f54b4d08-6fd7-40fc-b9d9-5776291ff5ad-crio-socket\") pod \"insights-runtime-extractor-v8x9r\" (UID: \"f54b4d08-6fd7-40fc-b9d9-5776291ff5ad\") " pod="openshift-insights/insights-runtime-extractor-v8x9r" Jul 08 13:24:02.421312 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:02.421181 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvlkp\" (UniqueName: \"kubernetes.io/projected/f54b4d08-6fd7-40fc-b9d9-5776291ff5ad-kube-api-access-xvlkp\") pod \"insights-runtime-extractor-v8x9r\" (UID: \"f54b4d08-6fd7-40fc-b9d9-5776291ff5ad\") " pod="openshift-insights/insights-runtime-extractor-v8x9r" Jul 08 13:24:02.421312 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:02.421230 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f54b4d08-6fd7-40fc-b9d9-5776291ff5ad-data-volume\") pod \"insights-runtime-extractor-v8x9r\" (UID: \"f54b4d08-6fd7-40fc-b9d9-5776291ff5ad\") " pod="openshift-insights/insights-runtime-extractor-v8x9r" Jul 08 13:24:02.421312 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:24:02.421232 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 13:24:02.421312 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:02.421275 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f54b4d08-6fd7-40fc-b9d9-5776291ff5ad-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v8x9r\" (UID: \"f54b4d08-6fd7-40fc-b9d9-5776291ff5ad\") " pod="openshift-insights/insights-runtime-extractor-v8x9r" Jul 08 13:24:02.421312 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:24:02.421310 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f54b4d08-6fd7-40fc-b9d9-5776291ff5ad-insights-runtime-extractor-tls podName:f54b4d08-6fd7-40fc-b9d9-5776291ff5ad nodeName:}" failed. No retries permitted until 2026-07-08 13:24:02.92129191 +0000 UTC m=+51.718232544 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f54b4d08-6fd7-40fc-b9d9-5776291ff5ad-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v8x9r" (UID: "f54b4d08-6fd7-40fc-b9d9-5776291ff5ad") : secret "insights-runtime-extractor-tls" not found Jul 08 13:24:02.421580 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:24:02.421334 2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 13:24:02.421580 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:02.421394 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f54b4d08-6fd7-40fc-b9d9-5776291ff5ad-crio-socket\") pod \"insights-runtime-extractor-v8x9r\" (UID: \"f54b4d08-6fd7-40fc-b9d9-5776291ff5ad\") " pod="openshift-insights/insights-runtime-extractor-v8x9r" Jul 08 13:24:02.421580 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:24:02.421411 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f54b4d08-6fd7-40fc-b9d9-5776291ff5ad-kube-rbac-proxy-cm podName:f54b4d08-6fd7-40fc-b9d9-5776291ff5ad nodeName:}" failed. No retries permitted until 2026-07-08 13:24:02.921395577 +0000 UTC m=+51.718336246 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f54b4d08-6fd7-40fc-b9d9-5776291ff5ad-kube-rbac-proxy-cm") pod "insights-runtime-extractor-v8x9r" (UID: "f54b4d08-6fd7-40fc-b9d9-5776291ff5ad") : configmap "kube-rbac-proxy" not found Jul 08 13:24:02.421689 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:02.421609 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f54b4d08-6fd7-40fc-b9d9-5776291ff5ad-data-volume\") pod \"insights-runtime-extractor-v8x9r\" (UID: \"f54b4d08-6fd7-40fc-b9d9-5776291ff5ad\") " pod="openshift-insights/insights-runtime-extractor-v8x9r" Jul 08 13:24:02.432954 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:02.432921 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvlkp\" (UniqueName: \"kubernetes.io/projected/f54b4d08-6fd7-40fc-b9d9-5776291ff5ad-kube-api-access-xvlkp\") pod \"insights-runtime-extractor-v8x9r\" (UID: \"f54b4d08-6fd7-40fc-b9d9-5776291ff5ad\") " pod="openshift-insights/insights-runtime-extractor-v8x9r" Jul 08 13:24:02.925783 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:02.925745 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f54b4d08-6fd7-40fc-b9d9-5776291ff5ad-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v8x9r\" (UID: \"f54b4d08-6fd7-40fc-b9d9-5776291ff5ad\") " pod="openshift-insights/insights-runtime-extractor-v8x9r" Jul 08 13:24:02.925963 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:02.925844 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f54b4d08-6fd7-40fc-b9d9-5776291ff5ad-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v8x9r\" (UID: \"f54b4d08-6fd7-40fc-b9d9-5776291ff5ad\") " pod="openshift-insights/insights-runtime-extractor-v8x9r" Jul 08 13:24:02.925963 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:24:02.925908 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 13:24:02.926041 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:24:02.925974 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f54b4d08-6fd7-40fc-b9d9-5776291ff5ad-insights-runtime-extractor-tls podName:f54b4d08-6fd7-40fc-b9d9-5776291ff5ad nodeName:}" failed. No retries permitted until 2026-07-08 13:24:03.925959792 +0000 UTC m=+52.722900425 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f54b4d08-6fd7-40fc-b9d9-5776291ff5ad-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v8x9r" (UID: "f54b4d08-6fd7-40fc-b9d9-5776291ff5ad") : secret "insights-runtime-extractor-tls" not found Jul 08 13:24:02.926041 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:24:02.925915 2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 13:24:02.926141 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:24:02.926051 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f54b4d08-6fd7-40fc-b9d9-5776291ff5ad-kube-rbac-proxy-cm podName:f54b4d08-6fd7-40fc-b9d9-5776291ff5ad nodeName:}" failed. No retries permitted until 2026-07-08 13:24:03.926038534 +0000 UTC m=+52.722979172 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f54b4d08-6fd7-40fc-b9d9-5776291ff5ad-kube-rbac-proxy-cm") pod "insights-runtime-extractor-v8x9r" (UID: "f54b4d08-6fd7-40fc-b9d9-5776291ff5ad") : configmap "kube-rbac-proxy" not found Jul 08 13:24:03.683624 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:03.683593 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gv29f_faf0a62e-56a2-449c-9aed-2859b1bfe9bc/dns/0.log" Jul 08 13:24:03.883519 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:03.883495 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gv29f_faf0a62e-56a2-449c-9aed-2859b1bfe9bc/kube-rbac-proxy/0.log" Jul 08 13:24:03.933768 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:03.933684 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f54b4d08-6fd7-40fc-b9d9-5776291ff5ad-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v8x9r\" (UID: \"f54b4d08-6fd7-40fc-b9d9-5776291ff5ad\") " pod="openshift-insights/insights-runtime-extractor-v8x9r" Jul 08 13:24:03.933768 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:03.933728 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f54b4d08-6fd7-40fc-b9d9-5776291ff5ad-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v8x9r\" (UID: \"f54b4d08-6fd7-40fc-b9d9-5776291ff5ad\") " pod="openshift-insights/insights-runtime-extractor-v8x9r" Jul 08 13:24:03.933946 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:24:03.933823 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 13:24:03.933946 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:24:03.933882 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f54b4d08-6fd7-40fc-b9d9-5776291ff5ad-insights-runtime-extractor-tls podName:f54b4d08-6fd7-40fc-b9d9-5776291ff5ad nodeName:}" failed. No retries permitted until 2026-07-08 13:24:05.933861541 +0000 UTC m=+54.730802175 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f54b4d08-6fd7-40fc-b9d9-5776291ff5ad-insights-runtime-extractor-tls") pod "insights-runtime-extractor-v8x9r" (UID: "f54b4d08-6fd7-40fc-b9d9-5776291ff5ad") : secret "insights-runtime-extractor-tls" not found Jul 08 13:24:03.934737 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:03.934708 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f54b4d08-6fd7-40fc-b9d9-5776291ff5ad-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v8x9r\" (UID: \"f54b4d08-6fd7-40fc-b9d9-5776291ff5ad\") " pod="openshift-insights/insights-runtime-extractor-v8x9r" Jul 08 13:24:04.484626 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:04.484601 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zs2fw_d492fe73-0e0b-4b21-8ea1-2ab7cfcefe74/dns-node-resolver/0.log" Jul 08 13:24:04.684743 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:04.684712 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-67596d5cf6-6tq7v_a5eefa2d-4b2e-4d78-9eff-086d7ec4ca2f/registry/0.log" Jul 08 13:24:04.883251 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:04.883172 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-679776947f-8st28_67cefd1d-608b-4db3-801e-4ca6c47aa8ee/registry/0.log" Jul 08 13:24:05.483675 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:05.483652 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-pn22q_67e86263-6fde-4f3a-a775-9e29eb8cacef/node-ca/0.log" Jul 08 13:24:05.948297 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:05.948213 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f54b4d08-6fd7-40fc-b9d9-5776291ff5ad-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v8x9r\" (UID: \"f54b4d08-6fd7-40fc-b9d9-5776291ff5ad\") " pod="openshift-insights/insights-runtime-extractor-v8x9r" Jul 08 13:24:05.950831 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:05.950801 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f54b4d08-6fd7-40fc-b9d9-5776291ff5ad-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v8x9r\" (UID: \"f54b4d08-6fd7-40fc-b9d9-5776291ff5ad\") " pod="openshift-insights/insights-runtime-extractor-v8x9r" Jul 08 13:24:06.149704 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:06.149655 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-v8x9r" Jul 08 13:24:06.271260 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:06.271229 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-v8x9r"] Jul 08 13:24:06.283703 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:06.283539 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wgz5x_1111874c-8d49-43f3-a84a-fa214f432735/serve-healthcheck-canary/0.log" Jul 08 13:24:07.091612 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:07.091540 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v8x9r" event={"ID":"f54b4d08-6fd7-40fc-b9d9-5776291ff5ad","Type":"ContainerStarted","Data":"c1ff96ea030a8f98411dcb9766656e9c5bd8e9f868c4d77ba09779afb054712a"} Jul 08 13:24:08.071513 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:08.071477 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-gv29f" Jul 08 13:24:10.102829 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:10.102800 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v8x9r" event={"ID":"f54b4d08-6fd7-40fc-b9d9-5776291ff5ad","Type":"ContainerStarted","Data":"711da8caeb3a0bb453ab7297160a8630ae3d247fb3985e1917b6e4c271f53072"} Jul 08 13:24:11.107252 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:11.107174 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v8x9r" event={"ID":"f54b4d08-6fd7-40fc-b9d9-5776291ff5ad","Type":"ContainerStarted","Data":"04ffb15bb0dcb848692e68f43f85ae1a5f4ad7f049533ad94925e85cab3f9d06"} Jul 08 13:24:12.018466 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:12.018431 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sqlwq" Jul 08 13:24:14.120974 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:14.120937 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v8x9r" event={"ID":"f54b4d08-6fd7-40fc-b9d9-5776291ff5ad","Type":"ContainerStarted","Data":"2b099dbd0227d7e112d98345f32d98a6b7550addfe8ff08bce9c408e29ff00d2"} Jul 08 13:24:14.138690 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:14.138628 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-v8x9r" podStartSLOduration=5.159477816 podStartE2EDuration="12.138611335s" totalImagesPullingTime="6.979133519s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 13:24:02 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:24:14.137802713 +0000 UTC m=+62.934743369" watchObservedRunningTime="2026-07-08 13:24:14.138611335 +0000 UTC m=+62.935551988" Jul 08 13:24:15.313015 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:15.312980 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-pm5fs"] Jul 08 13:24:15.314962 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:15.314944 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pm5fs" Jul 08 13:24:15.317356 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:15.317335 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 08 13:24:15.317356 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:15.317349 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 08 13:24:15.317814 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:15.317796 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-tlsmn\"" Jul 08 13:24:15.318646 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:15.318627 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 08 13:24:15.318758 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:15.318665 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 08 13:24:15.318758 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:15.318687 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 08 13:24:15.330167 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:15.330143 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-pm5fs"] Jul 08 13:24:15.331021 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:15.331005 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-dtkrv"] Jul 08 13:24:15.335367 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:15.335347 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dtkrv" Jul 08 13:24:15.338308 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:15.338287 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 08 13:24:15.338416 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:15.338291 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 08 13:24:15.338416 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:15.338325 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 08 13:24:15.338416 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:15.338344 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-dfn6w\"" Jul 08 13:24:15.425426 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:15.425398 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/36940947-ce58-4160-b7dd-474b1d58a790-node-exporter-tls\") pod \"node-exporter-dtkrv\" (UID: \"36940947-ce58-4160-b7dd-474b1d58a790\") " pod="openshift-monitoring/node-exporter-dtkrv" Jul 08 13:24:15.425426 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:15.425431 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/36940947-ce58-4160-b7dd-474b1d58a790-metrics-client-ca\") pod \"node-exporter-dtkrv\" (UID: \"36940947-ce58-4160-b7dd-474b1d58a790\") " pod="openshift-monitoring/node-exporter-dtkrv" Jul 08 13:24:15.425619 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:15.425450 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1fa12140-dbf1-49c1-afd1-29f85f64d4f5-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-pm5fs\" (UID: \"1fa12140-dbf1-49c1-afd1-29f85f64d4f5\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pm5fs" Jul 08 13:24:15.425619 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:15.425477 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vzf8\" (UniqueName: \"kubernetes.io/projected/1fa12140-dbf1-49c1-afd1-29f85f64d4f5-kube-api-access-9vzf8\") pod \"openshift-state-metrics-54cf96dfcf-pm5fs\" (UID: \"1fa12140-dbf1-49c1-afd1-29f85f64d4f5\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pm5fs" Jul 08 13:24:15.425619 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:15.425523 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/36940947-ce58-4160-b7dd-474b1d58a790-sys\") pod \"node-exporter-dtkrv\" (UID: \"36940947-ce58-4160-b7dd-474b1d58a790\") " pod="openshift-monitoring/node-exporter-dtkrv" Jul 08 13:24:15.425619 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:15.425562 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/36940947-ce58-4160-b7dd-474b1d58a790-root\") pod \"node-exporter-dtkrv\" (UID: \"36940947-ce58-4160-b7dd-474b1d58a790\") " pod="openshift-monitoring/node-exporter-dtkrv" Jul 08 13:24:15.425619 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:15.425580 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/36940947-ce58-4160-b7dd-474b1d58a790-node-exporter-wtmp\") pod \"node-exporter-dtkrv\" (UID: \"36940947-ce58-4160-b7dd-474b1d58a790\") " pod="openshift-monitoring/node-exporter-dtkrv" Jul 08 13:24:15.425619 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:15.425605 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1fa12140-dbf1-49c1-afd1-29f85f64d4f5-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-pm5fs\" (UID: \"1fa12140-dbf1-49c1-afd1-29f85f64d4f5\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pm5fs" Jul 08 13:24:15.425795 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:15.425627 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1fa12140-dbf1-49c1-afd1-29f85f64d4f5-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-pm5fs\" (UID: \"1fa12140-dbf1-49c1-afd1-29f85f64d4f5\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pm5fs" Jul 08 13:24:15.425795 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:15.425689 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/36940947-ce58-4160-b7dd-474b1d58a790-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dtkrv\" (UID: \"36940947-ce58-4160-b7dd-474b1d58a790\") " pod="openshift-monitoring/node-exporter-dtkrv" Jul 08 13:24:15.425795 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:15.425735 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tv99\" (UniqueName: \"kubernetes.io/projected/36940947-ce58-4160-b7dd-474b1d58a790-kube-api-access-5tv99\") pod \"node-exporter-dtkrv\" (UID: \"36940947-ce58-4160-b7dd-474b1d58a790\") " pod="openshift-monitoring/node-exporter-dtkrv" Jul 08 13:24:15.425795 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:15.425780 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/36940947-ce58-4160-b7dd-474b1d58a790-node-exporter-textfile\") pod \"node-exporter-dtkrv\" (UID: \"36940947-ce58-4160-b7dd-474b1d58a790\") " pod="openshift-monitoring/node-exporter-dtkrv" Jul 08 13:24:15.425917 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:15.425796 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/36940947-ce58-4160-b7dd-474b1d58a790-node-exporter-accelerators-collector-config\") pod \"node-exporter-dtkrv\" (UID: \"36940947-ce58-4160-b7dd-474b1d58a790\") " pod="openshift-monitoring/node-exporter-dtkrv" Jul 08 13:24:15.526134 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:15.526101 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vzf8\" (UniqueName: \"kubernetes.io/projected/1fa12140-dbf1-49c1-afd1-29f85f64d4f5-kube-api-access-9vzf8\") pod \"openshift-state-metrics-54cf96dfcf-pm5fs\" (UID: \"1fa12140-dbf1-49c1-afd1-29f85f64d4f5\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pm5fs" Jul 08 13:24:15.526134 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:15.526136 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/36940947-ce58-4160-b7dd-474b1d58a790-sys\") pod \"node-exporter-dtkrv\" (UID: \"36940947-ce58-4160-b7dd-474b1d58a790\") " pod="openshift-monitoring/node-exporter-dtkrv" Jul 08 13:24:15.526373 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:15.526157 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/36940947-ce58-4160-b7dd-474b1d58a790-root\") pod \"node-exporter-dtkrv\" (UID: \"36940947-ce58-4160-b7dd-474b1d58a790\") " pod="openshift-monitoring/node-exporter-dtkrv" Jul 08 13:24:15.526373 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:15.526173 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/36940947-ce58-4160-b7dd-474b1d58a790-node-exporter-wtmp\") pod \"node-exporter-dtkrv\" (UID: \"36940947-ce58-4160-b7dd-474b1d58a790\") " pod="openshift-monitoring/node-exporter-dtkrv" Jul 08 13:24:15.526373 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:15.526206 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1fa12140-dbf1-49c1-afd1-29f85f64d4f5-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-pm5fs\" (UID: \"1fa12140-dbf1-49c1-afd1-29f85f64d4f5\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pm5fs" Jul 08 13:24:15.526373 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:15.526213 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/36940947-ce58-4160-b7dd-474b1d58a790-sys\") pod \"node-exporter-dtkrv\" (UID: \"36940947-ce58-4160-b7dd-474b1d58a790\") " pod="openshift-monitoring/node-exporter-dtkrv" Jul 08 13:24:15.526373 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:15.526232 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1fa12140-dbf1-49c1-afd1-29f85f64d4f5-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-pm5fs\" (UID: \"1fa12140-dbf1-49c1-afd1-29f85f64d4f5\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pm5fs" Jul 08 13:24:15.526373 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:15.526237 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/36940947-ce58-4160-b7dd-474b1d58a790-root\") pod \"node-exporter-dtkrv\" (UID: \"36940947-ce58-4160-b7dd-474b1d58a790\") " pod="openshift-monitoring/node-exporter-dtkrv" Jul 08 13:24:15.526373 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:15.526267 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/36940947-ce58-4160-b7dd-474b1d58a790-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dtkrv\" (UID: \"36940947-ce58-4160-b7dd-474b1d58a790\") " pod="openshift-monitoring/node-exporter-dtkrv" Jul 08 13:24:15.526373 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:15.526304 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tv99\" (UniqueName: \"kubernetes.io/projected/36940947-ce58-4160-b7dd-474b1d58a790-kube-api-access-5tv99\") pod \"node-exporter-dtkrv\" (UID: \"36940947-ce58-4160-b7dd-474b1d58a790\") " pod="openshift-monitoring/node-exporter-dtkrv" Jul 08 13:24:15.526373 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:15.526344 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/36940947-ce58-4160-b7dd-474b1d58a790-node-exporter-textfile\") pod \"node-exporter-dtkrv\" (UID: \"36940947-ce58-4160-b7dd-474b1d58a790\") " pod="openshift-monitoring/node-exporter-dtkrv" Jul 08 13:24:15.526373 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:15.526370 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/36940947-ce58-4160-b7dd-474b1d58a790-node-exporter-accelerators-collector-config\") pod \"node-exporter-dtkrv\" (UID: \"36940947-ce58-4160-b7dd-474b1d58a790\") " pod="openshift-monitoring/node-exporter-dtkrv" Jul 08 13:24:15.526373 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:15.526376 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/36940947-ce58-4160-b7dd-474b1d58a790-node-exporter-wtmp\") pod \"node-exporter-dtkrv\" (UID: \"36940947-ce58-4160-b7dd-474b1d58a790\") " pod="openshift-monitoring/node-exporter-dtkrv" Jul 08 13:24:15.526949 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:15.526521 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/36940947-ce58-4160-b7dd-474b1d58a790-node-exporter-tls\") pod \"node-exporter-dtkrv\" (UID: \"36940947-ce58-4160-b7dd-474b1d58a790\") " pod="openshift-monitoring/node-exporter-dtkrv" Jul 08 13:24:15.526949 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:15.526560 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/36940947-ce58-4160-b7dd-474b1d58a790-metrics-client-ca\") pod \"node-exporter-dtkrv\" (UID: \"36940947-ce58-4160-b7dd-474b1d58a790\") " pod="openshift-monitoring/node-exporter-dtkrv" Jul 08 13:24:15.526949 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:15.526629 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1fa12140-dbf1-49c1-afd1-29f85f64d4f5-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-pm5fs\" (UID: \"1fa12140-dbf1-49c1-afd1-29f85f64d4f5\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pm5fs" Jul 08 13:24:15.526949 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:24:15.526717 2567 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 08 13:24:15.526949 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:15.526777 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/36940947-ce58-4160-b7dd-474b1d58a790-node-exporter-textfile\") pod \"node-exporter-dtkrv\" (UID: \"36940947-ce58-4160-b7dd-474b1d58a790\") " pod="openshift-monitoring/node-exporter-dtkrv" Jul 08 13:24:15.526949 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:24:15.526807 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1fa12140-dbf1-49c1-afd1-29f85f64d4f5-openshift-state-metrics-tls podName:1fa12140-dbf1-49c1-afd1-29f85f64d4f5 nodeName:}" failed. No retries permitted until 2026-07-08 13:24:16.026789586 +0000 UTC m=+64.823730219 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/1fa12140-dbf1-49c1-afd1-29f85f64d4f5-openshift-state-metrics-tls") pod "openshift-state-metrics-54cf96dfcf-pm5fs" (UID: "1fa12140-dbf1-49c1-afd1-29f85f64d4f5") : secret "openshift-state-metrics-tls" not found Jul 08 13:24:15.527250 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:15.526981 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1fa12140-dbf1-49c1-afd1-29f85f64d4f5-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-pm5fs\" (UID: \"1fa12140-dbf1-49c1-afd1-29f85f64d4f5\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pm5fs" Jul 08 13:24:15.527250 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:15.527100 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/36940947-ce58-4160-b7dd-474b1d58a790-node-exporter-accelerators-collector-config\") pod \"node-exporter-dtkrv\" (UID: \"36940947-ce58-4160-b7dd-474b1d58a790\") " pod="openshift-monitoring/node-exporter-dtkrv" Jul 08 13:24:15.527250 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:15.527233 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/36940947-ce58-4160-b7dd-474b1d58a790-metrics-client-ca\") pod \"node-exporter-dtkrv\" (UID: \"36940947-ce58-4160-b7dd-474b1d58a790\") " pod="openshift-monitoring/node-exporter-dtkrv" Jul 08 13:24:15.529347 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:15.529327 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/36940947-ce58-4160-b7dd-474b1d58a790-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dtkrv\" (UID: \"36940947-ce58-4160-b7dd-474b1d58a790\") " pod="openshift-monitoring/node-exporter-dtkrv" Jul 08 13:24:15.529461 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:15.529403 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/36940947-ce58-4160-b7dd-474b1d58a790-node-exporter-tls\") pod \"node-exporter-dtkrv\" (UID: \"36940947-ce58-4160-b7dd-474b1d58a790\") " pod="openshift-monitoring/node-exporter-dtkrv" Jul 08 13:24:15.529461 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:15.529428 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1fa12140-dbf1-49c1-afd1-29f85f64d4f5-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-pm5fs\" (UID: \"1fa12140-dbf1-49c1-afd1-29f85f64d4f5\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pm5fs" Jul 08 13:24:15.534265 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:15.534240 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tv99\" (UniqueName: \"kubernetes.io/projected/36940947-ce58-4160-b7dd-474b1d58a790-kube-api-access-5tv99\") pod \"node-exporter-dtkrv\" (UID: \"36940947-ce58-4160-b7dd-474b1d58a790\") " pod="openshift-monitoring/node-exporter-dtkrv" Jul 08 13:24:15.537347 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:15.537329 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vzf8\" (UniqueName: \"kubernetes.io/projected/1fa12140-dbf1-49c1-afd1-29f85f64d4f5-kube-api-access-9vzf8\") pod \"openshift-state-metrics-54cf96dfcf-pm5fs\" (UID: \"1fa12140-dbf1-49c1-afd1-29f85f64d4f5\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pm5fs" Jul 08 13:24:15.644512 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:15.644432 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dtkrv" Jul 08 13:24:15.654000 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:24:15.653973 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36940947_ce58_4160_b7dd_474b1d58a790.slice/crio-026b40b62f1dea88d8fab0b03e4ec8fabb092c88434a3ba06c3f266931c200e9 WatchSource:0}: Error finding container 026b40b62f1dea88d8fab0b03e4ec8fabb092c88434a3ba06c3f266931c200e9: Status 404 returned error can't find the container with id 026b40b62f1dea88d8fab0b03e4ec8fabb092c88434a3ba06c3f266931c200e9 Jul 08 13:24:16.030350 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.030318 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1fa12140-dbf1-49c1-afd1-29f85f64d4f5-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-pm5fs\" (UID: \"1fa12140-dbf1-49c1-afd1-29f85f64d4f5\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pm5fs" Jul 08 13:24:16.032611 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.032579 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1fa12140-dbf1-49c1-afd1-29f85f64d4f5-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-pm5fs\" (UID: \"1fa12140-dbf1-49c1-afd1-29f85f64d4f5\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pm5fs" Jul 08 13:24:16.127966 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.127932 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dtkrv" event={"ID":"36940947-ce58-4160-b7dd-474b1d58a790","Type":"ContainerStarted","Data":"026b40b62f1dea88d8fab0b03e4ec8fabb092c88434a3ba06c3f266931c200e9"} Jul 08 13:24:16.223946 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.223909 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pm5fs" Jul 08 13:24:16.367331 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.367295 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-pm5fs"] Jul 08 13:24:16.369364 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:24:16.369339 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fa12140_dbf1_49c1_afd1_29f85f64d4f5.slice/crio-511f64f401269646a92028d3c3c999cdf31573ccd2f9dcaeda884b14094f14d4 WatchSource:0}: Error finding container 511f64f401269646a92028d3c3c999cdf31573ccd2f9dcaeda884b14094f14d4: Status 404 returned error can't find the container with id 511f64f401269646a92028d3c3c999cdf31573ccd2f9dcaeda884b14094f14d4 Jul 08 13:24:16.395054 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.395027 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 13:24:16.397987 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.397971 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 13:24:16.400621 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.400438 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 08 13:24:16.400621 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.400612 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-856fp\"" Jul 08 13:24:16.401146 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.400805 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 08 13:24:16.401146 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.400948 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 08 13:24:16.401146 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.400981 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 08 13:24:16.401146 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.401013 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 08 13:24:16.401146 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.401020 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 08 13:24:16.401146 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.401104 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 08 13:24:16.401503 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.401312 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 08 13:24:16.401503 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.401361 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 08 13:24:16.412561 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.412540 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 13:24:16.433884 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.433860 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a3381953-6131-477f-be6d-92ef7dc333a7-metrics-certs\") pod \"network-metrics-daemon-cnv6f\" (UID: \"a3381953-6131-477f-be6d-92ef7dc333a7\") " pod="openshift-multus/network-metrics-daemon-cnv6f" Jul 08 13:24:16.433971 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.433908 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6dafca81-f493-4b6d-ada8-dd9a6043baed-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6dafca81-f493-4b6d-ada8-dd9a6043baed\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 13:24:16.433971 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.433937 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6dafca81-f493-4b6d-ada8-dd9a6043baed-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6dafca81-f493-4b6d-ada8-dd9a6043baed\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 13:24:16.433971 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.433963 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6dafca81-f493-4b6d-ada8-dd9a6043baed-web-config\") pod \"alertmanager-main-0\" (UID: \"6dafca81-f493-4b6d-ada8-dd9a6043baed\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 13:24:16.434157 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.433998 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6dafca81-f493-4b6d-ada8-dd9a6043baed-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6dafca81-f493-4b6d-ada8-dd9a6043baed\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 13:24:16.434157 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.434021 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6dafca81-f493-4b6d-ada8-dd9a6043baed-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6dafca81-f493-4b6d-ada8-dd9a6043baed\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 13:24:16.434157 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.434044 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kl5l\" (UniqueName: \"kubernetes.io/projected/6dafca81-f493-4b6d-ada8-dd9a6043baed-kube-api-access-4kl5l\") pod \"alertmanager-main-0\" (UID: \"6dafca81-f493-4b6d-ada8-dd9a6043baed\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 13:24:16.434157 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.434109 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6dafca81-f493-4b6d-ada8-dd9a6043baed-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6dafca81-f493-4b6d-ada8-dd9a6043baed\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 13:24:16.434323 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.434236 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6dafca81-f493-4b6d-ada8-dd9a6043baed-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6dafca81-f493-4b6d-ada8-dd9a6043baed\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 13:24:16.434323 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.434296 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6dafca81-f493-4b6d-ada8-dd9a6043baed-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6dafca81-f493-4b6d-ada8-dd9a6043baed\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 13:24:16.434428 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.434346 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6dafca81-f493-4b6d-ada8-dd9a6043baed-config-volume\") pod \"alertmanager-main-0\" (UID: \"6dafca81-f493-4b6d-ada8-dd9a6043baed\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 13:24:16.434428 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.434394 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6dafca81-f493-4b6d-ada8-dd9a6043baed-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6dafca81-f493-4b6d-ada8-dd9a6043baed\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 13:24:16.434507 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.434435 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6dafca81-f493-4b6d-ada8-dd9a6043baed-config-out\") pod \"alertmanager-main-0\" (UID: \"6dafca81-f493-4b6d-ada8-dd9a6043baed\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 13:24:16.434507 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.434460 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6dafca81-f493-4b6d-ada8-dd9a6043baed-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6dafca81-f493-4b6d-ada8-dd9a6043baed\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 13:24:16.437243 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.437220 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 13:24:16.446808 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.446785 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a3381953-6131-477f-be6d-92ef7dc333a7-metrics-certs\") pod \"network-metrics-daemon-cnv6f\" (UID: \"a3381953-6131-477f-be6d-92ef7dc333a7\") " pod="openshift-multus/network-metrics-daemon-cnv6f" Jul 08 13:24:16.504208 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.504188 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tktlz\"" Jul 08 13:24:16.512240 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.512214 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cnv6f" Jul 08 13:24:16.536361 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.536325 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6dafca81-f493-4b6d-ada8-dd9a6043baed-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6dafca81-f493-4b6d-ada8-dd9a6043baed\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 13:24:16.536526 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.536370 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6dafca81-f493-4b6d-ada8-dd9a6043baed-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6dafca81-f493-4b6d-ada8-dd9a6043baed\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 13:24:16.536526 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.536401 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6dafca81-f493-4b6d-ada8-dd9a6043baed-web-config\") pod \"alertmanager-main-0\" (UID: \"6dafca81-f493-4b6d-ada8-dd9a6043baed\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 13:24:16.536526 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.536454 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6dafca81-f493-4b6d-ada8-dd9a6043baed-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6dafca81-f493-4b6d-ada8-dd9a6043baed\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 13:24:16.536526 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.536479 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6dafca81-f493-4b6d-ada8-dd9a6043baed-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6dafca81-f493-4b6d-ada8-dd9a6043baed\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 13:24:16.536526 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.536506 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kl5l\" (UniqueName: \"kubernetes.io/projected/6dafca81-f493-4b6d-ada8-dd9a6043baed-kube-api-access-4kl5l\") pod \"alertmanager-main-0\" (UID: \"6dafca81-f493-4b6d-ada8-dd9a6043baed\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 13:24:16.536906 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.536547 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f93cae56-ca8c-4c37-866c-51c79f237448-original-pull-secret\") pod \"global-pull-secret-syncer-tp2lh\" (UID: \"f93cae56-ca8c-4c37-866c-51c79f237448\") " pod="kube-system/global-pull-secret-syncer-tp2lh" Jul 08 13:24:16.536906 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.536574 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6dafca81-f493-4b6d-ada8-dd9a6043baed-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6dafca81-f493-4b6d-ada8-dd9a6043baed\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 13:24:16.536906 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.536636 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6dafca81-f493-4b6d-ada8-dd9a6043baed-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6dafca81-f493-4b6d-ada8-dd9a6043baed\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 13:24:16.536906 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.536671 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6dafca81-f493-4b6d-ada8-dd9a6043baed-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6dafca81-f493-4b6d-ada8-dd9a6043baed\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 13:24:16.536906 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.536696 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6dafca81-f493-4b6d-ada8-dd9a6043baed-config-volume\") pod \"alertmanager-main-0\" (UID: \"6dafca81-f493-4b6d-ada8-dd9a6043baed\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 13:24:16.536906 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.536726 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6dafca81-f493-4b6d-ada8-dd9a6043baed-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6dafca81-f493-4b6d-ada8-dd9a6043baed\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 13:24:16.536906 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.536763 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6dafca81-f493-4b6d-ada8-dd9a6043baed-config-out\") pod \"alertmanager-main-0\" (UID: \"6dafca81-f493-4b6d-ada8-dd9a6043baed\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 13:24:16.536906 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.536785 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6dafca81-f493-4b6d-ada8-dd9a6043baed-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6dafca81-f493-4b6d-ada8-dd9a6043baed\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 13:24:16.537324 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:24:16.536950 2567 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 08 13:24:16.537324 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:24:16.537012 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6dafca81-f493-4b6d-ada8-dd9a6043baed-secret-alertmanager-main-tls podName:6dafca81-f493-4b6d-ada8-dd9a6043baed nodeName:}" failed. No retries permitted until 2026-07-08 13:24:17.036991324 +0000 UTC m=+65.833931980 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/6dafca81-f493-4b6d-ada8-dd9a6043baed-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "6dafca81-f493-4b6d-ada8-dd9a6043baed") : secret "alertmanager-main-tls" not found Jul 08 13:24:16.537933 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.537902 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6dafca81-f493-4b6d-ada8-dd9a6043baed-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6dafca81-f493-4b6d-ada8-dd9a6043baed\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 13:24:16.538591 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:24:16.538458 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6dafca81-f493-4b6d-ada8-dd9a6043baed-alertmanager-trusted-ca-bundle podName:6dafca81-f493-4b6d-ada8-dd9a6043baed nodeName:}" failed. No retries permitted until 2026-07-08 13:24:17.03843844 +0000 UTC m=+65.835379077 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/6dafca81-f493-4b6d-ada8-dd9a6043baed-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "6dafca81-f493-4b6d-ada8-dd9a6043baed") : configmap references non-existent config key: ca-bundle.crt Jul 08 13:24:16.539486 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.539450 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6dafca81-f493-4b6d-ada8-dd9a6043baed-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6dafca81-f493-4b6d-ada8-dd9a6043baed\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 13:24:16.540857 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.540800 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 13:24:16.541871 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.541832 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6dafca81-f493-4b6d-ada8-dd9a6043baed-web-config\") pod \"alertmanager-main-0\" (UID: \"6dafca81-f493-4b6d-ada8-dd9a6043baed\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 13:24:16.542663 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.542577 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6dafca81-f493-4b6d-ada8-dd9a6043baed-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6dafca81-f493-4b6d-ada8-dd9a6043baed\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 13:24:16.542663 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.542623 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6dafca81-f493-4b6d-ada8-dd9a6043baed-config-volume\") pod \"alertmanager-main-0\" (UID: \"6dafca81-f493-4b6d-ada8-dd9a6043baed\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 13:24:16.543778 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.543710 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6dafca81-f493-4b6d-ada8-dd9a6043baed-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6dafca81-f493-4b6d-ada8-dd9a6043baed\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 13:24:16.543935 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.543869 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6dafca81-f493-4b6d-ada8-dd9a6043baed-config-out\") pod \"alertmanager-main-0\" (UID: \"6dafca81-f493-4b6d-ada8-dd9a6043baed\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 13:24:16.544744 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.544707 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6dafca81-f493-4b6d-ada8-dd9a6043baed-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6dafca81-f493-4b6d-ada8-dd9a6043baed\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 13:24:16.545000 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.544953 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6dafca81-f493-4b6d-ada8-dd9a6043baed-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6dafca81-f493-4b6d-ada8-dd9a6043baed\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 13:24:16.546197 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.546156 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6dafca81-f493-4b6d-ada8-dd9a6043baed-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6dafca81-f493-4b6d-ada8-dd9a6043baed\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 13:24:16.548655 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.548623 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kl5l\" (UniqueName: \"kubernetes.io/projected/6dafca81-f493-4b6d-ada8-dd9a6043baed-kube-api-access-4kl5l\") pod \"alertmanager-main-0\" (UID: \"6dafca81-f493-4b6d-ada8-dd9a6043baed\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 13:24:16.551135 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.551059 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f93cae56-ca8c-4c37-866c-51c79f237448-original-pull-secret\") pod \"global-pull-secret-syncer-tp2lh\" (UID: \"f93cae56-ca8c-4c37-866c-51c79f237448\") " pod="kube-system/global-pull-secret-syncer-tp2lh" Jul 08 13:24:16.637727 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.637694 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wm2f\" (UniqueName: \"kubernetes.io/projected/b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5-kube-api-access-6wm2f\") pod \"network-check-target-rm9f4\" (UID: \"b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5\") " pod="openshift-network-diagnostics/network-check-target-rm9f4" Jul 08 13:24:16.640359 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.640334 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 13:24:16.651097 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.651053 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cnv6f"] Jul 08 13:24:16.651271 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.651256 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 13:24:16.661613 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.661588 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wm2f\" (UniqueName: \"kubernetes.io/projected/b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5-kube-api-access-6wm2f\") pod \"network-check-target-rm9f4\" (UID: \"b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5\") " pod="openshift-network-diagnostics/network-check-target-rm9f4" Jul 08 13:24:16.699373 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.699335 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tp2lh" Jul 08 13:24:16.706417 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.706393 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hs9z4\"" Jul 08 13:24:16.714796 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.714770 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rm9f4" Jul 08 13:24:16.764959 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:24:16.764918 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3381953_6131_477f_be6d_92ef7dc333a7.slice/crio-c10ed80c4ab141e063658071c3827ba6de7d63f57c8b5c52eeac84e514f55a22 WatchSource:0}: Error finding container c10ed80c4ab141e063658071c3827ba6de7d63f57c8b5c52eeac84e514f55a22: Status 404 returned error can't find the container with id c10ed80c4ab141e063658071c3827ba6de7d63f57c8b5c52eeac84e514f55a22 Jul 08 13:24:16.899720 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.899690 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tp2lh"] Jul 08 13:24:16.902123 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:24:16.902090 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf93cae56_ca8c_4c37_866c_51c79f237448.slice/crio-580c2394f8bdda7747a20b49a8ae69c49fa06e6ba2cfb691addbb70497009386 WatchSource:0}: Error finding container 580c2394f8bdda7747a20b49a8ae69c49fa06e6ba2cfb691addbb70497009386: Status 404 returned error can't find the container with id 580c2394f8bdda7747a20b49a8ae69c49fa06e6ba2cfb691addbb70497009386 Jul 08 13:24:16.914433 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:16.914408 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rm9f4"] Jul 08 13:24:16.917888 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:24:16.917851 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8c527d7_eaa0_43e5_8c21_9b6f0b03f9e5.slice/crio-f9d6e9fb96d3896eebed4b2b4af93afb87b66f1adf46dc4ad02732c358107055 WatchSource:0}: Error finding container f9d6e9fb96d3896eebed4b2b4af93afb87b66f1adf46dc4ad02732c358107055: Status 404 returned error can't find the container with id f9d6e9fb96d3896eebed4b2b4af93afb87b66f1adf46dc4ad02732c358107055 Jul 08 13:24:17.040564 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:17.040531 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6dafca81-f493-4b6d-ada8-dd9a6043baed-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6dafca81-f493-4b6d-ada8-dd9a6043baed\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 13:24:17.040736 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:17.040719 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6dafca81-f493-4b6d-ada8-dd9a6043baed-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6dafca81-f493-4b6d-ada8-dd9a6043baed\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 13:24:17.041358 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:17.041330 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6dafca81-f493-4b6d-ada8-dd9a6043baed-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6dafca81-f493-4b6d-ada8-dd9a6043baed\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 13:24:17.042959 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:17.042940 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6dafca81-f493-4b6d-ada8-dd9a6043baed-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6dafca81-f493-4b6d-ada8-dd9a6043baed\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 13:24:17.134516 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:17.134419 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pm5fs" event={"ID":"1fa12140-dbf1-49c1-afd1-29f85f64d4f5","Type":"ContainerStarted","Data":"b7fedee13a472812b1473f42a73b92cea63dfb925e2922bc26138721d76508b7"} Jul 08 13:24:17.134516 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:17.134496 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pm5fs" event={"ID":"1fa12140-dbf1-49c1-afd1-29f85f64d4f5","Type":"ContainerStarted","Data":"ff9e310a3d4b35e8e117edb7d691465d583a54c98b0e8cb5ea7c69a8240ce642"} Jul 08 13:24:17.134738 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:17.134525 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pm5fs" event={"ID":"1fa12140-dbf1-49c1-afd1-29f85f64d4f5","Type":"ContainerStarted","Data":"511f64f401269646a92028d3c3c999cdf31573ccd2f9dcaeda884b14094f14d4"} Jul 08 13:24:17.135668 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:17.135637 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cnv6f" event={"ID":"a3381953-6131-477f-be6d-92ef7dc333a7","Type":"ContainerStarted","Data":"c10ed80c4ab141e063658071c3827ba6de7d63f57c8b5c52eeac84e514f55a22"} Jul 08 13:24:17.137404 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:17.137347 2567 generic.go:356] "Generic (PLEG): container finished" podID="36940947-ce58-4160-b7dd-474b1d58a790" containerID="3342804a3fba3fb5ffb71944a16eac366834b4eb0b40161d9d748db6465316c2" exitCode=0 Jul 08 13:24:17.137521 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:17.137434 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dtkrv" event={"ID":"36940947-ce58-4160-b7dd-474b1d58a790","Type":"ContainerDied","Data":"3342804a3fba3fb5ffb71944a16eac366834b4eb0b40161d9d748db6465316c2"} Jul 08 13:24:17.138840 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:17.138812 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tp2lh" event={"ID":"f93cae56-ca8c-4c37-866c-51c79f237448","Type":"ContainerStarted","Data":"580c2394f8bdda7747a20b49a8ae69c49fa06e6ba2cfb691addbb70497009386"} Jul 08 13:24:17.140192 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:17.140164 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rm9f4" event={"ID":"b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5","Type":"ContainerStarted","Data":"f9d6e9fb96d3896eebed4b2b4af93afb87b66f1adf46dc4ad02732c358107055"} Jul 08 13:24:17.324144 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:17.324106 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 13:24:17.490270 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:17.490233 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 13:24:17.659533 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:24:17.659494 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dafca81_f493_4b6d_ada8_dd9a6043baed.slice/crio-7096248e67a9f7579a15b233d54f59f5ad1343428d71b6977400687fc7f7fa2b WatchSource:0}: Error finding container 7096248e67a9f7579a15b233d54f59f5ad1343428d71b6977400687fc7f7fa2b: Status 404 returned error can't find the container with id 7096248e67a9f7579a15b233d54f59f5ad1343428d71b6977400687fc7f7fa2b Jul 08 13:24:18.146288 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:18.146247 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pm5fs" event={"ID":"1fa12140-dbf1-49c1-afd1-29f85f64d4f5","Type":"ContainerStarted","Data":"2898bb5ad5a21edcb6364b9888fac207c8bcf077c084f1d1f69ef788e90f26c9"} Jul 08 13:24:18.147602 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:18.147566 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6dafca81-f493-4b6d-ada8-dd9a6043baed","Type":"ContainerStarted","Data":"7096248e67a9f7579a15b233d54f59f5ad1343428d71b6977400687fc7f7fa2b"} Jul 08 13:24:18.150036 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:18.150012 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dtkrv" event={"ID":"36940947-ce58-4160-b7dd-474b1d58a790","Type":"ContainerStarted","Data":"6c7a5888e259b4051c6aaf054c60d94bb184146ad66306648d0d0bcf48bd1491"} Jul 08 13:24:18.150167 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:18.150058 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dtkrv" event={"ID":"36940947-ce58-4160-b7dd-474b1d58a790","Type":"ContainerStarted","Data":"df699a74d0e02585642f9ab4fc53e5d7b5cd2a545319b979950374acc6996e78"} Jul 08 13:24:18.164059 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:18.164000 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-pm5fs" podStartSLOduration=1.992990814 podStartE2EDuration="3.163985156s" totalImagesPullingTime="1.170994342s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:24:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:24:18.163406119 +0000 UTC m=+66.960346787" watchObservedRunningTime="2026-07-08 13:24:18.163985156 +0000 UTC m=+66.960925812" Jul 08 13:24:18.185469 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:18.185409 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-dtkrv" podStartSLOduration=1.978002476 podStartE2EDuration="3.185387273s" totalImagesPullingTime="1.134355796s" totalInitContainerRuntime="73.029001ms" isStatefulPod=true podCreationTimestamp="2026-07-08 13:24:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:24:18.184300395 +0000 UTC m=+66.981241051" watchObservedRunningTime="2026-07-08 13:24:18.185387273 +0000 UTC m=+66.982327931" Jul 08 13:24:18.892978 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:18.892776 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-679776947f-8st28" Jul 08 13:24:19.617801 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:19.617765 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-7bb96f599b-drpkm"] Jul 08 13:24:19.620530 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:19.620509 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7bb96f599b-drpkm" Jul 08 13:24:19.624318 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:19.624295 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 08 13:24:19.624447 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:19.624361 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-71k8jesslafk7\"" Jul 08 13:24:19.624447 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:19.624398 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 08 13:24:19.624568 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:19.624295 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 08 13:24:19.624624 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:19.624578 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 08 13:24:19.624693 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:19.624672 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-85lcp\"" Jul 08 13:24:19.632275 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:19.632253 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7bb96f599b-drpkm"] Jul 08 13:24:19.664821 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:19.664789 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/061d2273-5038-4af2-a0e0-8659275d5095-metrics-server-audit-profiles\") pod \"metrics-server-7bb96f599b-drpkm\" (UID: \"061d2273-5038-4af2-a0e0-8659275d5095\") " pod="openshift-monitoring/metrics-server-7bb96f599b-drpkm" Jul 08 13:24:19.665003 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:19.664865 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/061d2273-5038-4af2-a0e0-8659275d5095-secret-metrics-server-tls\") pod \"metrics-server-7bb96f599b-drpkm\" (UID: \"061d2273-5038-4af2-a0e0-8659275d5095\") " pod="openshift-monitoring/metrics-server-7bb96f599b-drpkm" Jul 08 13:24:19.665003 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:19.664918 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r96fr\" (UniqueName: \"kubernetes.io/projected/061d2273-5038-4af2-a0e0-8659275d5095-kube-api-access-r96fr\") pod \"metrics-server-7bb96f599b-drpkm\" (UID: \"061d2273-5038-4af2-a0e0-8659275d5095\") " pod="openshift-monitoring/metrics-server-7bb96f599b-drpkm" Jul 08 13:24:19.665149 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:19.665007 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/061d2273-5038-4af2-a0e0-8659275d5095-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7bb96f599b-drpkm\" (UID: \"061d2273-5038-4af2-a0e0-8659275d5095\") " pod="openshift-monitoring/metrics-server-7bb96f599b-drpkm" Jul 08 13:24:19.665149 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:19.665050 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/061d2273-5038-4af2-a0e0-8659275d5095-client-ca-bundle\") pod \"metrics-server-7bb96f599b-drpkm\" (UID: \"061d2273-5038-4af2-a0e0-8659275d5095\") " pod="openshift-monitoring/metrics-server-7bb96f599b-drpkm" Jul 08 13:24:19.665149 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:19.665129 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/061d2273-5038-4af2-a0e0-8659275d5095-secret-metrics-server-client-certs\") pod \"metrics-server-7bb96f599b-drpkm\" (UID: \"061d2273-5038-4af2-a0e0-8659275d5095\") " pod="openshift-monitoring/metrics-server-7bb96f599b-drpkm" Jul 08 13:24:19.665294 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:19.665207 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/061d2273-5038-4af2-a0e0-8659275d5095-audit-log\") pod \"metrics-server-7bb96f599b-drpkm\" (UID: \"061d2273-5038-4af2-a0e0-8659275d5095\") " pod="openshift-monitoring/metrics-server-7bb96f599b-drpkm" Jul 08 13:24:19.766445 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:19.766403 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/061d2273-5038-4af2-a0e0-8659275d5095-secret-metrics-server-tls\") pod \"metrics-server-7bb96f599b-drpkm\" (UID: \"061d2273-5038-4af2-a0e0-8659275d5095\") " pod="openshift-monitoring/metrics-server-7bb96f599b-drpkm" Jul 08 13:24:19.766626 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:19.766469 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r96fr\" (UniqueName: \"kubernetes.io/projected/061d2273-5038-4af2-a0e0-8659275d5095-kube-api-access-r96fr\") pod \"metrics-server-7bb96f599b-drpkm\" (UID: \"061d2273-5038-4af2-a0e0-8659275d5095\") " pod="openshift-monitoring/metrics-server-7bb96f599b-drpkm" Jul 08 13:24:19.766626 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:19.766530 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/061d2273-5038-4af2-a0e0-8659275d5095-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7bb96f599b-drpkm\" (UID: \"061d2273-5038-4af2-a0e0-8659275d5095\") " pod="openshift-monitoring/metrics-server-7bb96f599b-drpkm" Jul 08 13:24:19.766626 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:19.766560 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/061d2273-5038-4af2-a0e0-8659275d5095-client-ca-bundle\") pod \"metrics-server-7bb96f599b-drpkm\" (UID: \"061d2273-5038-4af2-a0e0-8659275d5095\") " pod="openshift-monitoring/metrics-server-7bb96f599b-drpkm" Jul 08 13:24:19.766626 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:19.766590 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/061d2273-5038-4af2-a0e0-8659275d5095-secret-metrics-server-client-certs\") pod \"metrics-server-7bb96f599b-drpkm\" (UID: \"061d2273-5038-4af2-a0e0-8659275d5095\") " pod="openshift-monitoring/metrics-server-7bb96f599b-drpkm" Jul 08 13:24:19.766626 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:19.766608 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/061d2273-5038-4af2-a0e0-8659275d5095-audit-log\") pod \"metrics-server-7bb96f599b-drpkm\" (UID: \"061d2273-5038-4af2-a0e0-8659275d5095\") " pod="openshift-monitoring/metrics-server-7bb96f599b-drpkm" Jul 08 13:24:19.766880 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:19.766637 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/061d2273-5038-4af2-a0e0-8659275d5095-metrics-server-audit-profiles\") pod \"metrics-server-7bb96f599b-drpkm\" (UID: \"061d2273-5038-4af2-a0e0-8659275d5095\") " pod="openshift-monitoring/metrics-server-7bb96f599b-drpkm" Jul 08 13:24:19.767646 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:19.767574 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/061d2273-5038-4af2-a0e0-8659275d5095-audit-log\") pod \"metrics-server-7bb96f599b-drpkm\" (UID: \"061d2273-5038-4af2-a0e0-8659275d5095\") " pod="openshift-monitoring/metrics-server-7bb96f599b-drpkm" Jul 08 13:24:19.767646 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:19.767598 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/061d2273-5038-4af2-a0e0-8659275d5095-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7bb96f599b-drpkm\" (UID: \"061d2273-5038-4af2-a0e0-8659275d5095\") " pod="openshift-monitoring/metrics-server-7bb96f599b-drpkm" Jul 08 13:24:19.767900 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:19.767819 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/061d2273-5038-4af2-a0e0-8659275d5095-metrics-server-audit-profiles\") pod \"metrics-server-7bb96f599b-drpkm\" (UID: \"061d2273-5038-4af2-a0e0-8659275d5095\") " pod="openshift-monitoring/metrics-server-7bb96f599b-drpkm" Jul 08 13:24:19.769621 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:19.769573 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/061d2273-5038-4af2-a0e0-8659275d5095-secret-metrics-server-tls\") pod \"metrics-server-7bb96f599b-drpkm\" (UID: \"061d2273-5038-4af2-a0e0-8659275d5095\") " pod="openshift-monitoring/metrics-server-7bb96f599b-drpkm" Jul 08 13:24:19.769850 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:19.769822 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/061d2273-5038-4af2-a0e0-8659275d5095-client-ca-bundle\") pod \"metrics-server-7bb96f599b-drpkm\" (UID: \"061d2273-5038-4af2-a0e0-8659275d5095\") " pod="openshift-monitoring/metrics-server-7bb96f599b-drpkm" Jul 08 13:24:19.769948 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:19.769872 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/061d2273-5038-4af2-a0e0-8659275d5095-secret-metrics-server-client-certs\") pod \"metrics-server-7bb96f599b-drpkm\" (UID: \"061d2273-5038-4af2-a0e0-8659275d5095\") " pod="openshift-monitoring/metrics-server-7bb96f599b-drpkm" Jul 08 13:24:19.774718 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:19.774691 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r96fr\" (UniqueName: \"kubernetes.io/projected/061d2273-5038-4af2-a0e0-8659275d5095-kube-api-access-r96fr\") pod \"metrics-server-7bb96f599b-drpkm\" (UID: \"061d2273-5038-4af2-a0e0-8659275d5095\") " pod="openshift-monitoring/metrics-server-7bb96f599b-drpkm" Jul 08 13:24:19.931113 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:19.931005 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7bb96f599b-drpkm" Jul 08 13:24:20.159172 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:20.159131 2567 generic.go:356] "Generic (PLEG): container finished" podID="6dafca81-f493-4b6d-ada8-dd9a6043baed" containerID="4129c85bf96293b168fff2ed9054ae8f653eb151c63084ecc2ec6210adfdd74c" exitCode=0 Jul 08 13:24:20.159333 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:20.159231 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6dafca81-f493-4b6d-ada8-dd9a6043baed","Type":"ContainerDied","Data":"4129c85bf96293b168fff2ed9054ae8f653eb151c63084ecc2ec6210adfdd74c"} Jul 08 13:24:20.161369 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:20.161344 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cnv6f" event={"ID":"a3381953-6131-477f-be6d-92ef7dc333a7","Type":"ContainerStarted","Data":"c903b3f16dcac3648d84425412015d279283431c8f0461bb7c549b587bfdc97d"} Jul 08 13:24:20.161497 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:20.161392 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cnv6f" event={"ID":"a3381953-6131-477f-be6d-92ef7dc333a7","Type":"ContainerStarted","Data":"6b92156c83de8bce20df5422280306dd0ebda1255ee1c9b606a06199eceb9b67"} Jul 08 13:24:20.203816 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:20.203711 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-cnv6f" podStartSLOduration=66.836569043 podStartE2EDuration="1m9.203665445s" totalImagesPullingTime="2.367096402s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:23:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:24:20.202706298 +0000 UTC m=+68.999646955" watchObservedRunningTime="2026-07-08 13:24:20.203665445 +0000 UTC m=+69.000606102" Jul 08 13:24:20.531882 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:20.531847 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7bb96f599b-drpkm"] Jul 08 13:24:20.535614 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:24:20.535582 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod061d2273_5038_4af2_a0e0_8659275d5095.slice/crio-70a1493e31bdf3b26060d74ecb38f35d70b92f255d4ee997d578e2fca811ed52 WatchSource:0}: Error finding container 70a1493e31bdf3b26060d74ecb38f35d70b92f255d4ee997d578e2fca811ed52: Status 404 returned error can't find the container with id 70a1493e31bdf3b26060d74ecb38f35d70b92f255d4ee997d578e2fca811ed52 Jul 08 13:24:21.080277 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:21.080245 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-67596d5cf6-6tq7v" Jul 08 13:24:21.167683 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:21.167642 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7bb96f599b-drpkm" event={"ID":"061d2273-5038-4af2-a0e0-8659275d5095","Type":"ContainerStarted","Data":"70a1493e31bdf3b26060d74ecb38f35d70b92f255d4ee997d578e2fca811ed52"} Jul 08 13:24:21.169627 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:21.169597 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rm9f4" event={"ID":"b8c527d7-eaa0-43e5-8c21-9b6f0b03f9e5","Type":"ContainerStarted","Data":"0d96063047a045fc68a1673cdc25a91c9d70822b5c116775e72e755c7dbf986a"} Jul 08 13:24:21.169871 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:21.169849 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-rm9f4" Jul 08 13:24:21.188286 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:21.188230 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-rm9f4" podStartSLOduration=65.71263585 podStartE2EDuration="1m9.188212173s" totalImagesPullingTime="3.475576323s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:23:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:24:21.185951912 +0000 UTC m=+69.982892569" watchObservedRunningTime="2026-07-08 13:24:21.188212173 +0000 UTC m=+69.985152831" Jul 08 13:24:23.180194 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:23.180109 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tp2lh" event={"ID":"f93cae56-ca8c-4c37-866c-51c79f237448","Type":"ContainerStarted","Data":"3ef0b7c8ce774902eef5892961c8ceb01906be9556cae6f09bc3c410dbff09f1"} Jul 08 13:24:23.183989 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:23.183957 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6dafca81-f493-4b6d-ada8-dd9a6043baed","Type":"ContainerStarted","Data":"423eeb5d9ff97db87af79898a0e3f019791f7b188fdd5f456d11c10b01c4e0d8"} Jul 08 13:24:23.184132 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:23.184023 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6dafca81-f493-4b6d-ada8-dd9a6043baed","Type":"ContainerStarted","Data":"e82914750eda2f787a49d8789c69c7addb95366405cb5fb0fce8068ff1fdd710"} Jul 08 13:24:23.184132 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:23.184050 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6dafca81-f493-4b6d-ada8-dd9a6043baed","Type":"ContainerStarted","Data":"d6f6569dc23d60186d7d8055f9660be8a45576614747aa97bc3f1dfd93c19b25"} Jul 08 13:24:23.184132 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:23.184092 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6dafca81-f493-4b6d-ada8-dd9a6043baed","Type":"ContainerStarted","Data":"49a8b0617c1a21d90e89c2e18dc0bd7cc7aece41fd439a1d264f34058f771501"} Jul 08 13:24:23.184259 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:23.184132 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6dafca81-f493-4b6d-ada8-dd9a6043baed","Type":"ContainerStarted","Data":"d3a8b8d499ef971c45481df02823ae32794f168c47a606a0e59257a732a2f737"} Jul 08 13:24:23.196636 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:23.196577 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-tp2lh" podStartSLOduration=65.486128343 podStartE2EDuration="1m11.196559614s" totalImagesPullingTime="5.710431271s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 13:23:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:24:23.195324131 +0000 UTC m=+71.992264801" watchObservedRunningTime="2026-07-08 13:24:23.196559614 +0000 UTC m=+71.993500248" Jul 08 13:24:23.902868 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:23.902824 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-679776947f-8st28" podUID="67cefd1d-608b-4db3-801e-4ca6c47aa8ee" containerName="registry" containerID="cri-o://e1cb35d1af0f3a3c9992f13edf0bbb168820ce612051fc313e46d8e6d68c2d75" gracePeriod=30 Jul 08 13:24:24.189491 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:24.189459 2567 generic.go:356] "Generic (PLEG): container finished" podID="67cefd1d-608b-4db3-801e-4ca6c47aa8ee" containerID="e1cb35d1af0f3a3c9992f13edf0bbb168820ce612051fc313e46d8e6d68c2d75" exitCode=0 Jul 08 13:24:24.189919 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:24.189550 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-679776947f-8st28" event={"ID":"67cefd1d-608b-4db3-801e-4ca6c47aa8ee","Type":"ContainerDied","Data":"e1cb35d1af0f3a3c9992f13edf0bbb168820ce612051fc313e46d8e6d68c2d75"} Jul 08 13:24:24.191155 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:24.191132 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7bb96f599b-drpkm" event={"ID":"061d2273-5038-4af2-a0e0-8659275d5095","Type":"ContainerStarted","Data":"c59793d60a863140c1b0db799b4a701bcef07ba47af5eb948d50badf4ae958b8"} Jul 08 13:24:24.212114 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:24.212054 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-7bb96f599b-drpkm" podStartSLOduration=2.053721092 podStartE2EDuration="5.212040851s" totalImagesPullingTime="3.158319759s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:24:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:24:24.211378207 +0000 UTC m=+73.008318864" watchObservedRunningTime="2026-07-08 13:24:24.212040851 +0000 UTC m=+73.008981507" Jul 08 13:24:24.263900 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:24.263879 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-679776947f-8st28" Jul 08 13:24:24.407849 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:24.407814 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/67cefd1d-608b-4db3-801e-4ca6c47aa8ee-registry-tls\") pod \"67cefd1d-608b-4db3-801e-4ca6c47aa8ee\" (UID: \"67cefd1d-608b-4db3-801e-4ca6c47aa8ee\") " Jul 08 13:24:24.407849 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:24.407861 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/67cefd1d-608b-4db3-801e-4ca6c47aa8ee-installation-pull-secrets\") pod \"67cefd1d-608b-4db3-801e-4ca6c47aa8ee\" (UID: \"67cefd1d-608b-4db3-801e-4ca6c47aa8ee\") " Jul 08 13:24:24.408128 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:24.407887 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/67cefd1d-608b-4db3-801e-4ca6c47aa8ee-registry-certificates\") pod \"67cefd1d-608b-4db3-801e-4ca6c47aa8ee\" (UID: \"67cefd1d-608b-4db3-801e-4ca6c47aa8ee\") " Jul 08 13:24:24.408128 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:24.407920 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9lpp7\" (UniqueName: \"kubernetes.io/projected/67cefd1d-608b-4db3-801e-4ca6c47aa8ee-kube-api-access-9lpp7\") pod \"67cefd1d-608b-4db3-801e-4ca6c47aa8ee\" (UID: \"67cefd1d-608b-4db3-801e-4ca6c47aa8ee\") " Jul 08 13:24:24.408128 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:24.407965 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/67cefd1d-608b-4db3-801e-4ca6c47aa8ee-trusted-ca\") pod \"67cefd1d-608b-4db3-801e-4ca6c47aa8ee\" (UID: \"67cefd1d-608b-4db3-801e-4ca6c47aa8ee\") " Jul 08 13:24:24.408128 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:24.407989 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/67cefd1d-608b-4db3-801e-4ca6c47aa8ee-image-registry-private-configuration\") pod \"67cefd1d-608b-4db3-801e-4ca6c47aa8ee\" (UID: \"67cefd1d-608b-4db3-801e-4ca6c47aa8ee\") " Jul 08 13:24:24.408128 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:24.408013 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/67cefd1d-608b-4db3-801e-4ca6c47aa8ee-ca-trust-extracted\") pod \"67cefd1d-608b-4db3-801e-4ca6c47aa8ee\" (UID: \"67cefd1d-608b-4db3-801e-4ca6c47aa8ee\") " Jul 08 13:24:24.408128 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:24.408050 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/67cefd1d-608b-4db3-801e-4ca6c47aa8ee-bound-sa-token\") pod \"67cefd1d-608b-4db3-801e-4ca6c47aa8ee\" (UID: \"67cefd1d-608b-4db3-801e-4ca6c47aa8ee\") " Jul 08 13:24:24.408473 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:24.408444 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67cefd1d-608b-4db3-801e-4ca6c47aa8ee-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "67cefd1d-608b-4db3-801e-4ca6c47aa8ee" (UID: "67cefd1d-608b-4db3-801e-4ca6c47aa8ee"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 13:24:24.408752 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:24.408522 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67cefd1d-608b-4db3-801e-4ca6c47aa8ee-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "67cefd1d-608b-4db3-801e-4ca6c47aa8ee" (UID: "67cefd1d-608b-4db3-801e-4ca6c47aa8ee"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 13:24:24.410562 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:24.410537 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67cefd1d-608b-4db3-801e-4ca6c47aa8ee-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "67cefd1d-608b-4db3-801e-4ca6c47aa8ee" (UID: "67cefd1d-608b-4db3-801e-4ca6c47aa8ee"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 13:24:24.410716 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:24.410696 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67cefd1d-608b-4db3-801e-4ca6c47aa8ee-kube-api-access-9lpp7" (OuterVolumeSpecName: "kube-api-access-9lpp7") pod "67cefd1d-608b-4db3-801e-4ca6c47aa8ee" (UID: "67cefd1d-608b-4db3-801e-4ca6c47aa8ee"). InnerVolumeSpecName "kube-api-access-9lpp7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 13:24:24.410836 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:24.410809 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67cefd1d-608b-4db3-801e-4ca6c47aa8ee-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "67cefd1d-608b-4db3-801e-4ca6c47aa8ee" (UID: "67cefd1d-608b-4db3-801e-4ca6c47aa8ee"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 13:24:24.410925 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:24.410902 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67cefd1d-608b-4db3-801e-4ca6c47aa8ee-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "67cefd1d-608b-4db3-801e-4ca6c47aa8ee" (UID: "67cefd1d-608b-4db3-801e-4ca6c47aa8ee"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 13:24:24.410977 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:24.410916 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67cefd1d-608b-4db3-801e-4ca6c47aa8ee-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "67cefd1d-608b-4db3-801e-4ca6c47aa8ee" (UID: "67cefd1d-608b-4db3-801e-4ca6c47aa8ee"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 13:24:24.417242 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:24.417217 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67cefd1d-608b-4db3-801e-4ca6c47aa8ee-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "67cefd1d-608b-4db3-801e-4ca6c47aa8ee" (UID: "67cefd1d-608b-4db3-801e-4ca6c47aa8ee"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:24:24.509519 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:24.509444 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9lpp7\" (UniqueName: \"kubernetes.io/projected/67cefd1d-608b-4db3-801e-4ca6c47aa8ee-kube-api-access-9lpp7\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:24:24.509519 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:24.509473 2567 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/67cefd1d-608b-4db3-801e-4ca6c47aa8ee-trusted-ca\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:24:24.509519 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:24.509485 2567 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/67cefd1d-608b-4db3-801e-4ca6c47aa8ee-image-registry-private-configuration\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:24:24.509519 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:24.509495 2567 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/67cefd1d-608b-4db3-801e-4ca6c47aa8ee-ca-trust-extracted\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:24:24.509519 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:24.509504 2567 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/67cefd1d-608b-4db3-801e-4ca6c47aa8ee-bound-sa-token\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:24:24.509519 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:24.509512 2567 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/67cefd1d-608b-4db3-801e-4ca6c47aa8ee-registry-tls\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:24:24.509519 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:24.509520 2567 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/67cefd1d-608b-4db3-801e-4ca6c47aa8ee-installation-pull-secrets\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:24:24.509870 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:24.509527 2567 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/67cefd1d-608b-4db3-801e-4ca6c47aa8ee-registry-certificates\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:24:25.195647 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:25.195617 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-679776947f-8st28" Jul 08 13:24:25.195647 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:25.195634 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-679776947f-8st28" event={"ID":"67cefd1d-608b-4db3-801e-4ca6c47aa8ee","Type":"ContainerDied","Data":"753018ff9fea4bab4dbbfc442731cd6e06e1a886436a65bec2cef565287aeb11"} Jul 08 13:24:25.196157 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:25.195690 2567 scope.go:117] "RemoveContainer" containerID="e1cb35d1af0f3a3c9992f13edf0bbb168820ce612051fc313e46d8e6d68c2d75" Jul 08 13:24:25.199140 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:25.199108 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6dafca81-f493-4b6d-ada8-dd9a6043baed","Type":"ContainerStarted","Data":"f6c740c5b5b2073e389acb77e0c4a22516ce3335dae07e859a8523a7c3262a5a"} Jul 08 13:24:25.227094 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:25.226992 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.988087133 podStartE2EDuration="9.226969179s" totalImagesPullingTime="5.215400081s" totalInitContainerRuntime="23.481965ms" isStatefulPod=false podCreationTimestamp="2026-07-08 13:24:16 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:24:25.225564368 +0000 UTC m=+74.022505024" watchObservedRunningTime="2026-07-08 13:24:25.226969179 +0000 UTC m=+74.023909836" Jul 08 13:24:25.239759 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:25.239735 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-679776947f-8st28"] Jul 08 13:24:25.244687 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:25.244662 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-679776947f-8st28"] Jul 08 13:24:25.794814 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:25.794783 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67cefd1d-608b-4db3-801e-4ca6c47aa8ee" path="/var/lib/kubelet/pods/67cefd1d-608b-4db3-801e-4ca6c47aa8ee/volumes" Jul 08 13:24:39.932199 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:39.932028 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-7bb96f599b-drpkm" Jul 08 13:24:39.932199 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:39.932137 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-7bb96f599b-drpkm" Jul 08 13:24:52.177946 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:52.177907 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-rm9f4" Jul 08 13:24:59.937091 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:59.937041 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-7bb96f599b-drpkm" Jul 08 13:24:59.940973 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:24:59.940953 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-7bb96f599b-drpkm" Jul 08 13:26:07.341388 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:07.341307 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rhkrw"] Jul 08 13:26:07.341791 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:07.341583 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67cefd1d-608b-4db3-801e-4ca6c47aa8ee" containerName="registry" Jul 08 13:26:07.341791 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:07.341596 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="67cefd1d-608b-4db3-801e-4ca6c47aa8ee" containerName="registry" Jul 08 13:26:07.341791 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:07.341641 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="67cefd1d-608b-4db3-801e-4ca6c47aa8ee" containerName="registry" Jul 08 13:26:07.343609 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:07.343592 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rhkrw" Jul 08 13:26:07.346032 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:07.346007 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 08 13:26:07.346191 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:07.346165 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-mm72n\"" Jul 08 13:26:07.347182 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:07.347168 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 08 13:26:07.354470 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:07.354445 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rhkrw"] Jul 08 13:26:07.387371 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:07.387329 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zc4qq\" (UniqueName: \"kubernetes.io/projected/4ef226b7-ad2d-4b06-9efb-d13b9fae911d-kube-api-access-zc4qq\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rhkrw\" (UID: \"4ef226b7-ad2d-4b06-9efb-d13b9fae911d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rhkrw" Jul 08 13:26:07.387371 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:07.387377 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4ef226b7-ad2d-4b06-9efb-d13b9fae911d-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rhkrw\" (UID: \"4ef226b7-ad2d-4b06-9efb-d13b9fae911d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rhkrw" Jul 08 13:26:07.387591 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:07.387475 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4ef226b7-ad2d-4b06-9efb-d13b9fae911d-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rhkrw\" (UID: \"4ef226b7-ad2d-4b06-9efb-d13b9fae911d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rhkrw" Jul 08 13:26:07.488619 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:07.488585 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4ef226b7-ad2d-4b06-9efb-d13b9fae911d-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rhkrw\" (UID: \"4ef226b7-ad2d-4b06-9efb-d13b9fae911d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rhkrw" Jul 08 13:26:07.488711 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:07.488631 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zc4qq\" (UniqueName: \"kubernetes.io/projected/4ef226b7-ad2d-4b06-9efb-d13b9fae911d-kube-api-access-zc4qq\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rhkrw\" (UID: \"4ef226b7-ad2d-4b06-9efb-d13b9fae911d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rhkrw" Jul 08 13:26:07.488711 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:07.488657 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4ef226b7-ad2d-4b06-9efb-d13b9fae911d-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rhkrw\" (UID: \"4ef226b7-ad2d-4b06-9efb-d13b9fae911d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rhkrw" Jul 08 13:26:07.488972 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:07.488954 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4ef226b7-ad2d-4b06-9efb-d13b9fae911d-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rhkrw\" (UID: \"4ef226b7-ad2d-4b06-9efb-d13b9fae911d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rhkrw" Jul 08 13:26:07.489042 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:07.488995 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4ef226b7-ad2d-4b06-9efb-d13b9fae911d-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rhkrw\" (UID: \"4ef226b7-ad2d-4b06-9efb-d13b9fae911d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rhkrw" Jul 08 13:26:07.496708 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:07.496673 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zc4qq\" (UniqueName: \"kubernetes.io/projected/4ef226b7-ad2d-4b06-9efb-d13b9fae911d-kube-api-access-zc4qq\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rhkrw\" (UID: \"4ef226b7-ad2d-4b06-9efb-d13b9fae911d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rhkrw" Jul 08 13:26:07.652497 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:07.652383 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rhkrw" Jul 08 13:26:07.781530 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:07.781491 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rhkrw"] Jul 08 13:26:07.784334 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:26:07.784304 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ef226b7_ad2d_4b06_9efb_d13b9fae911d.slice/crio-39a28429b1456f28088d496a65b762625e4403f0f2cf8b82e65d87c1b07f8cbb WatchSource:0}: Error finding container 39a28429b1456f28088d496a65b762625e4403f0f2cf8b82e65d87c1b07f8cbb: Status 404 returned error can't find the container with id 39a28429b1456f28088d496a65b762625e4403f0f2cf8b82e65d87c1b07f8cbb Jul 08 13:26:08.552223 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:08.552186 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rhkrw" event={"ID":"4ef226b7-ad2d-4b06-9efb-d13b9fae911d","Type":"ContainerStarted","Data":"39a28429b1456f28088d496a65b762625e4403f0f2cf8b82e65d87c1b07f8cbb"} Jul 08 13:26:16.584789 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:16.584755 2567 generic.go:356] "Generic (PLEG): container finished" podID="4ef226b7-ad2d-4b06-9efb-d13b9fae911d" containerID="9b56c7bcd5b69f4cd96782074850b73d87768c5e5b698f0b72cfbe7c050a76ff" exitCode=0 Jul 08 13:26:16.585106 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:16.584806 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rhkrw" event={"ID":"4ef226b7-ad2d-4b06-9efb-d13b9fae911d","Type":"ContainerDied","Data":"9b56c7bcd5b69f4cd96782074850b73d87768c5e5b698f0b72cfbe7c050a76ff"} Jul 08 13:26:19.597582 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:19.597548 2567 generic.go:356] "Generic (PLEG): container finished" podID="4ef226b7-ad2d-4b06-9efb-d13b9fae911d" containerID="42de5744de6d3549a4679b66ba1dbfa2708dcf1d566a7722a8c784b0ec2c98a7" exitCode=0 Jul 08 13:26:19.598025 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:19.597616 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rhkrw" event={"ID":"4ef226b7-ad2d-4b06-9efb-d13b9fae911d","Type":"ContainerDied","Data":"42de5744de6d3549a4679b66ba1dbfa2708dcf1d566a7722a8c784b0ec2c98a7"} Jul 08 13:26:29.636741 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:29.636708 2567 generic.go:356] "Generic (PLEG): container finished" podID="4ef226b7-ad2d-4b06-9efb-d13b9fae911d" containerID="940b22a83d11c233541ac814e604bd466962d2465750b29fdb2ae99d791bf285" exitCode=0 Jul 08 13:26:29.637135 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:29.636781 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rhkrw" event={"ID":"4ef226b7-ad2d-4b06-9efb-d13b9fae911d","Type":"ContainerDied","Data":"940b22a83d11c233541ac814e604bd466962d2465750b29fdb2ae99d791bf285"} Jul 08 13:26:30.803452 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:30.803420 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rhkrw" Jul 08 13:26:30.882528 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:30.882494 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4ef226b7-ad2d-4b06-9efb-d13b9fae911d-util\") pod \"4ef226b7-ad2d-4b06-9efb-d13b9fae911d\" (UID: \"4ef226b7-ad2d-4b06-9efb-d13b9fae911d\") " Jul 08 13:26:30.882721 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:30.882566 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zc4qq\" (UniqueName: \"kubernetes.io/projected/4ef226b7-ad2d-4b06-9efb-d13b9fae911d-kube-api-access-zc4qq\") pod \"4ef226b7-ad2d-4b06-9efb-d13b9fae911d\" (UID: \"4ef226b7-ad2d-4b06-9efb-d13b9fae911d\") " Jul 08 13:26:30.882721 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:30.882586 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4ef226b7-ad2d-4b06-9efb-d13b9fae911d-bundle\") pod \"4ef226b7-ad2d-4b06-9efb-d13b9fae911d\" (UID: \"4ef226b7-ad2d-4b06-9efb-d13b9fae911d\") " Jul 08 13:26:30.883252 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:30.883224 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ef226b7-ad2d-4b06-9efb-d13b9fae911d-bundle" (OuterVolumeSpecName: "bundle") pod "4ef226b7-ad2d-4b06-9efb-d13b9fae911d" (UID: "4ef226b7-ad2d-4b06-9efb-d13b9fae911d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:26:30.884767 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:30.884743 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ef226b7-ad2d-4b06-9efb-d13b9fae911d-kube-api-access-zc4qq" (OuterVolumeSpecName: "kube-api-access-zc4qq") pod "4ef226b7-ad2d-4b06-9efb-d13b9fae911d" (UID: "4ef226b7-ad2d-4b06-9efb-d13b9fae911d"). InnerVolumeSpecName "kube-api-access-zc4qq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 13:26:30.887969 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:30.887945 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ef226b7-ad2d-4b06-9efb-d13b9fae911d-util" (OuterVolumeSpecName: "util") pod "4ef226b7-ad2d-4b06-9efb-d13b9fae911d" (UID: "4ef226b7-ad2d-4b06-9efb-d13b9fae911d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:26:30.983304 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:30.983267 2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4ef226b7-ad2d-4b06-9efb-d13b9fae911d-util\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:26:30.983304 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:30.983300 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zc4qq\" (UniqueName: \"kubernetes.io/projected/4ef226b7-ad2d-4b06-9efb-d13b9fae911d-kube-api-access-zc4qq\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:26:30.983497 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:30.983313 2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4ef226b7-ad2d-4b06-9efb-d13b9fae911d-bundle\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:26:31.645519 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:31.645483 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rhkrw" event={"ID":"4ef226b7-ad2d-4b06-9efb-d13b9fae911d","Type":"ContainerDied","Data":"39a28429b1456f28088d496a65b762625e4403f0f2cf8b82e65d87c1b07f8cbb"} Jul 08 13:26:31.645763 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:31.645556 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="39a28429b1456f28088d496a65b762625e4403f0f2cf8b82e65d87c1b07f8cbb" Jul 08 13:26:31.645763 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:31.645497 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rhkrw" Jul 08 13:26:35.355053 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:35.355018 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-kvqxp"] Jul 08 13:26:35.355463 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:35.355297 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ef226b7-ad2d-4b06-9efb-d13b9fae911d" containerName="util" Jul 08 13:26:35.355463 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:35.355309 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ef226b7-ad2d-4b06-9efb-d13b9fae911d" containerName="util" Jul 08 13:26:35.355463 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:35.355320 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ef226b7-ad2d-4b06-9efb-d13b9fae911d" containerName="pull" Jul 08 13:26:35.355463 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:35.355325 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ef226b7-ad2d-4b06-9efb-d13b9fae911d" containerName="pull" Jul 08 13:26:35.355463 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:35.355331 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ef226b7-ad2d-4b06-9efb-d13b9fae911d" containerName="extract" Jul 08 13:26:35.355463 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:35.355336 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ef226b7-ad2d-4b06-9efb-d13b9fae911d" containerName="extract" Jul 08 13:26:35.355463 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:35.355374 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ef226b7-ad2d-4b06-9efb-d13b9fae911d" containerName="extract" Jul 08 13:26:35.357069 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:35.357053 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-kvqxp" Jul 08 13:26:35.359583 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:35.359559 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 08 13:26:35.359695 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:35.359559 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 08 13:26:35.359695 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:35.359610 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-blqvd\"" Jul 08 13:26:35.373662 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:35.373641 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-kvqxp"] Jul 08 13:26:35.419775 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:35.419743 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f76eac41-b892-46b6-aede-7afbe091c102-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-kvqxp\" (UID: \"f76eac41-b892-46b6-aede-7afbe091c102\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-kvqxp" Jul 08 13:26:35.419899 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:35.419777 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfgnt\" (UniqueName: \"kubernetes.io/projected/f76eac41-b892-46b6-aede-7afbe091c102-kube-api-access-wfgnt\") pod \"cert-manager-operator-controller-manager-55c444bf86-kvqxp\" (UID: \"f76eac41-b892-46b6-aede-7afbe091c102\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-kvqxp" Jul 08 13:26:35.520579 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:35.520550 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f76eac41-b892-46b6-aede-7afbe091c102-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-kvqxp\" (UID: \"f76eac41-b892-46b6-aede-7afbe091c102\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-kvqxp" Jul 08 13:26:35.520714 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:35.520582 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfgnt\" (UniqueName: \"kubernetes.io/projected/f76eac41-b892-46b6-aede-7afbe091c102-kube-api-access-wfgnt\") pod \"cert-manager-operator-controller-manager-55c444bf86-kvqxp\" (UID: \"f76eac41-b892-46b6-aede-7afbe091c102\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-kvqxp" Jul 08 13:26:35.520959 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:35.520936 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f76eac41-b892-46b6-aede-7afbe091c102-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-kvqxp\" (UID: \"f76eac41-b892-46b6-aede-7afbe091c102\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-kvqxp" Jul 08 13:26:35.528855 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:35.528835 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfgnt\" (UniqueName: \"kubernetes.io/projected/f76eac41-b892-46b6-aede-7afbe091c102-kube-api-access-wfgnt\") pod \"cert-manager-operator-controller-manager-55c444bf86-kvqxp\" (UID: \"f76eac41-b892-46b6-aede-7afbe091c102\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-kvqxp" Jul 08 13:26:35.665898 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:35.665832 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-kvqxp" Jul 08 13:26:35.801373 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:35.801217 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-kvqxp"] Jul 08 13:26:35.804125 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:26:35.804098 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf76eac41_b892_46b6_aede_7afbe091c102.slice/crio-984d7f4032fa1697af2afd8fcdd6c3406355ee53c2b4323b284165c235dde1f7 WatchSource:0}: Error finding container 984d7f4032fa1697af2afd8fcdd6c3406355ee53c2b4323b284165c235dde1f7: Status 404 returned error can't find the container with id 984d7f4032fa1697af2afd8fcdd6c3406355ee53c2b4323b284165c235dde1f7 Jul 08 13:26:36.664674 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:36.664633 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-kvqxp" event={"ID":"f76eac41-b892-46b6-aede-7afbe091c102","Type":"ContainerStarted","Data":"984d7f4032fa1697af2afd8fcdd6c3406355ee53c2b4323b284165c235dde1f7"} Jul 08 13:26:37.670224 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:37.670189 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-kvqxp" event={"ID":"f76eac41-b892-46b6-aede-7afbe091c102","Type":"ContainerStarted","Data":"8960431f4f01e1149dc87ceeef2e5e793bc98ac14dbefed4d388b0e2fef3d836"} Jul 08 13:26:37.692999 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:37.692931 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-kvqxp" podStartSLOduration=1.325667259 podStartE2EDuration="2.692909321s" totalImagesPullingTime="1.367242062s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:26:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:26:37.69083978 +0000 UTC m=+206.487780460" watchObservedRunningTime="2026-07-08 13:26:37.692909321 +0000 UTC m=+206.489849982" Jul 08 13:26:39.031795 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:39.031757 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4c8w4"] Jul 08 13:26:39.033911 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:39.033895 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4c8w4" Jul 08 13:26:39.036407 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:39.036386 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 08 13:26:39.036407 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:39.036404 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-mm72n\"" Jul 08 13:26:39.037695 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:39.037673 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 08 13:26:39.044793 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:39.044770 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4c8w4"] Jul 08 13:26:39.152271 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:39.152237 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1e5ebf37-a8e9-47e7-9a41-60a656cca21f-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4c8w4\" (UID: \"1e5ebf37-a8e9-47e7-9a41-60a656cca21f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4c8w4" Jul 08 13:26:39.152456 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:39.152285 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1e5ebf37-a8e9-47e7-9a41-60a656cca21f-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4c8w4\" (UID: \"1e5ebf37-a8e9-47e7-9a41-60a656cca21f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4c8w4" Jul 08 13:26:39.152456 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:39.152336 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pwps\" (UniqueName: \"kubernetes.io/projected/1e5ebf37-a8e9-47e7-9a41-60a656cca21f-kube-api-access-7pwps\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4c8w4\" (UID: \"1e5ebf37-a8e9-47e7-9a41-60a656cca21f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4c8w4" Jul 08 13:26:39.215363 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:39.215337 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-8dvll"] Jul 08 13:26:39.217383 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:39.217368 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-8dvll" Jul 08 13:26:39.220297 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:39.220275 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 08 13:26:39.220432 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:39.220334 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 08 13:26:39.220432 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:39.220409 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-6vr9r\"" Jul 08 13:26:39.233417 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:39.233398 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-8dvll"] Jul 08 13:26:39.253662 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:39.253640 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1e5ebf37-a8e9-47e7-9a41-60a656cca21f-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4c8w4\" (UID: \"1e5ebf37-a8e9-47e7-9a41-60a656cca21f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4c8w4" Jul 08 13:26:39.253782 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:39.253686 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1e5ebf37-a8e9-47e7-9a41-60a656cca21f-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4c8w4\" (UID: \"1e5ebf37-a8e9-47e7-9a41-60a656cca21f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4c8w4" Jul 08 13:26:39.253782 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:39.253717 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pwps\" (UniqueName: \"kubernetes.io/projected/1e5ebf37-a8e9-47e7-9a41-60a656cca21f-kube-api-access-7pwps\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4c8w4\" (UID: \"1e5ebf37-a8e9-47e7-9a41-60a656cca21f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4c8w4" Jul 08 13:26:39.254026 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:39.254005 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1e5ebf37-a8e9-47e7-9a41-60a656cca21f-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4c8w4\" (UID: \"1e5ebf37-a8e9-47e7-9a41-60a656cca21f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4c8w4" Jul 08 13:26:39.254061 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:39.254030 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1e5ebf37-a8e9-47e7-9a41-60a656cca21f-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4c8w4\" (UID: \"1e5ebf37-a8e9-47e7-9a41-60a656cca21f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4c8w4" Jul 08 13:26:39.261817 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:39.261789 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pwps\" (UniqueName: \"kubernetes.io/projected/1e5ebf37-a8e9-47e7-9a41-60a656cca21f-kube-api-access-7pwps\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4c8w4\" (UID: \"1e5ebf37-a8e9-47e7-9a41-60a656cca21f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4c8w4" Jul 08 13:26:39.342891 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:39.342814 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4c8w4" Jul 08 13:26:39.354053 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:39.354019 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7748c5c8-602f-4374-8f56-651d52fddc79-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-8dvll\" (UID: \"7748c5c8-602f-4374-8f56-651d52fddc79\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8dvll" Jul 08 13:26:39.354178 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:39.354094 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cthrr\" (UniqueName: \"kubernetes.io/projected/7748c5c8-602f-4374-8f56-651d52fddc79-kube-api-access-cthrr\") pod \"cert-manager-webhook-6486ffd997-8dvll\" (UID: \"7748c5c8-602f-4374-8f56-651d52fddc79\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8dvll" Jul 08 13:26:39.454526 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:39.454495 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7748c5c8-602f-4374-8f56-651d52fddc79-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-8dvll\" (UID: \"7748c5c8-602f-4374-8f56-651d52fddc79\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8dvll" Jul 08 13:26:39.454666 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:39.454554 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cthrr\" (UniqueName: \"kubernetes.io/projected/7748c5c8-602f-4374-8f56-651d52fddc79-kube-api-access-cthrr\") pod \"cert-manager-webhook-6486ffd997-8dvll\" (UID: \"7748c5c8-602f-4374-8f56-651d52fddc79\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8dvll" Jul 08 13:26:39.462478 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:39.462447 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7748c5c8-602f-4374-8f56-651d52fddc79-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-8dvll\" (UID: \"7748c5c8-602f-4374-8f56-651d52fddc79\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8dvll" Jul 08 13:26:39.462674 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:39.462654 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cthrr\" (UniqueName: \"kubernetes.io/projected/7748c5c8-602f-4374-8f56-651d52fddc79-kube-api-access-cthrr\") pod \"cert-manager-webhook-6486ffd997-8dvll\" (UID: \"7748c5c8-602f-4374-8f56-651d52fddc79\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8dvll" Jul 08 13:26:39.466932 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:39.466913 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4c8w4"] Jul 08 13:26:39.469610 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:26:39.469586 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e5ebf37_a8e9_47e7_9a41_60a656cca21f.slice/crio-8e9a73e05f63c51cb7660b68ee94c17e439c33e04ca646715e40e4ea3de62e2a WatchSource:0}: Error finding container 8e9a73e05f63c51cb7660b68ee94c17e439c33e04ca646715e40e4ea3de62e2a: Status 404 returned error can't find the container with id 8e9a73e05f63c51cb7660b68ee94c17e439c33e04ca646715e40e4ea3de62e2a Jul 08 13:26:39.540378 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:39.540347 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-8dvll" Jul 08 13:26:39.664697 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:39.664591 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-8dvll"] Jul 08 13:26:39.667128 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:26:39.667107 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7748c5c8_602f_4374_8f56_651d52fddc79.slice/crio-62a6e3af695e4c353ff66b2f87093364a801a38b157f643e7f47a5bf3f63046c WatchSource:0}: Error finding container 62a6e3af695e4c353ff66b2f87093364a801a38b157f643e7f47a5bf3f63046c: Status 404 returned error can't find the container with id 62a6e3af695e4c353ff66b2f87093364a801a38b157f643e7f47a5bf3f63046c Jul 08 13:26:39.679385 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:39.679355 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-8dvll" event={"ID":"7748c5c8-602f-4374-8f56-651d52fddc79","Type":"ContainerStarted","Data":"62a6e3af695e4c353ff66b2f87093364a801a38b157f643e7f47a5bf3f63046c"} Jul 08 13:26:39.680653 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:39.680627 2567 generic.go:356] "Generic (PLEG): container finished" podID="1e5ebf37-a8e9-47e7-9a41-60a656cca21f" containerID="3249170faa05deca07d932f201d2ed376c4beac948659d269482f5a1a02a58c9" exitCode=0 Jul 08 13:26:39.680735 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:39.680715 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4c8w4" event={"ID":"1e5ebf37-a8e9-47e7-9a41-60a656cca21f","Type":"ContainerDied","Data":"3249170faa05deca07d932f201d2ed376c4beac948659d269482f5a1a02a58c9"} Jul 08 13:26:39.680785 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:39.680760 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4c8w4" event={"ID":"1e5ebf37-a8e9-47e7-9a41-60a656cca21f","Type":"ContainerStarted","Data":"8e9a73e05f63c51cb7660b68ee94c17e439c33e04ca646715e40e4ea3de62e2a"} Jul 08 13:26:41.209955 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:41.209917 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-m8zdx"] Jul 08 13:26:41.216131 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:41.216109 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-m8zdx" Jul 08 13:26:41.219895 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:41.219866 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-khfc2\"" Jul 08 13:26:41.220313 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:41.220267 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-m8zdx"] Jul 08 13:26:41.371530 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:41.371492 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1383c69a-30e5-4f5b-a593-8aaeec480e8c-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-m8zdx\" (UID: \"1383c69a-30e5-4f5b-a593-8aaeec480e8c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-m8zdx" Jul 08 13:26:41.371719 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:41.371600 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lqqw\" (UniqueName: \"kubernetes.io/projected/1383c69a-30e5-4f5b-a593-8aaeec480e8c-kube-api-access-8lqqw\") pod \"cert-manager-cainjector-559cdd8c87-m8zdx\" (UID: \"1383c69a-30e5-4f5b-a593-8aaeec480e8c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-m8zdx" Jul 08 13:26:41.472825 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:41.472790 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1383c69a-30e5-4f5b-a593-8aaeec480e8c-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-m8zdx\" (UID: \"1383c69a-30e5-4f5b-a593-8aaeec480e8c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-m8zdx" Jul 08 13:26:41.473016 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:41.472854 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lqqw\" (UniqueName: \"kubernetes.io/projected/1383c69a-30e5-4f5b-a593-8aaeec480e8c-kube-api-access-8lqqw\") pod \"cert-manager-cainjector-559cdd8c87-m8zdx\" (UID: \"1383c69a-30e5-4f5b-a593-8aaeec480e8c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-m8zdx" Jul 08 13:26:41.481800 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:41.481767 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1383c69a-30e5-4f5b-a593-8aaeec480e8c-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-m8zdx\" (UID: \"1383c69a-30e5-4f5b-a593-8aaeec480e8c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-m8zdx" Jul 08 13:26:41.481933 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:41.481883 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lqqw\" (UniqueName: \"kubernetes.io/projected/1383c69a-30e5-4f5b-a593-8aaeec480e8c-kube-api-access-8lqqw\") pod \"cert-manager-cainjector-559cdd8c87-m8zdx\" (UID: \"1383c69a-30e5-4f5b-a593-8aaeec480e8c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-m8zdx" Jul 08 13:26:41.529391 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:41.529356 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-m8zdx" Jul 08 13:26:41.764415 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:41.764153 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-m8zdx"] Jul 08 13:26:41.768151 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:26:41.768123 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1383c69a_30e5_4f5b_a593_8aaeec480e8c.slice/crio-3126dde20a25ab000aacad997a3e8c69adbc5fd50d3f2611db3c37ccf27b11ea WatchSource:0}: Error finding container 3126dde20a25ab000aacad997a3e8c69adbc5fd50d3f2611db3c37ccf27b11ea: Status 404 returned error can't find the container with id 3126dde20a25ab000aacad997a3e8c69adbc5fd50d3f2611db3c37ccf27b11ea Jul 08 13:26:42.697401 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:42.697361 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-m8zdx" event={"ID":"1383c69a-30e5-4f5b-a593-8aaeec480e8c","Type":"ContainerStarted","Data":"3126dde20a25ab000aacad997a3e8c69adbc5fd50d3f2611db3c37ccf27b11ea"} Jul 08 13:26:42.699607 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:42.699579 2567 generic.go:356] "Generic (PLEG): container finished" podID="1e5ebf37-a8e9-47e7-9a41-60a656cca21f" containerID="e372e3d086e09de35600a7af020e3f988203342adcc67a60fa4cf17a0cf1e4d2" exitCode=0 Jul 08 13:26:42.699760 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:42.699624 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4c8w4" event={"ID":"1e5ebf37-a8e9-47e7-9a41-60a656cca21f","Type":"ContainerDied","Data":"e372e3d086e09de35600a7af020e3f988203342adcc67a60fa4cf17a0cf1e4d2"} Jul 08 13:26:43.704723 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:43.704669 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-m8zdx" event={"ID":"1383c69a-30e5-4f5b-a593-8aaeec480e8c","Type":"ContainerStarted","Data":"d1dc96165c0462a1b474a06f95b336673eb54fe4b68de807cf5371f286c80428"} Jul 08 13:26:43.706189 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:43.706156 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-8dvll" event={"ID":"7748c5c8-602f-4374-8f56-651d52fddc79","Type":"ContainerStarted","Data":"10f783bc0ad2158721fd97eb44582f1188cc64b426e03a60d30a102bb9324373"} Jul 08 13:26:43.706325 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:43.706252 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-8dvll" Jul 08 13:26:43.708183 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:43.708162 2567 generic.go:356] "Generic (PLEG): container finished" podID="1e5ebf37-a8e9-47e7-9a41-60a656cca21f" containerID="2d52b6de3b2ca6b2160d956ca1c29506f8f1182053bbbd3337b1ea2ab56fd56a" exitCode=0 Jul 08 13:26:43.708286 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:43.708206 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4c8w4" event={"ID":"1e5ebf37-a8e9-47e7-9a41-60a656cca21f","Type":"ContainerDied","Data":"2d52b6de3b2ca6b2160d956ca1c29506f8f1182053bbbd3337b1ea2ab56fd56a"} Jul 08 13:26:43.720052 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:43.720010 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-m8zdx" podStartSLOduration=1.723745478 podStartE2EDuration="2.719998075s" totalImagesPullingTime="996.252597ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:26:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:26:43.718629713 +0000 UTC m=+212.515570368" watchObservedRunningTime="2026-07-08 13:26:43.719998075 +0000 UTC m=+212.516938732" Jul 08 13:26:43.752531 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:43.752480 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-8dvll" podStartSLOduration=1.655160114 podStartE2EDuration="4.75246224s" totalImagesPullingTime="3.097302126s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:26:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:26:43.751595485 +0000 UTC m=+212.548536142" watchObservedRunningTime="2026-07-08 13:26:43.75246224 +0000 UTC m=+212.549402895" Jul 08 13:26:44.831901 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:44.831877 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4c8w4" Jul 08 13:26:45.003436 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:45.003348 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7pwps\" (UniqueName: \"kubernetes.io/projected/1e5ebf37-a8e9-47e7-9a41-60a656cca21f-kube-api-access-7pwps\") pod \"1e5ebf37-a8e9-47e7-9a41-60a656cca21f\" (UID: \"1e5ebf37-a8e9-47e7-9a41-60a656cca21f\") " Jul 08 13:26:45.003436 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:45.003388 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1e5ebf37-a8e9-47e7-9a41-60a656cca21f-util\") pod \"1e5ebf37-a8e9-47e7-9a41-60a656cca21f\" (UID: \"1e5ebf37-a8e9-47e7-9a41-60a656cca21f\") " Jul 08 13:26:45.003664 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:45.003503 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1e5ebf37-a8e9-47e7-9a41-60a656cca21f-bundle\") pod \"1e5ebf37-a8e9-47e7-9a41-60a656cca21f\" (UID: \"1e5ebf37-a8e9-47e7-9a41-60a656cca21f\") " Jul 08 13:26:45.003872 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:45.003850 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e5ebf37-a8e9-47e7-9a41-60a656cca21f-bundle" (OuterVolumeSpecName: "bundle") pod "1e5ebf37-a8e9-47e7-9a41-60a656cca21f" (UID: "1e5ebf37-a8e9-47e7-9a41-60a656cca21f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:26:45.005611 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:45.005581 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e5ebf37-a8e9-47e7-9a41-60a656cca21f-kube-api-access-7pwps" (OuterVolumeSpecName: "kube-api-access-7pwps") pod "1e5ebf37-a8e9-47e7-9a41-60a656cca21f" (UID: "1e5ebf37-a8e9-47e7-9a41-60a656cca21f"). InnerVolumeSpecName "kube-api-access-7pwps". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 13:26:45.008145 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:45.008122 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e5ebf37-a8e9-47e7-9a41-60a656cca21f-util" (OuterVolumeSpecName: "util") pod "1e5ebf37-a8e9-47e7-9a41-60a656cca21f" (UID: "1e5ebf37-a8e9-47e7-9a41-60a656cca21f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:26:45.104873 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:45.104844 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7pwps\" (UniqueName: \"kubernetes.io/projected/1e5ebf37-a8e9-47e7-9a41-60a656cca21f-kube-api-access-7pwps\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:26:45.104873 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:45.104869 2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1e5ebf37-a8e9-47e7-9a41-60a656cca21f-util\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:26:45.104873 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:45.104879 2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1e5ebf37-a8e9-47e7-9a41-60a656cca21f-bundle\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:26:45.718481 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:45.718450 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4c8w4" Jul 08 13:26:45.718481 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:45.718459 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f4c8w4" event={"ID":"1e5ebf37-a8e9-47e7-9a41-60a656cca21f","Type":"ContainerDied","Data":"8e9a73e05f63c51cb7660b68ee94c17e439c33e04ca646715e40e4ea3de62e2a"} Jul 08 13:26:45.718677 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:45.718507 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8e9a73e05f63c51cb7660b68ee94c17e439c33e04ca646715e40e4ea3de62e2a" Jul 08 13:26:49.403463 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:49.403417 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-2mgb8"] Jul 08 13:26:49.404040 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:49.404014 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e5ebf37-a8e9-47e7-9a41-60a656cca21f" containerName="util" Jul 08 13:26:49.404156 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:49.404041 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e5ebf37-a8e9-47e7-9a41-60a656cca21f" containerName="util" Jul 08 13:26:49.404156 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:49.404059 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e5ebf37-a8e9-47e7-9a41-60a656cca21f" containerName="pull" Jul 08 13:26:49.404156 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:49.404067 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e5ebf37-a8e9-47e7-9a41-60a656cca21f" containerName="pull" Jul 08 13:26:49.404156 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:49.404144 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e5ebf37-a8e9-47e7-9a41-60a656cca21f" containerName="extract" Jul 08 13:26:49.404156 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:49.404154 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e5ebf37-a8e9-47e7-9a41-60a656cca21f" containerName="extract" Jul 08 13:26:49.404384 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:49.404350 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e5ebf37-a8e9-47e7-9a41-60a656cca21f" containerName="extract" Jul 08 13:26:49.406913 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:49.406894 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-2mgb8" Jul 08 13:26:49.409491 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:49.409472 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-swxtg\"" Jul 08 13:26:49.414155 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:49.414135 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-2mgb8"] Jul 08 13:26:49.544857 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:49.544829 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nknrj\" (UniqueName: \"kubernetes.io/projected/2fc52caf-5205-4190-9c04-b9845adb7193-kube-api-access-nknrj\") pod \"cert-manager-55b4d954d5-2mgb8\" (UID: \"2fc52caf-5205-4190-9c04-b9845adb7193\") " pod="cert-manager/cert-manager-55b4d954d5-2mgb8" Jul 08 13:26:49.545033 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:49.544875 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2fc52caf-5205-4190-9c04-b9845adb7193-bound-sa-token\") pod \"cert-manager-55b4d954d5-2mgb8\" (UID: \"2fc52caf-5205-4190-9c04-b9845adb7193\") " pod="cert-manager/cert-manager-55b4d954d5-2mgb8" Jul 08 13:26:49.645424 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:49.645384 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2fc52caf-5205-4190-9c04-b9845adb7193-bound-sa-token\") pod \"cert-manager-55b4d954d5-2mgb8\" (UID: \"2fc52caf-5205-4190-9c04-b9845adb7193\") " pod="cert-manager/cert-manager-55b4d954d5-2mgb8" Jul 08 13:26:49.645622 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:49.645445 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nknrj\" (UniqueName: \"kubernetes.io/projected/2fc52caf-5205-4190-9c04-b9845adb7193-kube-api-access-nknrj\") pod \"cert-manager-55b4d954d5-2mgb8\" (UID: \"2fc52caf-5205-4190-9c04-b9845adb7193\") " pod="cert-manager/cert-manager-55b4d954d5-2mgb8" Jul 08 13:26:49.653462 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:49.653430 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2fc52caf-5205-4190-9c04-b9845adb7193-bound-sa-token\") pod \"cert-manager-55b4d954d5-2mgb8\" (UID: \"2fc52caf-5205-4190-9c04-b9845adb7193\") " pod="cert-manager/cert-manager-55b4d954d5-2mgb8" Jul 08 13:26:49.653668 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:49.653621 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nknrj\" (UniqueName: \"kubernetes.io/projected/2fc52caf-5205-4190-9c04-b9845adb7193-kube-api-access-nknrj\") pod \"cert-manager-55b4d954d5-2mgb8\" (UID: \"2fc52caf-5205-4190-9c04-b9845adb7193\") " pod="cert-manager/cert-manager-55b4d954d5-2mgb8" Jul 08 13:26:49.714741 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:49.714710 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-8dvll" Jul 08 13:26:49.716139 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:49.716121 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-2mgb8" Jul 08 13:26:49.844979 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:49.844948 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-2mgb8"] Jul 08 13:26:49.848957 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:26:49.848931 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2fc52caf_5205_4190_9c04_b9845adb7193.slice/crio-bd1ab93f105d88770aec1e31f9c08dd51a5bace370c9d1e9dc7b5681ae107367 WatchSource:0}: Error finding container bd1ab93f105d88770aec1e31f9c08dd51a5bace370c9d1e9dc7b5681ae107367: Status 404 returned error can't find the container with id bd1ab93f105d88770aec1e31f9c08dd51a5bace370c9d1e9dc7b5681ae107367 Jul 08 13:26:50.742926 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:50.742889 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-2mgb8" event={"ID":"2fc52caf-5205-4190-9c04-b9845adb7193","Type":"ContainerStarted","Data":"4328b840c097148abf740ccce63694608009d06fa00a66a26e23425655f1df1f"} Jul 08 13:26:50.742926 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:50.742932 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-2mgb8" event={"ID":"2fc52caf-5205-4190-9c04-b9845adb7193","Type":"ContainerStarted","Data":"bd1ab93f105d88770aec1e31f9c08dd51a5bace370c9d1e9dc7b5681ae107367"} Jul 08 13:26:50.757975 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:50.757929 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-2mgb8" podStartSLOduration=1.7579145189999998 podStartE2EDuration="1.757914519s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:26:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:26:50.757559738 +0000 UTC m=+219.554500397" watchObservedRunningTime="2026-07-08 13:26:50.757914519 +0000 UTC m=+219.554855174" Jul 08 13:26:59.849500 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:59.849461 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftm5br"] Jul 08 13:26:59.873877 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:59.873836 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftm5br"] Jul 08 13:26:59.874054 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:59.873961 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftm5br" Jul 08 13:26:59.876830 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:59.876806 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-mm72n\"" Jul 08 13:26:59.876960 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:59.876862 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 08 13:26:59.878208 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:59.878188 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 08 13:26:59.918163 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:59.918136 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/82f03698-a579-42b3-974d-3ea67311b06d-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftm5br\" (UID: \"82f03698-a579-42b3-974d-3ea67311b06d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftm5br" Jul 08 13:26:59.918163 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:59.918165 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/82f03698-a579-42b3-974d-3ea67311b06d-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftm5br\" (UID: \"82f03698-a579-42b3-974d-3ea67311b06d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftm5br" Jul 08 13:26:59.918301 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:26:59.918194 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4ccp\" (UniqueName: \"kubernetes.io/projected/82f03698-a579-42b3-974d-3ea67311b06d-kube-api-access-t4ccp\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftm5br\" (UID: \"82f03698-a579-42b3-974d-3ea67311b06d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftm5br" Jul 08 13:27:00.019255 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:00.019227 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/82f03698-a579-42b3-974d-3ea67311b06d-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftm5br\" (UID: \"82f03698-a579-42b3-974d-3ea67311b06d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftm5br" Jul 08 13:27:00.019255 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:00.019259 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/82f03698-a579-42b3-974d-3ea67311b06d-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftm5br\" (UID: \"82f03698-a579-42b3-974d-3ea67311b06d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftm5br" Jul 08 13:27:00.019490 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:00.019307 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4ccp\" (UniqueName: \"kubernetes.io/projected/82f03698-a579-42b3-974d-3ea67311b06d-kube-api-access-t4ccp\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftm5br\" (UID: \"82f03698-a579-42b3-974d-3ea67311b06d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftm5br" Jul 08 13:27:00.019634 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:00.019614 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/82f03698-a579-42b3-974d-3ea67311b06d-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftm5br\" (UID: \"82f03698-a579-42b3-974d-3ea67311b06d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftm5br" Jul 08 13:27:00.019694 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:00.019671 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/82f03698-a579-42b3-974d-3ea67311b06d-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftm5br\" (UID: \"82f03698-a579-42b3-974d-3ea67311b06d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftm5br" Jul 08 13:27:00.027373 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:00.027350 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4ccp\" (UniqueName: \"kubernetes.io/projected/82f03698-a579-42b3-974d-3ea67311b06d-kube-api-access-t4ccp\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftm5br\" (UID: \"82f03698-a579-42b3-974d-3ea67311b06d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftm5br" Jul 08 13:27:00.183303 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:00.183218 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftm5br" Jul 08 13:27:00.336284 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:00.336260 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftm5br"] Jul 08 13:27:00.338760 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:27:00.338737 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82f03698_a579_42b3_974d_3ea67311b06d.slice/crio-0b2fbb0bdb00320b2bfc5732509dc2189785db952d49d2c564bde01297c7306b WatchSource:0}: Error finding container 0b2fbb0bdb00320b2bfc5732509dc2189785db952d49d2c564bde01297c7306b: Status 404 returned error can't find the container with id 0b2fbb0bdb00320b2bfc5732509dc2189785db952d49d2c564bde01297c7306b Jul 08 13:27:00.784302 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:00.784263 2567 generic.go:356] "Generic (PLEG): container finished" podID="82f03698-a579-42b3-974d-3ea67311b06d" containerID="5d3b73474e7e48b1178614a0a31bd9a3bb4cef3b7b9c7e91bec9fd929d099f88" exitCode=0 Jul 08 13:27:00.784538 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:00.784348 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftm5br" event={"ID":"82f03698-a579-42b3-974d-3ea67311b06d","Type":"ContainerDied","Data":"5d3b73474e7e48b1178614a0a31bd9a3bb4cef3b7b9c7e91bec9fd929d099f88"} Jul 08 13:27:00.784538 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:00.784417 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftm5br" event={"ID":"82f03698-a579-42b3-974d-3ea67311b06d","Type":"ContainerStarted","Data":"0b2fbb0bdb00320b2bfc5732509dc2189785db952d49d2c564bde01297c7306b"} Jul 08 13:27:02.794533 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:02.794489 2567 generic.go:356] "Generic (PLEG): container finished" podID="82f03698-a579-42b3-974d-3ea67311b06d" containerID="f94b6f258cf4df38f12fdccb540a8922fdb141cd265c6d5626ae3b08ad252f92" exitCode=0 Jul 08 13:27:02.794982 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:02.794577 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftm5br" event={"ID":"82f03698-a579-42b3-974d-3ea67311b06d","Type":"ContainerDied","Data":"f94b6f258cf4df38f12fdccb540a8922fdb141cd265c6d5626ae3b08ad252f92"} Jul 08 13:27:03.799901 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:03.799871 2567 generic.go:356] "Generic (PLEG): container finished" podID="82f03698-a579-42b3-974d-3ea67311b06d" containerID="4ae6aadeb06ecc8bdfeb2ecf224e0d67d39eaae64264f483a4220d3637197ed7" exitCode=0 Jul 08 13:27:03.800310 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:03.799952 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftm5br" event={"ID":"82f03698-a579-42b3-974d-3ea67311b06d","Type":"ContainerDied","Data":"4ae6aadeb06ecc8bdfeb2ecf224e0d67d39eaae64264f483a4220d3637197ed7"} Jul 08 13:27:04.934095 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:04.934051 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftm5br" Jul 08 13:27:04.948884 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:04.948857 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t4ccp\" (UniqueName: \"kubernetes.io/projected/82f03698-a579-42b3-974d-3ea67311b06d-kube-api-access-t4ccp\") pod \"82f03698-a579-42b3-974d-3ea67311b06d\" (UID: \"82f03698-a579-42b3-974d-3ea67311b06d\") " Jul 08 13:27:04.949019 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:04.948902 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/82f03698-a579-42b3-974d-3ea67311b06d-util\") pod \"82f03698-a579-42b3-974d-3ea67311b06d\" (UID: \"82f03698-a579-42b3-974d-3ea67311b06d\") " Jul 08 13:27:04.949019 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:04.948934 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/82f03698-a579-42b3-974d-3ea67311b06d-bundle\") pod \"82f03698-a579-42b3-974d-3ea67311b06d\" (UID: \"82f03698-a579-42b3-974d-3ea67311b06d\") " Jul 08 13:27:04.949795 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:04.949728 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82f03698-a579-42b3-974d-3ea67311b06d-bundle" (OuterVolumeSpecName: "bundle") pod "82f03698-a579-42b3-974d-3ea67311b06d" (UID: "82f03698-a579-42b3-974d-3ea67311b06d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:27:04.951310 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:04.951279 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82f03698-a579-42b3-974d-3ea67311b06d-kube-api-access-t4ccp" (OuterVolumeSpecName: "kube-api-access-t4ccp") pod "82f03698-a579-42b3-974d-3ea67311b06d" (UID: "82f03698-a579-42b3-974d-3ea67311b06d"). InnerVolumeSpecName "kube-api-access-t4ccp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 13:27:04.957991 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:04.957962 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82f03698-a579-42b3-974d-3ea67311b06d-util" (OuterVolumeSpecName: "util") pod "82f03698-a579-42b3-974d-3ea67311b06d" (UID: "82f03698-a579-42b3-974d-3ea67311b06d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:27:05.050295 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:05.050260 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t4ccp\" (UniqueName: \"kubernetes.io/projected/82f03698-a579-42b3-974d-3ea67311b06d-kube-api-access-t4ccp\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:27:05.050295 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:05.050289 2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/82f03698-a579-42b3-974d-3ea67311b06d-util\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:27:05.050295 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:05.050300 2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/82f03698-a579-42b3-974d-3ea67311b06d-bundle\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:27:05.809389 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:05.809359 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftm5br" event={"ID":"82f03698-a579-42b3-974d-3ea67311b06d","Type":"ContainerDied","Data":"0b2fbb0bdb00320b2bfc5732509dc2189785db952d49d2c564bde01297c7306b"} Jul 08 13:27:05.809542 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:05.809396 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftm5br" Jul 08 13:27:05.809542 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:05.809404 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0b2fbb0bdb00320b2bfc5732509dc2189785db952d49d2c564bde01297c7306b" Jul 08 13:27:12.398025 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:12.397990 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7fcdcfc8c4-mrdm8"] Jul 08 13:27:12.398478 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:12.398283 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82f03698-a579-42b3-974d-3ea67311b06d" containerName="pull" Jul 08 13:27:12.398478 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:12.398294 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="82f03698-a579-42b3-974d-3ea67311b06d" containerName="pull" Jul 08 13:27:12.398478 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:12.398303 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82f03698-a579-42b3-974d-3ea67311b06d" containerName="util" Jul 08 13:27:12.398478 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:12.398310 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="82f03698-a579-42b3-974d-3ea67311b06d" containerName="util" Jul 08 13:27:12.398478 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:12.398319 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82f03698-a579-42b3-974d-3ea67311b06d" containerName="extract" Jul 08 13:27:12.398478 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:12.398326 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="82f03698-a579-42b3-974d-3ea67311b06d" containerName="extract" Jul 08 13:27:12.398478 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:12.398372 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="82f03698-a579-42b3-974d-3ea67311b06d" containerName="extract" Jul 08 13:27:12.403510 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:12.403491 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7fcdcfc8c4-mrdm8" Jul 08 13:27:12.406777 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:12.406750 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 08 13:27:12.406907 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:12.406785 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 08 13:27:12.407903 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:12.407883 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 08 13:27:12.407997 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:12.407980 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 08 13:27:12.408043 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:12.407981 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-lk7ng\"" Jul 08 13:27:12.408096 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:12.407987 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 08 13:27:12.416657 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:12.416635 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7fcdcfc8c4-mrdm8"] Jul 08 13:27:12.506554 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:12.506517 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgtgk\" (UniqueName: \"kubernetes.io/projected/a2776123-af46-4009-a214-9aeffc5677b6-kube-api-access-cgtgk\") pod \"lws-controller-manager-7fcdcfc8c4-mrdm8\" (UID: \"a2776123-af46-4009-a214-9aeffc5677b6\") " pod="openshift-lws-operator/lws-controller-manager-7fcdcfc8c4-mrdm8" Jul 08 13:27:12.506742 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:12.506575 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a2776123-af46-4009-a214-9aeffc5677b6-cert\") pod \"lws-controller-manager-7fcdcfc8c4-mrdm8\" (UID: \"a2776123-af46-4009-a214-9aeffc5677b6\") " pod="openshift-lws-operator/lws-controller-manager-7fcdcfc8c4-mrdm8" Jul 08 13:27:12.506742 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:12.506599 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a2776123-af46-4009-a214-9aeffc5677b6-manager-config\") pod \"lws-controller-manager-7fcdcfc8c4-mrdm8\" (UID: \"a2776123-af46-4009-a214-9aeffc5677b6\") " pod="openshift-lws-operator/lws-controller-manager-7fcdcfc8c4-mrdm8" Jul 08 13:27:12.506742 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:12.506653 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a2776123-af46-4009-a214-9aeffc5677b6-metrics-cert\") pod \"lws-controller-manager-7fcdcfc8c4-mrdm8\" (UID: \"a2776123-af46-4009-a214-9aeffc5677b6\") " pod="openshift-lws-operator/lws-controller-manager-7fcdcfc8c4-mrdm8" Jul 08 13:27:12.607713 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:12.607673 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgtgk\" (UniqueName: \"kubernetes.io/projected/a2776123-af46-4009-a214-9aeffc5677b6-kube-api-access-cgtgk\") pod \"lws-controller-manager-7fcdcfc8c4-mrdm8\" (UID: \"a2776123-af46-4009-a214-9aeffc5677b6\") " pod="openshift-lws-operator/lws-controller-manager-7fcdcfc8c4-mrdm8" Jul 08 13:27:12.607893 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:12.607737 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a2776123-af46-4009-a214-9aeffc5677b6-cert\") pod \"lws-controller-manager-7fcdcfc8c4-mrdm8\" (UID: \"a2776123-af46-4009-a214-9aeffc5677b6\") " pod="openshift-lws-operator/lws-controller-manager-7fcdcfc8c4-mrdm8" Jul 08 13:27:12.607893 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:12.607763 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a2776123-af46-4009-a214-9aeffc5677b6-manager-config\") pod \"lws-controller-manager-7fcdcfc8c4-mrdm8\" (UID: \"a2776123-af46-4009-a214-9aeffc5677b6\") " pod="openshift-lws-operator/lws-controller-manager-7fcdcfc8c4-mrdm8" Jul 08 13:27:12.607893 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:12.607790 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a2776123-af46-4009-a214-9aeffc5677b6-metrics-cert\") pod \"lws-controller-manager-7fcdcfc8c4-mrdm8\" (UID: \"a2776123-af46-4009-a214-9aeffc5677b6\") " pod="openshift-lws-operator/lws-controller-manager-7fcdcfc8c4-mrdm8" Jul 08 13:27:12.608557 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:12.608531 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a2776123-af46-4009-a214-9aeffc5677b6-manager-config\") pod \"lws-controller-manager-7fcdcfc8c4-mrdm8\" (UID: \"a2776123-af46-4009-a214-9aeffc5677b6\") " pod="openshift-lws-operator/lws-controller-manager-7fcdcfc8c4-mrdm8" Jul 08 13:27:12.610245 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:12.610217 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a2776123-af46-4009-a214-9aeffc5677b6-cert\") pod \"lws-controller-manager-7fcdcfc8c4-mrdm8\" (UID: \"a2776123-af46-4009-a214-9aeffc5677b6\") " pod="openshift-lws-operator/lws-controller-manager-7fcdcfc8c4-mrdm8" Jul 08 13:27:12.610340 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:12.610267 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a2776123-af46-4009-a214-9aeffc5677b6-metrics-cert\") pod \"lws-controller-manager-7fcdcfc8c4-mrdm8\" (UID: \"a2776123-af46-4009-a214-9aeffc5677b6\") " pod="openshift-lws-operator/lws-controller-manager-7fcdcfc8c4-mrdm8" Jul 08 13:27:12.617391 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:12.617363 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgtgk\" (UniqueName: \"kubernetes.io/projected/a2776123-af46-4009-a214-9aeffc5677b6-kube-api-access-cgtgk\") pod \"lws-controller-manager-7fcdcfc8c4-mrdm8\" (UID: \"a2776123-af46-4009-a214-9aeffc5677b6\") " pod="openshift-lws-operator/lws-controller-manager-7fcdcfc8c4-mrdm8" Jul 08 13:27:12.712390 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:12.712349 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7fcdcfc8c4-mrdm8" Jul 08 13:27:12.872272 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:12.872240 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7fcdcfc8c4-mrdm8"] Jul 08 13:27:12.874606 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:27:12.874581 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2776123_af46_4009_a214_9aeffc5677b6.slice/crio-baf858dcca8c324ce046d2ee262420f9e3b4a6e6b9d1fe1c2c7a77df015ac588 WatchSource:0}: Error finding container baf858dcca8c324ce046d2ee262420f9e3b4a6e6b9d1fe1c2c7a77df015ac588: Status 404 returned error can't find the container with id baf858dcca8c324ce046d2ee262420f9e3b4a6e6b9d1fe1c2c7a77df015ac588 Jul 08 13:27:13.844741 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:13.844699 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7fcdcfc8c4-mrdm8" event={"ID":"a2776123-af46-4009-a214-9aeffc5677b6","Type":"ContainerStarted","Data":"baf858dcca8c324ce046d2ee262420f9e3b4a6e6b9d1fe1c2c7a77df015ac588"} Jul 08 13:27:14.157890 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:14.157807 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-774b77d499-hgmh9"] Jul 08 13:27:14.160035 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:14.160011 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-774b77d499-hgmh9" Jul 08 13:27:14.164872 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:14.164797 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 08 13:27:14.165037 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:14.164981 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-hjkcl\"" Jul 08 13:27:14.165128 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:14.165067 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 08 13:27:14.165272 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:14.165219 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 08 13:27:14.165399 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:14.165288 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 08 13:27:14.220828 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:14.220795 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e5d7ed9d-7e90-4e91-9588-ac7b72e29049-webhook-cert\") pod \"opendatahub-operator-controller-manager-774b77d499-hgmh9\" (UID: \"e5d7ed9d-7e90-4e91-9588-ac7b72e29049\") " pod="opendatahub/opendatahub-operator-controller-manager-774b77d499-hgmh9" Jul 08 13:27:14.220828 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:14.220838 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e5d7ed9d-7e90-4e91-9588-ac7b72e29049-apiservice-cert\") pod \"opendatahub-operator-controller-manager-774b77d499-hgmh9\" (UID: \"e5d7ed9d-7e90-4e91-9588-ac7b72e29049\") " pod="opendatahub/opendatahub-operator-controller-manager-774b77d499-hgmh9" Jul 08 13:27:14.221060 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:14.220918 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnq8z\" (UniqueName: \"kubernetes.io/projected/e5d7ed9d-7e90-4e91-9588-ac7b72e29049-kube-api-access-qnq8z\") pod \"opendatahub-operator-controller-manager-774b77d499-hgmh9\" (UID: \"e5d7ed9d-7e90-4e91-9588-ac7b72e29049\") " pod="opendatahub/opendatahub-operator-controller-manager-774b77d499-hgmh9" Jul 08 13:27:14.221060 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:14.220995 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e5d7ed9d-7e90-4e91-9588-ac7b72e29049-manifests\") pod \"opendatahub-operator-controller-manager-774b77d499-hgmh9\" (UID: \"e5d7ed9d-7e90-4e91-9588-ac7b72e29049\") " pod="opendatahub/opendatahub-operator-controller-manager-774b77d499-hgmh9" Jul 08 13:27:14.221060 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:14.221051 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e5d7ed9d-7e90-4e91-9588-ac7b72e29049-tmp\") pod \"opendatahub-operator-controller-manager-774b77d499-hgmh9\" (UID: \"e5d7ed9d-7e90-4e91-9588-ac7b72e29049\") " pod="opendatahub/opendatahub-operator-controller-manager-774b77d499-hgmh9" Jul 08 13:27:14.234159 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:14.233477 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-774b77d499-hgmh9"] Jul 08 13:27:14.321593 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:14.321549 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e5d7ed9d-7e90-4e91-9588-ac7b72e29049-manifests\") pod \"opendatahub-operator-controller-manager-774b77d499-hgmh9\" (UID: \"e5d7ed9d-7e90-4e91-9588-ac7b72e29049\") " pod="opendatahub/opendatahub-operator-controller-manager-774b77d499-hgmh9" Jul 08 13:27:14.321802 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:14.321632 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e5d7ed9d-7e90-4e91-9588-ac7b72e29049-tmp\") pod \"opendatahub-operator-controller-manager-774b77d499-hgmh9\" (UID: \"e5d7ed9d-7e90-4e91-9588-ac7b72e29049\") " pod="opendatahub/opendatahub-operator-controller-manager-774b77d499-hgmh9" Jul 08 13:27:14.321802 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:14.321677 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e5d7ed9d-7e90-4e91-9588-ac7b72e29049-webhook-cert\") pod \"opendatahub-operator-controller-manager-774b77d499-hgmh9\" (UID: \"e5d7ed9d-7e90-4e91-9588-ac7b72e29049\") " pod="opendatahub/opendatahub-operator-controller-manager-774b77d499-hgmh9" Jul 08 13:27:14.321802 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:14.321696 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e5d7ed9d-7e90-4e91-9588-ac7b72e29049-apiservice-cert\") pod \"opendatahub-operator-controller-manager-774b77d499-hgmh9\" (UID: \"e5d7ed9d-7e90-4e91-9588-ac7b72e29049\") " pod="opendatahub/opendatahub-operator-controller-manager-774b77d499-hgmh9" Jul 08 13:27:14.321802 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:14.321786 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnq8z\" (UniqueName: \"kubernetes.io/projected/e5d7ed9d-7e90-4e91-9588-ac7b72e29049-kube-api-access-qnq8z\") pod \"opendatahub-operator-controller-manager-774b77d499-hgmh9\" (UID: \"e5d7ed9d-7e90-4e91-9588-ac7b72e29049\") " pod="opendatahub/opendatahub-operator-controller-manager-774b77d499-hgmh9" Jul 08 13:27:14.322050 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:14.322026 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e5d7ed9d-7e90-4e91-9588-ac7b72e29049-manifests\") pod \"opendatahub-operator-controller-manager-774b77d499-hgmh9\" (UID: \"e5d7ed9d-7e90-4e91-9588-ac7b72e29049\") " pod="opendatahub/opendatahub-operator-controller-manager-774b77d499-hgmh9" Jul 08 13:27:14.324659 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:14.324634 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e5d7ed9d-7e90-4e91-9588-ac7b72e29049-tmp\") pod \"opendatahub-operator-controller-manager-774b77d499-hgmh9\" (UID: \"e5d7ed9d-7e90-4e91-9588-ac7b72e29049\") " pod="opendatahub/opendatahub-operator-controller-manager-774b77d499-hgmh9" Jul 08 13:27:14.324796 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:14.324767 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e5d7ed9d-7e90-4e91-9588-ac7b72e29049-apiservice-cert\") pod \"opendatahub-operator-controller-manager-774b77d499-hgmh9\" (UID: \"e5d7ed9d-7e90-4e91-9588-ac7b72e29049\") " pod="opendatahub/opendatahub-operator-controller-manager-774b77d499-hgmh9" Jul 08 13:27:14.324873 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:14.324829 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e5d7ed9d-7e90-4e91-9588-ac7b72e29049-webhook-cert\") pod \"opendatahub-operator-controller-manager-774b77d499-hgmh9\" (UID: \"e5d7ed9d-7e90-4e91-9588-ac7b72e29049\") " pod="opendatahub/opendatahub-operator-controller-manager-774b77d499-hgmh9" Jul 08 13:27:14.332475 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:14.332437 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnq8z\" (UniqueName: \"kubernetes.io/projected/e5d7ed9d-7e90-4e91-9588-ac7b72e29049-kube-api-access-qnq8z\") pod \"opendatahub-operator-controller-manager-774b77d499-hgmh9\" (UID: \"e5d7ed9d-7e90-4e91-9588-ac7b72e29049\") " pod="opendatahub/opendatahub-operator-controller-manager-774b77d499-hgmh9" Jul 08 13:27:14.472733 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:14.472707 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-774b77d499-hgmh9" Jul 08 13:27:14.634738 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:14.634712 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-774b77d499-hgmh9"] Jul 08 13:27:14.635171 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:27:14.635142 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5d7ed9d_7e90_4e91_9588_ac7b72e29049.slice/crio-253bf80a55da7dafc458b9a9aa29881ed76822310ee71424717f5618f01d9f75 WatchSource:0}: Error finding container 253bf80a55da7dafc458b9a9aa29881ed76822310ee71424717f5618f01d9f75: Status 404 returned error can't find the container with id 253bf80a55da7dafc458b9a9aa29881ed76822310ee71424717f5618f01d9f75 Jul 08 13:27:14.850165 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:14.850127 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7fcdcfc8c4-mrdm8" event={"ID":"a2776123-af46-4009-a214-9aeffc5677b6","Type":"ContainerStarted","Data":"aa2d8f3f86ba0d3ef49174d1f94f998db8a4bde6665c3eeaba1ba308d29e6eb2"} Jul 08 13:27:14.850578 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:14.850227 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7fcdcfc8c4-mrdm8" Jul 08 13:27:14.851370 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:14.851348 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-774b77d499-hgmh9" event={"ID":"e5d7ed9d-7e90-4e91-9588-ac7b72e29049","Type":"ContainerStarted","Data":"253bf80a55da7dafc458b9a9aa29881ed76822310ee71424717f5618f01d9f75"} Jul 08 13:27:14.867517 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:14.867461 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7fcdcfc8c4-mrdm8" podStartSLOduration=1.320353901 podStartE2EDuration="2.867447352s" totalImagesPullingTime="1.547093451s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:27:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:27:14.865732476 +0000 UTC m=+243.662673132" watchObservedRunningTime="2026-07-08 13:27:14.867447352 +0000 UTC m=+243.664388007" Jul 08 13:27:18.872696 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:18.872656 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-774b77d499-hgmh9" event={"ID":"e5d7ed9d-7e90-4e91-9588-ac7b72e29049","Type":"ContainerStarted","Data":"aa0323ed75edc577aecf260d08c85ebe9110c8a6c2c54bd7ebc46843a0525bfb"} Jul 08 13:27:19.878838 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:19.878801 2567 generic.go:356] "Generic (PLEG): container finished" podID="e5d7ed9d-7e90-4e91-9588-ac7b72e29049" containerID="aa0323ed75edc577aecf260d08c85ebe9110c8a6c2c54bd7ebc46843a0525bfb" exitCode=0 Jul 08 13:27:19.879229 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:19.878898 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-774b77d499-hgmh9" event={"ID":"e5d7ed9d-7e90-4e91-9588-ac7b72e29049","Type":"ContainerDied","Data":"aa0323ed75edc577aecf260d08c85ebe9110c8a6c2c54bd7ebc46843a0525bfb"} Jul 08 13:27:20.885502 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:20.885460 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-774b77d499-hgmh9" event={"ID":"e5d7ed9d-7e90-4e91-9588-ac7b72e29049","Type":"ContainerStarted","Data":"d7e0aa5f3b6c149ddb18ca4d59d796466c4f2807dad00d30b8712965208051ed"} Jul 08 13:27:20.885890 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:20.885564 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-774b77d499-hgmh9" Jul 08 13:27:20.906982 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:20.906926 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-774b77d499-hgmh9" podStartSLOduration=2.294485521 podStartE2EDuration="6.906913299s" totalImagesPullingTime="3.61059771s" totalInitContainerRuntime="1.001830068s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:27:14 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:27:20.905322128 +0000 UTC m=+249.702262783" watchObservedRunningTime="2026-07-08 13:27:20.906913299 +0000 UTC m=+249.703853955" Jul 08 13:27:25.859068 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:25.859037 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7fcdcfc8c4-mrdm8" Jul 08 13:27:31.891245 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:31.891217 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-774b77d499-hgmh9" Jul 08 13:27:42.110601 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.110521 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678lqnzl"] Jul 08 13:27:42.114165 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.114146 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678lqnzl" Jul 08 13:27:42.116895 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.116872 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 08 13:27:42.116993 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.116883 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 08 13:27:42.117192 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.117175 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 08 13:27:42.117243 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.117229 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-wp7qh\"" Jul 08 13:27:42.128537 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.128501 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678lqnzl"] Jul 08 13:27:42.151008 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.150973 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sfrfw"] Jul 08 13:27:42.155541 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.155516 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sfrfw" Jul 08 13:27:42.158326 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.158304 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 08 13:27:42.158590 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.158572 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 08 13:27:42.158643 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.158574 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-bx88f\"" Jul 08 13:27:42.158643 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.158581 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 08 13:27:42.165458 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.165433 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sfrfw"] Jul 08 13:27:42.249301 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.249268 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/15f6cf7d-af37-41ff-a35e-9757e3d29f6b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-sfrfw\" (UID: \"15f6cf7d-af37-41ff-a35e-9757e3d29f6b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sfrfw" Jul 08 13:27:42.249500 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.249311 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3db34d8c-4001-4369-8dc6-2e6e00fcb4e1-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678lqnzl\" (UID: \"3db34d8c-4001-4369-8dc6-2e6e00fcb4e1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678lqnzl" Jul 08 13:27:42.249500 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.249334 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3db34d8c-4001-4369-8dc6-2e6e00fcb4e1-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678lqnzl\" (UID: \"3db34d8c-4001-4369-8dc6-2e6e00fcb4e1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678lqnzl" Jul 08 13:27:42.249500 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.249422 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15f6cf7d-af37-41ff-a35e-9757e3d29f6b-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-sfrfw\" (UID: \"15f6cf7d-af37-41ff-a35e-9757e3d29f6b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sfrfw" Jul 08 13:27:42.249500 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.249470 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/15f6cf7d-af37-41ff-a35e-9757e3d29f6b-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-sfrfw\" (UID: \"15f6cf7d-af37-41ff-a35e-9757e3d29f6b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sfrfw" Jul 08 13:27:42.249500 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.249494 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/15f6cf7d-af37-41ff-a35e-9757e3d29f6b-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-sfrfw\" (UID: \"15f6cf7d-af37-41ff-a35e-9757e3d29f6b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sfrfw" Jul 08 13:27:42.249770 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.249555 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3db34d8c-4001-4369-8dc6-2e6e00fcb4e1-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678lqnzl\" (UID: \"3db34d8c-4001-4369-8dc6-2e6e00fcb4e1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678lqnzl" Jul 08 13:27:42.249770 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.249582 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/15f6cf7d-af37-41ff-a35e-9757e3d29f6b-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-sfrfw\" (UID: \"15f6cf7d-af37-41ff-a35e-9757e3d29f6b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sfrfw" Jul 08 13:27:42.249770 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.249650 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3db34d8c-4001-4369-8dc6-2e6e00fcb4e1-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678lqnzl\" (UID: \"3db34d8c-4001-4369-8dc6-2e6e00fcb4e1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678lqnzl" Jul 08 13:27:42.249770 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.249677 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3db34d8c-4001-4369-8dc6-2e6e00fcb4e1-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678lqnzl\" (UID: \"3db34d8c-4001-4369-8dc6-2e6e00fcb4e1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678lqnzl" Jul 08 13:27:42.249770 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.249710 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3db34d8c-4001-4369-8dc6-2e6e00fcb4e1-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678lqnzl\" (UID: \"3db34d8c-4001-4369-8dc6-2e6e00fcb4e1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678lqnzl" Jul 08 13:27:42.249948 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.249765 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/15f6cf7d-af37-41ff-a35e-9757e3d29f6b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-sfrfw\" (UID: \"15f6cf7d-af37-41ff-a35e-9757e3d29f6b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sfrfw" Jul 08 13:27:42.249948 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.249798 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3db34d8c-4001-4369-8dc6-2e6e00fcb4e1-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678lqnzl\" (UID: \"3db34d8c-4001-4369-8dc6-2e6e00fcb4e1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678lqnzl" Jul 08 13:27:42.249948 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.249815 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3db34d8c-4001-4369-8dc6-2e6e00fcb4e1-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678lqnzl\" (UID: \"3db34d8c-4001-4369-8dc6-2e6e00fcb4e1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678lqnzl" Jul 08 13:27:42.249948 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.249834 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thjvt\" (UniqueName: \"kubernetes.io/projected/3db34d8c-4001-4369-8dc6-2e6e00fcb4e1-kube-api-access-thjvt\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678lqnzl\" (UID: \"3db34d8c-4001-4369-8dc6-2e6e00fcb4e1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678lqnzl" Jul 08 13:27:42.249948 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.249855 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmqpk\" (UniqueName: \"kubernetes.io/projected/15f6cf7d-af37-41ff-a35e-9757e3d29f6b-kube-api-access-qmqpk\") pod \"istiod-openshift-gateway-94bb8fbfd-sfrfw\" (UID: \"15f6cf7d-af37-41ff-a35e-9757e3d29f6b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sfrfw" Jul 08 13:27:42.351120 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.351044 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3db34d8c-4001-4369-8dc6-2e6e00fcb4e1-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678lqnzl\" (UID: \"3db34d8c-4001-4369-8dc6-2e6e00fcb4e1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678lqnzl" Jul 08 13:27:42.351120 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.351128 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/15f6cf7d-af37-41ff-a35e-9757e3d29f6b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-sfrfw\" (UID: \"15f6cf7d-af37-41ff-a35e-9757e3d29f6b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sfrfw" Jul 08 13:27:42.351371 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.351147 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3db34d8c-4001-4369-8dc6-2e6e00fcb4e1-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678lqnzl\" (UID: \"3db34d8c-4001-4369-8dc6-2e6e00fcb4e1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678lqnzl" Jul 08 13:27:42.351371 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.351167 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3db34d8c-4001-4369-8dc6-2e6e00fcb4e1-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678lqnzl\" (UID: \"3db34d8c-4001-4369-8dc6-2e6e00fcb4e1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678lqnzl" Jul 08 13:27:42.351371 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.351189 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thjvt\" (UniqueName: \"kubernetes.io/projected/3db34d8c-4001-4369-8dc6-2e6e00fcb4e1-kube-api-access-thjvt\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678lqnzl\" (UID: \"3db34d8c-4001-4369-8dc6-2e6e00fcb4e1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678lqnzl" Jul 08 13:27:42.351371 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.351206 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmqpk\" (UniqueName: \"kubernetes.io/projected/15f6cf7d-af37-41ff-a35e-9757e3d29f6b-kube-api-access-qmqpk\") pod \"istiod-openshift-gateway-94bb8fbfd-sfrfw\" (UID: \"15f6cf7d-af37-41ff-a35e-9757e3d29f6b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sfrfw" Jul 08 13:27:42.351371 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.351239 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/15f6cf7d-af37-41ff-a35e-9757e3d29f6b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-sfrfw\" (UID: \"15f6cf7d-af37-41ff-a35e-9757e3d29f6b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sfrfw" Jul 08 13:27:42.351371 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.351272 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3db34d8c-4001-4369-8dc6-2e6e00fcb4e1-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678lqnzl\" (UID: \"3db34d8c-4001-4369-8dc6-2e6e00fcb4e1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678lqnzl" Jul 08 13:27:42.351371 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.351297 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3db34d8c-4001-4369-8dc6-2e6e00fcb4e1-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678lqnzl\" (UID: \"3db34d8c-4001-4369-8dc6-2e6e00fcb4e1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678lqnzl" Jul 08 13:27:42.351371 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.351323 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15f6cf7d-af37-41ff-a35e-9757e3d29f6b-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-sfrfw\" (UID: \"15f6cf7d-af37-41ff-a35e-9757e3d29f6b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sfrfw" Jul 08 13:27:42.351371 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.351363 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/15f6cf7d-af37-41ff-a35e-9757e3d29f6b-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-sfrfw\" (UID: \"15f6cf7d-af37-41ff-a35e-9757e3d29f6b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sfrfw" Jul 08 13:27:42.351813 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.351387 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/15f6cf7d-af37-41ff-a35e-9757e3d29f6b-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-sfrfw\" (UID: \"15f6cf7d-af37-41ff-a35e-9757e3d29f6b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sfrfw" Jul 08 13:27:42.351813 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.351447 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3db34d8c-4001-4369-8dc6-2e6e00fcb4e1-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678lqnzl\" (UID: \"3db34d8c-4001-4369-8dc6-2e6e00fcb4e1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678lqnzl" Jul 08 13:27:42.351813 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.351516 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3db34d8c-4001-4369-8dc6-2e6e00fcb4e1-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678lqnzl\" (UID: \"3db34d8c-4001-4369-8dc6-2e6e00fcb4e1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678lqnzl" Jul 08 13:27:42.351813 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.351552 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3db34d8c-4001-4369-8dc6-2e6e00fcb4e1-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678lqnzl\" (UID: \"3db34d8c-4001-4369-8dc6-2e6e00fcb4e1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678lqnzl" Jul 08 13:27:42.351813 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.351701 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/15f6cf7d-af37-41ff-a35e-9757e3d29f6b-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-sfrfw\" (UID: \"15f6cf7d-af37-41ff-a35e-9757e3d29f6b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sfrfw" Jul 08 13:27:42.351813 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.351757 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3db34d8c-4001-4369-8dc6-2e6e00fcb4e1-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678lqnzl\" (UID: \"3db34d8c-4001-4369-8dc6-2e6e00fcb4e1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678lqnzl" Jul 08 13:27:42.352161 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.351910 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/15f6cf7d-af37-41ff-a35e-9757e3d29f6b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-sfrfw\" (UID: \"15f6cf7d-af37-41ff-a35e-9757e3d29f6b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sfrfw" Jul 08 13:27:42.352161 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.352015 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3db34d8c-4001-4369-8dc6-2e6e00fcb4e1-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678lqnzl\" (UID: \"3db34d8c-4001-4369-8dc6-2e6e00fcb4e1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678lqnzl" Jul 08 13:27:42.352327 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.352241 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3db34d8c-4001-4369-8dc6-2e6e00fcb4e1-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678lqnzl\" (UID: \"3db34d8c-4001-4369-8dc6-2e6e00fcb4e1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678lqnzl" Jul 08 13:27:42.352327 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.352302 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3db34d8c-4001-4369-8dc6-2e6e00fcb4e1-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678lqnzl\" (UID: \"3db34d8c-4001-4369-8dc6-2e6e00fcb4e1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678lqnzl" Jul 08 13:27:42.352615 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.352594 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3db34d8c-4001-4369-8dc6-2e6e00fcb4e1-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678lqnzl\" (UID: \"3db34d8c-4001-4369-8dc6-2e6e00fcb4e1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678lqnzl" Jul 08 13:27:42.354303 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.354277 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/15f6cf7d-af37-41ff-a35e-9757e3d29f6b-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-sfrfw\" (UID: \"15f6cf7d-af37-41ff-a35e-9757e3d29f6b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sfrfw" Jul 08 13:27:42.354411 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.354344 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/15f6cf7d-af37-41ff-a35e-9757e3d29f6b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-sfrfw\" (UID: \"15f6cf7d-af37-41ff-a35e-9757e3d29f6b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sfrfw" Jul 08 13:27:42.354411 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.354368 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/15f6cf7d-af37-41ff-a35e-9757e3d29f6b-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-sfrfw\" (UID: \"15f6cf7d-af37-41ff-a35e-9757e3d29f6b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sfrfw" Jul 08 13:27:42.354411 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.354351 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/15f6cf7d-af37-41ff-a35e-9757e3d29f6b-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-sfrfw\" (UID: \"15f6cf7d-af37-41ff-a35e-9757e3d29f6b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sfrfw" Jul 08 13:27:42.354564 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.354438 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3db34d8c-4001-4369-8dc6-2e6e00fcb4e1-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678lqnzl\" (UID: \"3db34d8c-4001-4369-8dc6-2e6e00fcb4e1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678lqnzl" Jul 08 13:27:42.354618 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.354605 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3db34d8c-4001-4369-8dc6-2e6e00fcb4e1-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678lqnzl\" (UID: \"3db34d8c-4001-4369-8dc6-2e6e00fcb4e1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678lqnzl" Jul 08 13:27:42.371368 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.371289 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3db34d8c-4001-4369-8dc6-2e6e00fcb4e1-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678lqnzl\" (UID: \"3db34d8c-4001-4369-8dc6-2e6e00fcb4e1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678lqnzl" Jul 08 13:27:42.371540 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.371518 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15f6cf7d-af37-41ff-a35e-9757e3d29f6b-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-sfrfw\" (UID: \"15f6cf7d-af37-41ff-a35e-9757e3d29f6b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sfrfw" Jul 08 13:27:42.372281 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.372261 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmqpk\" (UniqueName: \"kubernetes.io/projected/15f6cf7d-af37-41ff-a35e-9757e3d29f6b-kube-api-access-qmqpk\") pod \"istiod-openshift-gateway-94bb8fbfd-sfrfw\" (UID: \"15f6cf7d-af37-41ff-a35e-9757e3d29f6b\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sfrfw" Jul 08 13:27:42.373540 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.373520 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-thjvt\" (UniqueName: \"kubernetes.io/projected/3db34d8c-4001-4369-8dc6-2e6e00fcb4e1-kube-api-access-thjvt\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678lqnzl\" (UID: \"3db34d8c-4001-4369-8dc6-2e6e00fcb4e1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678lqnzl" Jul 08 13:27:42.424845 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.424814 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678lqnzl" Jul 08 13:27:42.465216 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.465185 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sfrfw" Jul 08 13:27:42.581717 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.581685 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678lqnzl"] Jul 08 13:27:42.583518 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:27:42.583484 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3db34d8c_4001_4369_8dc6_2e6e00fcb4e1.slice/crio-929040599c4d93aa59b3585b0b6be75ab9b2fbead34fb8aaba457d9085a32973 WatchSource:0}: Error finding container 929040599c4d93aa59b3585b0b6be75ab9b2fbead34fb8aaba457d9085a32973: Status 404 returned error can't find the container with id 929040599c4d93aa59b3585b0b6be75ab9b2fbead34fb8aaba457d9085a32973 Jul 08 13:27:42.620102 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.620055 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sfrfw"] Jul 08 13:27:42.622022 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:27:42.621961 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15f6cf7d_af37_41ff_a35e_9757e3d29f6b.slice/crio-d1acbfc43fb1e82df0d6551d763d269067d67feef477b80d35180418dae904d8 WatchSource:0}: Error finding container d1acbfc43fb1e82df0d6551d763d269067d67feef477b80d35180418dae904d8: Status 404 returned error can't find the container with id d1acbfc43fb1e82df0d6551d763d269067d67feef477b80d35180418dae904d8 Jul 08 13:27:42.983501 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.983458 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678lqnzl" event={"ID":"3db34d8c-4001-4369-8dc6-2e6e00fcb4e1","Type":"ContainerStarted","Data":"929040599c4d93aa59b3585b0b6be75ab9b2fbead34fb8aaba457d9085a32973"} Jul 08 13:27:42.984638 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:42.984613 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sfrfw" event={"ID":"15f6cf7d-af37-41ff-a35e-9757e3d29f6b","Type":"ContainerStarted","Data":"d1acbfc43fb1e82df0d6551d763d269067d67feef477b80d35180418dae904d8"} Jul 08 13:27:46.285764 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:46.285721 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:27:46.286178 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:46.285820 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:27:46.289358 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:46.289325 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:27:46.289448 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:46.289389 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:27:46.289448 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:46.289417 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:27:47.006248 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:47.006209 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678lqnzl" event={"ID":"3db34d8c-4001-4369-8dc6-2e6e00fcb4e1","Type":"ContainerStarted","Data":"813a487abf4458306444bb2354614022e25e7da4c7197d704b4c814bb36f1674"} Jul 08 13:27:47.007736 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:47.007708 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sfrfw" event={"ID":"15f6cf7d-af37-41ff-a35e-9757e3d29f6b","Type":"ContainerStarted","Data":"03f49dd4e1e16b39c2777e2b43876156201fa24f96bf4a4bb0abdfa3ac86938a"} Jul 08 13:27:47.007891 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:47.007855 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sfrfw" Jul 08 13:27:47.009623 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:47.009602 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sfrfw" Jul 08 13:27:47.031344 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:47.031295 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678lqnzl" podStartSLOduration=1.328329989 podStartE2EDuration="5.031283731s" totalImagesPullingTime="3.702953742s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:27:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:27:47.029728503 +0000 UTC m=+275.826669172" watchObservedRunningTime="2026-07-08 13:27:47.031283731 +0000 UTC m=+275.828224388" Jul 08 13:27:47.053412 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:47.053365 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-sfrfw" podStartSLOduration=1.391836026 podStartE2EDuration="5.053350705s" totalImagesPullingTime="3.661514679s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:27:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:27:47.052464356 +0000 UTC m=+275.849405014" watchObservedRunningTime="2026-07-08 13:27:47.053350705 +0000 UTC m=+275.850291360" Jul 08 13:27:47.425362 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:47.425259 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678lqnzl" Jul 08 13:27:47.430296 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:47.430267 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678lqnzl" Jul 08 13:27:48.012188 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:48.012157 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678lqnzl" Jul 08 13:27:48.012991 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:27:48.012974 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678lqnzl" Jul 08 13:28:11.656485 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:11.656446 2567 kubelet.go:1648] "Image garbage collection succeeded" Jul 08 13:28:51.156221 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:51.156189 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzvzk"] Jul 08 13:28:51.158265 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:51.158250 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzvzk" Jul 08 13:28:51.160701 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:51.160676 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 08 13:28:51.160836 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:51.160721 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 08 13:28:51.161723 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:51.161700 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-4zqwg\"" Jul 08 13:28:51.167122 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:51.167101 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzvzk"] Jul 08 13:28:51.351168 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:51.351138 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kk2r7\" (UniqueName: \"kubernetes.io/projected/16c5fe64-76fb-4a59-b11f-93e409225a1f-kube-api-access-kk2r7\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzvzk\" (UID: \"16c5fe64-76fb-4a59-b11f-93e409225a1f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzvzk" Jul 08 13:28:51.351318 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:51.351190 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/16c5fe64-76fb-4a59-b11f-93e409225a1f-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzvzk\" (UID: \"16c5fe64-76fb-4a59-b11f-93e409225a1f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzvzk" Jul 08 13:28:51.351318 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:51.351280 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/16c5fe64-76fb-4a59-b11f-93e409225a1f-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzvzk\" (UID: \"16c5fe64-76fb-4a59-b11f-93e409225a1f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzvzk" Jul 08 13:28:51.452770 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:51.452679 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/16c5fe64-76fb-4a59-b11f-93e409225a1f-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzvzk\" (UID: \"16c5fe64-76fb-4a59-b11f-93e409225a1f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzvzk" Jul 08 13:28:51.452770 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:51.452726 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/16c5fe64-76fb-4a59-b11f-93e409225a1f-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzvzk\" (UID: \"16c5fe64-76fb-4a59-b11f-93e409225a1f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzvzk" Jul 08 13:28:51.452991 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:51.452781 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kk2r7\" (UniqueName: \"kubernetes.io/projected/16c5fe64-76fb-4a59-b11f-93e409225a1f-kube-api-access-kk2r7\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzvzk\" (UID: \"16c5fe64-76fb-4a59-b11f-93e409225a1f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzvzk" Jul 08 13:28:51.453191 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:51.453168 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/16c5fe64-76fb-4a59-b11f-93e409225a1f-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzvzk\" (UID: \"16c5fe64-76fb-4a59-b11f-93e409225a1f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzvzk" Jul 08 13:28:51.453276 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:51.453177 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/16c5fe64-76fb-4a59-b11f-93e409225a1f-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzvzk\" (UID: \"16c5fe64-76fb-4a59-b11f-93e409225a1f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzvzk" Jul 08 13:28:51.461577 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:51.461553 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kk2r7\" (UniqueName: \"kubernetes.io/projected/16c5fe64-76fb-4a59-b11f-93e409225a1f-kube-api-access-kk2r7\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzvzk\" (UID: \"16c5fe64-76fb-4a59-b11f-93e409225a1f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzvzk" Jul 08 13:28:51.467180 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:51.467159 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzvzk" Jul 08 13:28:51.595028 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:51.594997 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzvzk"] Jul 08 13:28:51.596595 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:28:51.596567 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16c5fe64_76fb_4a59_b11f_93e409225a1f.slice/crio-aabb3cf5d97e657e43f29caba9263a2234550c82b4d9cd39f3d33768609f5f98 WatchSource:0}: Error finding container aabb3cf5d97e657e43f29caba9263a2234550c82b4d9cd39f3d33768609f5f98: Status 404 returned error can't find the container with id aabb3cf5d97e657e43f29caba9263a2234550c82b4d9cd39f3d33768609f5f98 Jul 08 13:28:51.755573 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:51.755539 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qkqv6"] Jul 08 13:28:51.757755 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:51.757737 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qkqv6" Jul 08 13:28:51.767100 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:51.767054 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qkqv6"] Jul 08 13:28:51.856302 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:51.856266 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qkqv6\" (UID: \"ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qkqv6" Jul 08 13:28:51.856472 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:51.856331 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qkqv6\" (UID: \"ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qkqv6" Jul 08 13:28:51.856472 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:51.856382 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ttfg\" (UniqueName: \"kubernetes.io/projected/ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb-kube-api-access-5ttfg\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qkqv6\" (UID: \"ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qkqv6" Jul 08 13:28:51.957242 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:51.957139 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qkqv6\" (UID: \"ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qkqv6" Jul 08 13:28:51.957242 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:51.957219 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ttfg\" (UniqueName: \"kubernetes.io/projected/ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb-kube-api-access-5ttfg\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qkqv6\" (UID: \"ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qkqv6" Jul 08 13:28:51.957468 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:51.957260 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qkqv6\" (UID: \"ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qkqv6" Jul 08 13:28:51.957623 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:51.957596 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qkqv6\" (UID: \"ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qkqv6" Jul 08 13:28:51.957746 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:51.957602 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qkqv6\" (UID: \"ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qkqv6" Jul 08 13:28:51.966418 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:51.966391 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ttfg\" (UniqueName: \"kubernetes.io/projected/ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb-kube-api-access-5ttfg\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qkqv6\" (UID: \"ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qkqv6" Jul 08 13:28:52.071407 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:52.071368 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qkqv6" Jul 08 13:28:52.153624 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:52.153589 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wwwml"] Jul 08 13:28:52.156158 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:52.156135 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wwwml" Jul 08 13:28:52.167499 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:52.167471 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wwwml"] Jul 08 13:28:52.216457 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:52.216432 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qkqv6"] Jul 08 13:28:52.218289 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:28:52.218268 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab99e9a3_0ef3_40c2_92dd_ff2daaba12bb.slice/crio-2b91ef93ab544bf88f8257826fb5354b138c91b6173d9d4c7a286913fe94eeef WatchSource:0}: Error finding container 2b91ef93ab544bf88f8257826fb5354b138c91b6173d9d4c7a286913fe94eeef: Status 404 returned error can't find the container with id 2b91ef93ab544bf88f8257826fb5354b138c91b6173d9d4c7a286913fe94eeef Jul 08 13:28:52.259143 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:52.259110 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/67eb1d4c-685e-45a3-84e2-9d02835f795b-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wwwml\" (UID: \"67eb1d4c-685e-45a3-84e2-9d02835f795b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wwwml" Jul 08 13:28:52.259252 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:52.259234 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68t59\" (UniqueName: \"kubernetes.io/projected/67eb1d4c-685e-45a3-84e2-9d02835f795b-kube-api-access-68t59\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wwwml\" (UID: \"67eb1d4c-685e-45a3-84e2-9d02835f795b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wwwml" Jul 08 13:28:52.259304 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:52.259280 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/67eb1d4c-685e-45a3-84e2-9d02835f795b-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wwwml\" (UID: \"67eb1d4c-685e-45a3-84e2-9d02835f795b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wwwml" Jul 08 13:28:52.296168 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:52.296122 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qkqv6" event={"ID":"ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb","Type":"ContainerStarted","Data":"18331f7ba486c9614b3aebf212325a3aa7ec8a72871bd13e2802ea88a56f1465"} Jul 08 13:28:52.296353 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:52.296188 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qkqv6" event={"ID":"ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb","Type":"ContainerStarted","Data":"2b91ef93ab544bf88f8257826fb5354b138c91b6173d9d4c7a286913fe94eeef"} Jul 08 13:28:52.297899 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:52.297869 2567 generic.go:356] "Generic (PLEG): container finished" podID="16c5fe64-76fb-4a59-b11f-93e409225a1f" containerID="1b8f036459122c6709d51cc3ae651460dc269a410ad27684cd6642521d2b929d" exitCode=0 Jul 08 13:28:52.298027 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:52.297956 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzvzk" event={"ID":"16c5fe64-76fb-4a59-b11f-93e409225a1f","Type":"ContainerDied","Data":"1b8f036459122c6709d51cc3ae651460dc269a410ad27684cd6642521d2b929d"} Jul 08 13:28:52.298027 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:52.298002 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzvzk" event={"ID":"16c5fe64-76fb-4a59-b11f-93e409225a1f","Type":"ContainerStarted","Data":"aabb3cf5d97e657e43f29caba9263a2234550c82b4d9cd39f3d33768609f5f98"} Jul 08 13:28:52.298398 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:52.298382 2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 13:28:52.359753 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:52.359722 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/67eb1d4c-685e-45a3-84e2-9d02835f795b-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wwwml\" (UID: \"67eb1d4c-685e-45a3-84e2-9d02835f795b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wwwml" Jul 08 13:28:52.359907 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:52.359789 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68t59\" (UniqueName: \"kubernetes.io/projected/67eb1d4c-685e-45a3-84e2-9d02835f795b-kube-api-access-68t59\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wwwml\" (UID: \"67eb1d4c-685e-45a3-84e2-9d02835f795b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wwwml" Jul 08 13:28:52.359907 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:52.359830 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/67eb1d4c-685e-45a3-84e2-9d02835f795b-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wwwml\" (UID: \"67eb1d4c-685e-45a3-84e2-9d02835f795b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wwwml" Jul 08 13:28:52.360233 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:52.360213 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/67eb1d4c-685e-45a3-84e2-9d02835f795b-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wwwml\" (UID: \"67eb1d4c-685e-45a3-84e2-9d02835f795b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wwwml" Jul 08 13:28:52.360280 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:52.360222 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/67eb1d4c-685e-45a3-84e2-9d02835f795b-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wwwml\" (UID: \"67eb1d4c-685e-45a3-84e2-9d02835f795b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wwwml" Jul 08 13:28:52.368898 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:52.368868 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-68t59\" (UniqueName: \"kubernetes.io/projected/67eb1d4c-685e-45a3-84e2-9d02835f795b-kube-api-access-68t59\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wwwml\" (UID: \"67eb1d4c-685e-45a3-84e2-9d02835f795b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wwwml" Jul 08 13:28:52.467274 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:52.467194 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wwwml" Jul 08 13:28:52.556804 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:52.556770 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wpdwr"] Jul 08 13:28:52.560051 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:52.560026 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wpdwr" Jul 08 13:28:52.568207 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:52.568138 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wpdwr"] Jul 08 13:28:52.614930 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:52.614903 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wwwml"] Jul 08 13:28:52.616960 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:28:52.616931 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67eb1d4c_685e_45a3_84e2_9d02835f795b.slice/crio-fb6a2a70838ccdf1ffcf9db2892eb2541e687c8550301697c391f5d0a59830ea WatchSource:0}: Error finding container fb6a2a70838ccdf1ffcf9db2892eb2541e687c8550301697c391f5d0a59830ea: Status 404 returned error can't find the container with id fb6a2a70838ccdf1ffcf9db2892eb2541e687c8550301697c391f5d0a59830ea Jul 08 13:28:52.662183 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:52.662153 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f2b8d968-c4b7-4000-a1ee-b71b786d76a8-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wpdwr\" (UID: \"f2b8d968-c4b7-4000-a1ee-b71b786d76a8\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wpdwr" Jul 08 13:28:52.662336 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:52.662192 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ts4k\" (UniqueName: \"kubernetes.io/projected/f2b8d968-c4b7-4000-a1ee-b71b786d76a8-kube-api-access-2ts4k\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wpdwr\" (UID: \"f2b8d968-c4b7-4000-a1ee-b71b786d76a8\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wpdwr" Jul 08 13:28:52.662336 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:52.662253 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f2b8d968-c4b7-4000-a1ee-b71b786d76a8-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wpdwr\" (UID: \"f2b8d968-c4b7-4000-a1ee-b71b786d76a8\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wpdwr" Jul 08 13:28:52.763491 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:52.763450 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f2b8d968-c4b7-4000-a1ee-b71b786d76a8-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wpdwr\" (UID: \"f2b8d968-c4b7-4000-a1ee-b71b786d76a8\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wpdwr" Jul 08 13:28:52.763681 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:52.763545 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f2b8d968-c4b7-4000-a1ee-b71b786d76a8-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wpdwr\" (UID: \"f2b8d968-c4b7-4000-a1ee-b71b786d76a8\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wpdwr" Jul 08 13:28:52.763681 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:52.763578 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ts4k\" (UniqueName: \"kubernetes.io/projected/f2b8d968-c4b7-4000-a1ee-b71b786d76a8-kube-api-access-2ts4k\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wpdwr\" (UID: \"f2b8d968-c4b7-4000-a1ee-b71b786d76a8\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wpdwr" Jul 08 13:28:52.763913 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:52.763889 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f2b8d968-c4b7-4000-a1ee-b71b786d76a8-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wpdwr\" (UID: \"f2b8d968-c4b7-4000-a1ee-b71b786d76a8\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wpdwr" Jul 08 13:28:52.763957 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:52.763902 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f2b8d968-c4b7-4000-a1ee-b71b786d76a8-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wpdwr\" (UID: \"f2b8d968-c4b7-4000-a1ee-b71b786d76a8\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wpdwr" Jul 08 13:28:52.772593 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:52.772570 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ts4k\" (UniqueName: \"kubernetes.io/projected/f2b8d968-c4b7-4000-a1ee-b71b786d76a8-kube-api-access-2ts4k\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wpdwr\" (UID: \"f2b8d968-c4b7-4000-a1ee-b71b786d76a8\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wpdwr" Jul 08 13:28:52.871991 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:52.871953 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wpdwr" Jul 08 13:28:53.004780 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:53.004750 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wpdwr"] Jul 08 13:28:53.007034 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:28:53.006994 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2b8d968_c4b7_4000_a1ee_b71b786d76a8.slice/crio-5ff22dc94c0a24e5ccebc8f5e306e80199c8f001bcafcab3af5fc50d995856f9 WatchSource:0}: Error finding container 5ff22dc94c0a24e5ccebc8f5e306e80199c8f001bcafcab3af5fc50d995856f9: Status 404 returned error can't find the container with id 5ff22dc94c0a24e5ccebc8f5e306e80199c8f001bcafcab3af5fc50d995856f9 Jul 08 13:28:53.304337 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:53.304254 2567 generic.go:356] "Generic (PLEG): container finished" podID="ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb" containerID="18331f7ba486c9614b3aebf212325a3aa7ec8a72871bd13e2802ea88a56f1465" exitCode=0 Jul 08 13:28:53.304753 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:53.304338 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qkqv6" event={"ID":"ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb","Type":"ContainerDied","Data":"18331f7ba486c9614b3aebf212325a3aa7ec8a72871bd13e2802ea88a56f1465"} Jul 08 13:28:53.305889 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:53.305862 2567 generic.go:356] "Generic (PLEG): container finished" podID="67eb1d4c-685e-45a3-84e2-9d02835f795b" containerID="c31dcf555f5740f1c3c474eadbf68194333e02d6c00ec9ee14835233a72b1ce9" exitCode=0 Jul 08 13:28:53.306018 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:53.305926 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wwwml" event={"ID":"67eb1d4c-685e-45a3-84e2-9d02835f795b","Type":"ContainerDied","Data":"c31dcf555f5740f1c3c474eadbf68194333e02d6c00ec9ee14835233a72b1ce9"} Jul 08 13:28:53.306018 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:53.305973 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wwwml" event={"ID":"67eb1d4c-685e-45a3-84e2-9d02835f795b","Type":"ContainerStarted","Data":"fb6a2a70838ccdf1ffcf9db2892eb2541e687c8550301697c391f5d0a59830ea"} Jul 08 13:28:53.307498 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:53.307467 2567 generic.go:356] "Generic (PLEG): container finished" podID="f2b8d968-c4b7-4000-a1ee-b71b786d76a8" containerID="43149f4ba2a4d948c3964a7e911aed4dcdf725734ff5511a465db24ebdd70dd3" exitCode=0 Jul 08 13:28:53.307620 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:53.307497 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wpdwr" event={"ID":"f2b8d968-c4b7-4000-a1ee-b71b786d76a8","Type":"ContainerDied","Data":"43149f4ba2a4d948c3964a7e911aed4dcdf725734ff5511a465db24ebdd70dd3"} Jul 08 13:28:53.307620 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:53.307549 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wpdwr" event={"ID":"f2b8d968-c4b7-4000-a1ee-b71b786d76a8","Type":"ContainerStarted","Data":"5ff22dc94c0a24e5ccebc8f5e306e80199c8f001bcafcab3af5fc50d995856f9"} Jul 08 13:28:53.309398 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:53.309376 2567 generic.go:356] "Generic (PLEG): container finished" podID="16c5fe64-76fb-4a59-b11f-93e409225a1f" containerID="979725f146aaaf162e459793f8288388d6d1d01b267db634eac2517dca5c2fc9" exitCode=0 Jul 08 13:28:53.309485 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:53.309438 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzvzk" event={"ID":"16c5fe64-76fb-4a59-b11f-93e409225a1f","Type":"ContainerDied","Data":"979725f146aaaf162e459793f8288388d6d1d01b267db634eac2517dca5c2fc9"} Jul 08 13:28:54.319692 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:54.319667 2567 generic.go:356] "Generic (PLEG): container finished" podID="16c5fe64-76fb-4a59-b11f-93e409225a1f" containerID="c9048abca78264fbc7ed2ef2b60901a2cc7b9a62ad87afe58c0cdfb54e752db8" exitCode=0 Jul 08 13:28:54.319974 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:54.319733 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzvzk" event={"ID":"16c5fe64-76fb-4a59-b11f-93e409225a1f","Type":"ContainerDied","Data":"c9048abca78264fbc7ed2ef2b60901a2cc7b9a62ad87afe58c0cdfb54e752db8"} Jul 08 13:28:55.326188 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:55.326153 2567 generic.go:356] "Generic (PLEG): container finished" podID="ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb" containerID="9b933da075e6f6e9f339a1f37b011ebe631208c6c275a2b4535680b6209f8822" exitCode=0 Jul 08 13:28:55.326601 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:55.326221 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qkqv6" event={"ID":"ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb","Type":"ContainerDied","Data":"9b933da075e6f6e9f339a1f37b011ebe631208c6c275a2b4535680b6209f8822"} Jul 08 13:28:55.328252 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:55.328223 2567 generic.go:356] "Generic (PLEG): container finished" podID="67eb1d4c-685e-45a3-84e2-9d02835f795b" containerID="26567e70faa7916cbf90376b9cdaa456c800417572956084f31b10da4c1691d3" exitCode=0 Jul 08 13:28:55.328378 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:55.328315 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wwwml" event={"ID":"67eb1d4c-685e-45a3-84e2-9d02835f795b","Type":"ContainerDied","Data":"26567e70faa7916cbf90376b9cdaa456c800417572956084f31b10da4c1691d3"} Jul 08 13:28:55.330237 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:55.330214 2567 generic.go:356] "Generic (PLEG): container finished" podID="f2b8d968-c4b7-4000-a1ee-b71b786d76a8" containerID="eb97f4c1163914d77668fbf78e453a7511daf7c896accc37d723e8de4aeb8f70" exitCode=0 Jul 08 13:28:55.330351 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:55.330331 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wpdwr" event={"ID":"f2b8d968-c4b7-4000-a1ee-b71b786d76a8","Type":"ContainerDied","Data":"eb97f4c1163914d77668fbf78e453a7511daf7c896accc37d723e8de4aeb8f70"} Jul 08 13:28:55.480371 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:55.480040 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzvzk" Jul 08 13:28:55.484863 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:55.484780 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/16c5fe64-76fb-4a59-b11f-93e409225a1f-bundle\") pod \"16c5fe64-76fb-4a59-b11f-93e409225a1f\" (UID: \"16c5fe64-76fb-4a59-b11f-93e409225a1f\") " Jul 08 13:28:55.484863 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:55.484843 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kk2r7\" (UniqueName: \"kubernetes.io/projected/16c5fe64-76fb-4a59-b11f-93e409225a1f-kube-api-access-kk2r7\") pod \"16c5fe64-76fb-4a59-b11f-93e409225a1f\" (UID: \"16c5fe64-76fb-4a59-b11f-93e409225a1f\") " Jul 08 13:28:55.485053 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:55.484872 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/16c5fe64-76fb-4a59-b11f-93e409225a1f-util\") pod \"16c5fe64-76fb-4a59-b11f-93e409225a1f\" (UID: \"16c5fe64-76fb-4a59-b11f-93e409225a1f\") " Jul 08 13:28:55.485306 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:55.485278 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16c5fe64-76fb-4a59-b11f-93e409225a1f-bundle" (OuterVolumeSpecName: "bundle") pod "16c5fe64-76fb-4a59-b11f-93e409225a1f" (UID: "16c5fe64-76fb-4a59-b11f-93e409225a1f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:28:55.487296 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:55.487271 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16c5fe64-76fb-4a59-b11f-93e409225a1f-kube-api-access-kk2r7" (OuterVolumeSpecName: "kube-api-access-kk2r7") pod "16c5fe64-76fb-4a59-b11f-93e409225a1f" (UID: "16c5fe64-76fb-4a59-b11f-93e409225a1f"). InnerVolumeSpecName "kube-api-access-kk2r7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 13:28:55.491664 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:55.490936 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16c5fe64-76fb-4a59-b11f-93e409225a1f-util" (OuterVolumeSpecName: "util") pod "16c5fe64-76fb-4a59-b11f-93e409225a1f" (UID: "16c5fe64-76fb-4a59-b11f-93e409225a1f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:28:55.586203 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:55.586176 2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/16c5fe64-76fb-4a59-b11f-93e409225a1f-util\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:28:55.586203 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:55.586200 2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/16c5fe64-76fb-4a59-b11f-93e409225a1f-bundle\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:28:55.586339 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:55.586209 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kk2r7\" (UniqueName: \"kubernetes.io/projected/16c5fe64-76fb-4a59-b11f-93e409225a1f-kube-api-access-kk2r7\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:28:56.336907 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:56.336874 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzvzk" Jul 08 13:28:56.336907 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:56.336901 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzvzk" event={"ID":"16c5fe64-76fb-4a59-b11f-93e409225a1f","Type":"ContainerDied","Data":"aabb3cf5d97e657e43f29caba9263a2234550c82b4d9cd39f3d33768609f5f98"} Jul 08 13:28:56.337419 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:56.336950 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aabb3cf5d97e657e43f29caba9263a2234550c82b4d9cd39f3d33768609f5f98" Jul 08 13:28:56.339157 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:56.339129 2567 generic.go:356] "Generic (PLEG): container finished" podID="ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb" containerID="bfaab8f675301b418b9e6f17e3c6c38b049212f13211ee55ac4835d6dc7fe25b" exitCode=0 Jul 08 13:28:56.339298 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:56.339184 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qkqv6" event={"ID":"ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb","Type":"ContainerDied","Data":"bfaab8f675301b418b9e6f17e3c6c38b049212f13211ee55ac4835d6dc7fe25b"} Jul 08 13:28:56.341169 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:56.341148 2567 generic.go:356] "Generic (PLEG): container finished" podID="67eb1d4c-685e-45a3-84e2-9d02835f795b" containerID="b7fbcebf394eb58d1f8c48f352ecc92938ef7965d72f26371eaab925551ac72f" exitCode=0 Jul 08 13:28:56.341257 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:56.341220 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wwwml" event={"ID":"67eb1d4c-685e-45a3-84e2-9d02835f795b","Type":"ContainerDied","Data":"b7fbcebf394eb58d1f8c48f352ecc92938ef7965d72f26371eaab925551ac72f"} Jul 08 13:28:56.343221 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:56.343197 2567 generic.go:356] "Generic (PLEG): container finished" podID="f2b8d968-c4b7-4000-a1ee-b71b786d76a8" containerID="97df2f3cb8da410df5980b0219896243d97c885cbf4136002bb08bf5a1c1373d" exitCode=0 Jul 08 13:28:56.343311 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:56.343229 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wpdwr" event={"ID":"f2b8d968-c4b7-4000-a1ee-b71b786d76a8","Type":"ContainerDied","Data":"97df2f3cb8da410df5980b0219896243d97c885cbf4136002bb08bf5a1c1373d"} Jul 08 13:28:57.499600 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:57.499570 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wpdwr" Jul 08 13:28:57.536025 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:57.536000 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wwwml" Jul 08 13:28:57.538665 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:57.538639 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qkqv6" Jul 08 13:28:57.601334 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:57.601242 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-68t59\" (UniqueName: \"kubernetes.io/projected/67eb1d4c-685e-45a3-84e2-9d02835f795b-kube-api-access-68t59\") pod \"67eb1d4c-685e-45a3-84e2-9d02835f795b\" (UID: \"67eb1d4c-685e-45a3-84e2-9d02835f795b\") " Jul 08 13:28:57.601334 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:57.601313 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f2b8d968-c4b7-4000-a1ee-b71b786d76a8-bundle\") pod \"f2b8d968-c4b7-4000-a1ee-b71b786d76a8\" (UID: \"f2b8d968-c4b7-4000-a1ee-b71b786d76a8\") " Jul 08 13:28:57.601334 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:57.601338 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2ts4k\" (UniqueName: \"kubernetes.io/projected/f2b8d968-c4b7-4000-a1ee-b71b786d76a8-kube-api-access-2ts4k\") pod \"f2b8d968-c4b7-4000-a1ee-b71b786d76a8\" (UID: \"f2b8d968-c4b7-4000-a1ee-b71b786d76a8\") " Jul 08 13:28:57.601594 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:57.601360 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/67eb1d4c-685e-45a3-84e2-9d02835f795b-bundle\") pod \"67eb1d4c-685e-45a3-84e2-9d02835f795b\" (UID: \"67eb1d4c-685e-45a3-84e2-9d02835f795b\") " Jul 08 13:28:57.601594 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:57.601382 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5ttfg\" (UniqueName: \"kubernetes.io/projected/ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb-kube-api-access-5ttfg\") pod \"ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb\" (UID: \"ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb\") " Jul 08 13:28:57.601594 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:57.601412 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/67eb1d4c-685e-45a3-84e2-9d02835f795b-util\") pod \"67eb1d4c-685e-45a3-84e2-9d02835f795b\" (UID: \"67eb1d4c-685e-45a3-84e2-9d02835f795b\") " Jul 08 13:28:57.601594 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:57.601456 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb-util\") pod \"ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb\" (UID: \"ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb\") " Jul 08 13:28:57.601594 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:57.601481 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f2b8d968-c4b7-4000-a1ee-b71b786d76a8-util\") pod \"f2b8d968-c4b7-4000-a1ee-b71b786d76a8\" (UID: \"f2b8d968-c4b7-4000-a1ee-b71b786d76a8\") " Jul 08 13:28:57.601594 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:57.601509 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb-bundle\") pod \"ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb\" (UID: \"ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb\") " Jul 08 13:28:57.602151 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:57.602064 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67eb1d4c-685e-45a3-84e2-9d02835f795b-bundle" (OuterVolumeSpecName: "bundle") pod "67eb1d4c-685e-45a3-84e2-9d02835f795b" (UID: "67eb1d4c-685e-45a3-84e2-9d02835f795b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:28:57.602434 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:57.602185 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb-bundle" (OuterVolumeSpecName: "bundle") pod "ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb" (UID: "ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:28:57.602434 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:57.602225 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2b8d968-c4b7-4000-a1ee-b71b786d76a8-bundle" (OuterVolumeSpecName: "bundle") pod "f2b8d968-c4b7-4000-a1ee-b71b786d76a8" (UID: "f2b8d968-c4b7-4000-a1ee-b71b786d76a8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:28:57.603718 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:57.603689 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2b8d968-c4b7-4000-a1ee-b71b786d76a8-kube-api-access-2ts4k" (OuterVolumeSpecName: "kube-api-access-2ts4k") pod "f2b8d968-c4b7-4000-a1ee-b71b786d76a8" (UID: "f2b8d968-c4b7-4000-a1ee-b71b786d76a8"). InnerVolumeSpecName "kube-api-access-2ts4k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 13:28:57.603962 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:57.603939 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb-kube-api-access-5ttfg" (OuterVolumeSpecName: "kube-api-access-5ttfg") pod "ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb" (UID: "ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb"). InnerVolumeSpecName "kube-api-access-5ttfg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 13:28:57.604035 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:57.603958 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67eb1d4c-685e-45a3-84e2-9d02835f795b-kube-api-access-68t59" (OuterVolumeSpecName: "kube-api-access-68t59") pod "67eb1d4c-685e-45a3-84e2-9d02835f795b" (UID: "67eb1d4c-685e-45a3-84e2-9d02835f795b"). InnerVolumeSpecName "kube-api-access-68t59". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 13:28:57.607309 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:57.607260 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb-util" (OuterVolumeSpecName: "util") pod "ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb" (UID: "ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:28:57.608573 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:57.608543 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2b8d968-c4b7-4000-a1ee-b71b786d76a8-util" (OuterVolumeSpecName: "util") pod "f2b8d968-c4b7-4000-a1ee-b71b786d76a8" (UID: "f2b8d968-c4b7-4000-a1ee-b71b786d76a8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:28:57.608702 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:57.608676 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67eb1d4c-685e-45a3-84e2-9d02835f795b-util" (OuterVolumeSpecName: "util") pod "67eb1d4c-685e-45a3-84e2-9d02835f795b" (UID: "67eb1d4c-685e-45a3-84e2-9d02835f795b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:28:57.702595 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:57.702559 2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f2b8d968-c4b7-4000-a1ee-b71b786d76a8-bundle\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:28:57.702595 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:57.702591 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2ts4k\" (UniqueName: \"kubernetes.io/projected/f2b8d968-c4b7-4000-a1ee-b71b786d76a8-kube-api-access-2ts4k\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:28:57.702595 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:57.702601 2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/67eb1d4c-685e-45a3-84e2-9d02835f795b-bundle\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:28:57.702816 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:57.702610 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5ttfg\" (UniqueName: \"kubernetes.io/projected/ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb-kube-api-access-5ttfg\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:28:57.702816 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:57.702620 2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/67eb1d4c-685e-45a3-84e2-9d02835f795b-util\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:28:57.702816 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:57.702628 2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb-util\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:28:57.702816 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:57.702635 2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f2b8d968-c4b7-4000-a1ee-b71b786d76a8-util\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:28:57.702816 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:57.702642 2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb-bundle\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:28:57.702816 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:57.702650 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-68t59\" (UniqueName: \"kubernetes.io/projected/67eb1d4c-685e-45a3-84e2-9d02835f795b-kube-api-access-68t59\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:28:58.354126 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:58.354087 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qkqv6" event={"ID":"ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb","Type":"ContainerDied","Data":"2b91ef93ab544bf88f8257826fb5354b138c91b6173d9d4c7a286913fe94eeef"} Jul 08 13:28:58.354126 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:58.354120 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qkqv6" Jul 08 13:28:58.354333 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:58.354137 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2b91ef93ab544bf88f8257826fb5354b138c91b6173d9d4c7a286913fe94eeef" Jul 08 13:28:58.356025 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:58.355967 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wwwml" Jul 08 13:28:58.356226 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:58.355968 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wwwml" event={"ID":"67eb1d4c-685e-45a3-84e2-9d02835f795b","Type":"ContainerDied","Data":"fb6a2a70838ccdf1ffcf9db2892eb2541e687c8550301697c391f5d0a59830ea"} Jul 08 13:28:58.356316 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:58.356271 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fb6a2a70838ccdf1ffcf9db2892eb2541e687c8550301697c391f5d0a59830ea" Jul 08 13:28:58.358181 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:58.358150 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wpdwr" Jul 08 13:28:58.358181 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:58.358165 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wpdwr" event={"ID":"f2b8d968-c4b7-4000-a1ee-b71b786d76a8","Type":"ContainerDied","Data":"5ff22dc94c0a24e5ccebc8f5e306e80199c8f001bcafcab3af5fc50d995856f9"} Jul 08 13:28:58.358299 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:28:58.358212 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5ff22dc94c0a24e5ccebc8f5e306e80199c8f001bcafcab3af5fc50d995856f9" Jul 08 13:30:27.744578 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:30:27.744542 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:30:27.744998 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:30:27.744849 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67eb1d4c-685e-45a3-84e2-9d02835f795b" containerName="util" Jul 08 13:30:27.744998 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:30:27.744860 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="67eb1d4c-685e-45a3-84e2-9d02835f795b" containerName="util" Jul 08 13:30:27.744998 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:30:27.744870 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67eb1d4c-685e-45a3-84e2-9d02835f795b" containerName="extract" Jul 08 13:30:27.744998 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:30:27.744875 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="67eb1d4c-685e-45a3-84e2-9d02835f795b" containerName="extract" Jul 08 13:30:27.744998 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:30:27.744884 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16c5fe64-76fb-4a59-b11f-93e409225a1f" containerName="pull" Jul 08 13:30:27.744998 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:30:27.744889 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="16c5fe64-76fb-4a59-b11f-93e409225a1f" containerName="pull" Jul 08 13:30:27.744998 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:30:27.744895 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16c5fe64-76fb-4a59-b11f-93e409225a1f" containerName="extract" Jul 08 13:30:27.744998 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:30:27.744901 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="16c5fe64-76fb-4a59-b11f-93e409225a1f" containerName="extract" Jul 08 13:30:27.744998 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:30:27.744907 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb" containerName="util" Jul 08 13:30:27.744998 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:30:27.744913 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb" containerName="util" Jul 08 13:30:27.744998 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:30:27.744918 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb" containerName="pull" Jul 08 13:30:27.744998 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:30:27.744923 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb" containerName="pull" Jul 08 13:30:27.744998 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:30:27.744934 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2b8d968-c4b7-4000-a1ee-b71b786d76a8" containerName="extract" Jul 08 13:30:27.744998 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:30:27.744939 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2b8d968-c4b7-4000-a1ee-b71b786d76a8" containerName="extract" Jul 08 13:30:27.744998 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:30:27.744947 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2b8d968-c4b7-4000-a1ee-b71b786d76a8" containerName="pull" Jul 08 13:30:27.744998 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:30:27.744951 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2b8d968-c4b7-4000-a1ee-b71b786d76a8" containerName="pull" Jul 08 13:30:27.744998 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:30:27.744957 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16c5fe64-76fb-4a59-b11f-93e409225a1f" containerName="util" Jul 08 13:30:27.744998 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:30:27.744962 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="16c5fe64-76fb-4a59-b11f-93e409225a1f" containerName="util" Jul 08 13:30:27.744998 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:30:27.744967 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67eb1d4c-685e-45a3-84e2-9d02835f795b" containerName="pull" Jul 08 13:30:27.744998 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:30:27.744971 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="67eb1d4c-685e-45a3-84e2-9d02835f795b" containerName="pull" Jul 08 13:30:27.744998 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:30:27.744977 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2b8d968-c4b7-4000-a1ee-b71b786d76a8" containerName="util" Jul 08 13:30:27.744998 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:30:27.744981 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2b8d968-c4b7-4000-a1ee-b71b786d76a8" containerName="util" Jul 08 13:30:27.744998 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:30:27.744986 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb" containerName="extract" Jul 08 13:30:27.744998 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:30:27.744991 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb" containerName="extract" Jul 08 13:30:27.745749 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:30:27.745036 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb" containerName="extract" Jul 08 13:30:27.745749 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:30:27.745044 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2b8d968-c4b7-4000-a1ee-b71b786d76a8" containerName="extract" Jul 08 13:30:27.745749 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:30:27.745051 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="16c5fe64-76fb-4a59-b11f-93e409225a1f" containerName="extract" Jul 08 13:30:27.745749 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:30:27.745057 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="67eb1d4c-685e-45a3-84e2-9d02835f795b" containerName="extract" Jul 08 13:30:27.748148 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:30:27.748130 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-hpf2w" Jul 08 13:30:27.750598 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:30:27.750574 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 08 13:30:27.750711 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:30:27.750595 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 08 13:30:27.751529 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:30:27.751506 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-4zqwg\"" Jul 08 13:30:27.751649 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:30:27.751509 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 08 13:30:27.758507 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:30:27.758482 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:30:27.785692 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:30:27.785661 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:30:27.873689 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:30:27.873648 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ae3f52b5-288d-4736-bfb5-f4999246d155-config-file\") pod \"limitador-limitador-54d6d988d5-hpf2w\" (UID: \"ae3f52b5-288d-4736-bfb5-f4999246d155\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-hpf2w" Jul 08 13:30:27.873689 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:30:27.873692 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8cqj\" (UniqueName: \"kubernetes.io/projected/ae3f52b5-288d-4736-bfb5-f4999246d155-kube-api-access-w8cqj\") pod \"limitador-limitador-54d6d988d5-hpf2w\" (UID: \"ae3f52b5-288d-4736-bfb5-f4999246d155\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-hpf2w" Jul 08 13:30:27.974939 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:30:27.974897 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8cqj\" (UniqueName: \"kubernetes.io/projected/ae3f52b5-288d-4736-bfb5-f4999246d155-kube-api-access-w8cqj\") pod \"limitador-limitador-54d6d988d5-hpf2w\" (UID: \"ae3f52b5-288d-4736-bfb5-f4999246d155\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-hpf2w" Jul 08 13:30:27.975173 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:30:27.975018 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ae3f52b5-288d-4736-bfb5-f4999246d155-config-file\") pod \"limitador-limitador-54d6d988d5-hpf2w\" (UID: \"ae3f52b5-288d-4736-bfb5-f4999246d155\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-hpf2w" Jul 08 13:30:27.975653 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:30:27.975629 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ae3f52b5-288d-4736-bfb5-f4999246d155-config-file\") pod \"limitador-limitador-54d6d988d5-hpf2w\" (UID: \"ae3f52b5-288d-4736-bfb5-f4999246d155\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-hpf2w" Jul 08 13:30:27.983810 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:30:27.983776 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8cqj\" (UniqueName: \"kubernetes.io/projected/ae3f52b5-288d-4736-bfb5-f4999246d155-kube-api-access-w8cqj\") pod \"limitador-limitador-54d6d988d5-hpf2w\" (UID: \"ae3f52b5-288d-4736-bfb5-f4999246d155\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-hpf2w" Jul 08 13:30:28.059287 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:30:28.059182 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-hpf2w" Jul 08 13:30:28.191840 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:30:28.191814 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:30:28.194493 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:30:28.194464 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae3f52b5_288d_4736_bfb5_f4999246d155.slice/crio-4e6f6964f22d8fa9aa4227a96f7279db2a25b1889b6631ef53321443c8e6af11 WatchSource:0}: Error finding container 4e6f6964f22d8fa9aa4227a96f7279db2a25b1889b6631ef53321443c8e6af11: Status 404 returned error can't find the container with id 4e6f6964f22d8fa9aa4227a96f7279db2a25b1889b6631ef53321443c8e6af11 Jul 08 13:30:28.793631 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:30:28.793481 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-hpf2w" event={"ID":"ae3f52b5-288d-4736-bfb5-f4999246d155","Type":"ContainerStarted","Data":"4e6f6964f22d8fa9aa4227a96f7279db2a25b1889b6631ef53321443c8e6af11"} Jul 08 13:30:30.804639 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:30:30.804600 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-hpf2w" event={"ID":"ae3f52b5-288d-4736-bfb5-f4999246d155","Type":"ContainerStarted","Data":"931d06b9d1662b95f30baeee6bd338e89e468e6dd0a984a9d533d2a50cbf5599"} Jul 08 13:30:30.804987 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:30:30.804771 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-hpf2w" Jul 08 13:30:30.829560 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:30:30.829517 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-hpf2w" podStartSLOduration=1.331547371 podStartE2EDuration="3.829503554s" totalImagesPullingTime="2.497956183s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:30:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:30:30.827567272 +0000 UTC m=+439.624507928" watchObservedRunningTime="2026-07-08 13:30:30.829503554 +0000 UTC m=+439.626444210" Jul 08 13:30:41.809476 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:30:41.809394 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-hpf2w" Jul 08 13:31:40.615842 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:31:40.615809 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:31:57.587907 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:31:57.587870 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:32:05.796690 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:32:05.796612 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:32:13.591320 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:32:13.591282 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:32:17.988688 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:32:17.988654 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:32:21.395867 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:32:21.395831 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-cn54g"] Jul 08 13:32:21.399160 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:32:21.399143 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cn54g" Jul 08 13:32:21.401974 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:32:21.401949 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-2p2fd\"" Jul 08 13:32:21.403169 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:32:21.403148 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 08 13:32:21.403169 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:32:21.403154 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 08 13:32:21.403325 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:32:21.403238 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 08 13:32:21.413031 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:32:21.413001 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-cn54g"] Jul 08 13:32:21.506530 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:32:21.506492 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f738043a-0ae8-4e6f-ab4c-c8acbafa822a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cn54g\" (UID: \"f738043a-0ae8-4e6f-ab4c-c8acbafa822a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cn54g" Jul 08 13:32:21.506693 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:32:21.506538 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f738043a-0ae8-4e6f-ab4c-c8acbafa822a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cn54g\" (UID: \"f738043a-0ae8-4e6f-ab4c-c8acbafa822a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cn54g" Jul 08 13:32:21.506693 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:32:21.506566 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f738043a-0ae8-4e6f-ab4c-c8acbafa822a-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cn54g\" (UID: \"f738043a-0ae8-4e6f-ab4c-c8acbafa822a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cn54g" Jul 08 13:32:21.506693 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:32:21.506643 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4b2s\" (UniqueName: \"kubernetes.io/projected/f738043a-0ae8-4e6f-ab4c-c8acbafa822a-kube-api-access-r4b2s\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cn54g\" (UID: \"f738043a-0ae8-4e6f-ab4c-c8acbafa822a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cn54g" Jul 08 13:32:21.506693 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:32:21.506689 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f738043a-0ae8-4e6f-ab4c-c8acbafa822a-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cn54g\" (UID: \"f738043a-0ae8-4e6f-ab4c-c8acbafa822a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cn54g" Jul 08 13:32:21.506830 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:32:21.506744 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f738043a-0ae8-4e6f-ab4c-c8acbafa822a-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cn54g\" (UID: \"f738043a-0ae8-4e6f-ab4c-c8acbafa822a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cn54g" Jul 08 13:32:21.607878 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:32:21.607842 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f738043a-0ae8-4e6f-ab4c-c8acbafa822a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cn54g\" (UID: \"f738043a-0ae8-4e6f-ab4c-c8acbafa822a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cn54g" Jul 08 13:32:21.608065 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:32:21.607886 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f738043a-0ae8-4e6f-ab4c-c8acbafa822a-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cn54g\" (UID: \"f738043a-0ae8-4e6f-ab4c-c8acbafa822a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cn54g" Jul 08 13:32:21.608065 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:32:21.607918 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4b2s\" (UniqueName: \"kubernetes.io/projected/f738043a-0ae8-4e6f-ab4c-c8acbafa822a-kube-api-access-r4b2s\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cn54g\" (UID: \"f738043a-0ae8-4e6f-ab4c-c8acbafa822a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cn54g" Jul 08 13:32:21.608065 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:32:21.607953 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f738043a-0ae8-4e6f-ab4c-c8acbafa822a-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cn54g\" (UID: \"f738043a-0ae8-4e6f-ab4c-c8acbafa822a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cn54g" Jul 08 13:32:21.608065 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:32:21.607990 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f738043a-0ae8-4e6f-ab4c-c8acbafa822a-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cn54g\" (UID: \"f738043a-0ae8-4e6f-ab4c-c8acbafa822a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cn54g" Jul 08 13:32:21.608374 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:32:21.608071 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f738043a-0ae8-4e6f-ab4c-c8acbafa822a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cn54g\" (UID: \"f738043a-0ae8-4e6f-ab4c-c8acbafa822a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cn54g" Jul 08 13:32:21.608374 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:32:21.608273 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f738043a-0ae8-4e6f-ab4c-c8acbafa822a-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cn54g\" (UID: \"f738043a-0ae8-4e6f-ab4c-c8acbafa822a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cn54g" Jul 08 13:32:21.608374 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:32:21.608365 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f738043a-0ae8-4e6f-ab4c-c8acbafa822a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cn54g\" (UID: \"f738043a-0ae8-4e6f-ab4c-c8acbafa822a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cn54g" Jul 08 13:32:21.608477 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:32:21.608405 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f738043a-0ae8-4e6f-ab4c-c8acbafa822a-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cn54g\" (UID: \"f738043a-0ae8-4e6f-ab4c-c8acbafa822a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cn54g" Jul 08 13:32:21.610314 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:32:21.610289 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f738043a-0ae8-4e6f-ab4c-c8acbafa822a-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cn54g\" (UID: \"f738043a-0ae8-4e6f-ab4c-c8acbafa822a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cn54g" Jul 08 13:32:21.610453 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:32:21.610437 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f738043a-0ae8-4e6f-ab4c-c8acbafa822a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cn54g\" (UID: \"f738043a-0ae8-4e6f-ab4c-c8acbafa822a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cn54g" Jul 08 13:32:21.615539 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:32:21.615514 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4b2s\" (UniqueName: \"kubernetes.io/projected/f738043a-0ae8-4e6f-ab4c-c8acbafa822a-kube-api-access-r4b2s\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cn54g\" (UID: \"f738043a-0ae8-4e6f-ab4c-c8acbafa822a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cn54g" Jul 08 13:32:21.709794 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:32:21.709711 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cn54g" Jul 08 13:32:21.840806 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:32:21.840783 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-cn54g"] Jul 08 13:32:21.842400 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:32:21.842369 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf738043a_0ae8_4e6f_ab4c_c8acbafa822a.slice/crio-6b65210abf728906e1153aac1ab86e9b8e6b5aae6707622a998cc60435118d8c WatchSource:0}: Error finding container 6b65210abf728906e1153aac1ab86e9b8e6b5aae6707622a998cc60435118d8c: Status 404 returned error can't find the container with id 6b65210abf728906e1153aac1ab86e9b8e6b5aae6707622a998cc60435118d8c Jul 08 13:32:22.329824 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:32:22.329774 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cn54g" event={"ID":"f738043a-0ae8-4e6f-ab4c-c8acbafa822a","Type":"ContainerStarted","Data":"6b65210abf728906e1153aac1ab86e9b8e6b5aae6707622a998cc60435118d8c"} Jul 08 13:32:23.096107 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:32:23.096058 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:32:23.335549 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:32:23.335513 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cn54g" event={"ID":"f738043a-0ae8-4e6f-ab4c-c8acbafa822a","Type":"ContainerStarted","Data":"d62fb84f41ebd6902e888ab433ea33aeaf8e6dfcf445b0f6e8ff3c3a8ce43489"} Jul 08 13:32:23.353479 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:32:23.353423 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cn54g" podStartSLOduration=1.307957605 podStartE2EDuration="2.353407986s" totalImagesPullingTime="1.045450381s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:32:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:32:23.352533925 +0000 UTC m=+552.149474583" watchObservedRunningTime="2026-07-08 13:32:23.353407986 +0000 UTC m=+552.150348641" Jul 08 13:32:31.709944 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:32:31.709906 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cn54g" Jul 08 13:32:31.710431 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:32:31.709970 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cn54g" Jul 08 13:32:31.722407 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:32:31.722384 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cn54g" Jul 08 13:32:32.389714 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:32:32.389683 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cn54g" Jul 08 13:32:39.590597 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:32:39.590563 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:34:50.593765 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:34:50.593725 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:34:59.784525 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:34:59.784489 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:35:11.789231 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:35:11.789151 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:35:19.988644 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:35:19.988611 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:35:31.398128 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:35:31.398071 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:35:58.883944 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:35:58.883906 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:36:10.990175 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:36:10.990136 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:36:43.197968 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:36:43.197933 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:36:52.793690 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:36:52.793649 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:37:02.490397 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:37:02.490362 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:37:09.996351 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:37:09.996316 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:37:21.096536 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:37:21.096500 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:37:30.387521 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:37:30.387486 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:37:41.796419 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:37:41.796388 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:37:58.288366 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:37:58.288329 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:38:05.796489 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:38:05.796405 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:38:14.789809 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:38:14.789770 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:38:40.604440 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:38:40.604407 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:38:49.689965 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:38:49.689928 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:39:15.400107 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:39:15.400052 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:39:25.999578 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:39:25.999542 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:39:41.590701 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:39:41.590624 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:39:53.393892 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:39:53.393860 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:40:02.985595 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:40:02.985556 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:40:12.292739 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:40:12.292691 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:40:21.707357 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:40:21.707328 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:42:09.784925 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:42:09.784892 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:42:21.003301 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:42:21.003268 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:42:29.590698 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:42:29.590661 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:42:41.996322 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:42:41.996236 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:42:51.582474 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:42:51.582434 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:43:02.988700 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:43:02.988663 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:43:12.989586 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:43:12.989544 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:43:23.493357 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:43:23.493322 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:43:33.991845 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:43:33.991806 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:43:42.891486 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:43:42.891448 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:43:53.589987 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:43:53.589957 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:44:15.602658 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:44:15.602620 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:44:24.799010 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:44:24.798976 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:44:34.387060 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:44:34.387024 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:44:45.794408 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:44:45.790460 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:44:53.702107 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:44:53.702060 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:45:05.294403 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:45:05.294367 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:45:15.994677 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:45:15.994640 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:45:25.194459 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:45:25.194422 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:45:34.691589 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:45:34.691508 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:45:45.988294 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:45:45.988257 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:45:55.488430 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:45:55.488400 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:46:06.161023 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:46:06.160983 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:46:17.587183 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:46:17.587143 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:46:26.990365 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:46:26.990331 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:46:36.094610 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:46:36.094576 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:46:48.687975 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:46:48.687943 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:46:56.691015 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:46:56.690979 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:47:08.794567 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:08.794486 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:47:20.694896 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:20.694862 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:47:31.284976 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:31.284936 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:47:42.474219 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:42.474183 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk"] Jul 08 13:47:42.477695 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:42.477673 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk" Jul 08 13:47:42.480202 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:42.480179 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-72d0489f-openshift-default-dockercfg-7hcw5\"" Jul 08 13:47:42.492524 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:42.492492 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk"] Jul 08 13:47:42.515068 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:42.515024 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eaf9ef50-d192-4249-afe8-5b687be822f3-workload-certs\") pod \"e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk\" (UID: \"eaf9ef50-d192-4249-afe8-5b687be822f3\") " pod="openshift-ingress/e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk" Jul 08 13:47:42.515255 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:42.515100 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stj6t\" (UniqueName: \"kubernetes.io/projected/eaf9ef50-d192-4249-afe8-5b687be822f3-kube-api-access-stj6t\") pod \"e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk\" (UID: \"eaf9ef50-d192-4249-afe8-5b687be822f3\") " pod="openshift-ingress/e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk" Jul 08 13:47:42.515255 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:42.515161 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eaf9ef50-d192-4249-afe8-5b687be822f3-istio-podinfo\") pod \"e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk\" (UID: \"eaf9ef50-d192-4249-afe8-5b687be822f3\") " pod="openshift-ingress/e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk" Jul 08 13:47:42.515255 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:42.515196 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eaf9ef50-d192-4249-afe8-5b687be822f3-istio-envoy\") pod \"e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk\" (UID: \"eaf9ef50-d192-4249-afe8-5b687be822f3\") " pod="openshift-ingress/e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk" Jul 08 13:47:42.515255 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:42.515234 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eaf9ef50-d192-4249-afe8-5b687be822f3-workload-socket\") pod \"e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk\" (UID: \"eaf9ef50-d192-4249-afe8-5b687be822f3\") " pod="openshift-ingress/e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk" Jul 08 13:47:42.515390 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:42.515263 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eaf9ef50-d192-4249-afe8-5b687be822f3-credential-socket\") pod \"e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk\" (UID: \"eaf9ef50-d192-4249-afe8-5b687be822f3\") " pod="openshift-ingress/e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk" Jul 08 13:47:42.515390 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:42.515299 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eaf9ef50-d192-4249-afe8-5b687be822f3-istiod-ca-cert\") pod \"e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk\" (UID: \"eaf9ef50-d192-4249-afe8-5b687be822f3\") " pod="openshift-ingress/e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk" Jul 08 13:47:42.515390 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:42.515363 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eaf9ef50-d192-4249-afe8-5b687be822f3-istio-token\") pod \"e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk\" (UID: \"eaf9ef50-d192-4249-afe8-5b687be822f3\") " pod="openshift-ingress/e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk" Jul 08 13:47:42.515481 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:42.515405 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eaf9ef50-d192-4249-afe8-5b687be822f3-istio-data\") pod \"e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk\" (UID: \"eaf9ef50-d192-4249-afe8-5b687be822f3\") " pod="openshift-ingress/e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk" Jul 08 13:47:42.616218 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:42.616171 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eaf9ef50-d192-4249-afe8-5b687be822f3-workload-socket\") pod \"e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk\" (UID: \"eaf9ef50-d192-4249-afe8-5b687be822f3\") " pod="openshift-ingress/e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk" Jul 08 13:47:42.616218 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:42.616216 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eaf9ef50-d192-4249-afe8-5b687be822f3-credential-socket\") pod \"e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk\" (UID: \"eaf9ef50-d192-4249-afe8-5b687be822f3\") " pod="openshift-ingress/e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk" Jul 08 13:47:42.616451 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:42.616236 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eaf9ef50-d192-4249-afe8-5b687be822f3-istiod-ca-cert\") pod \"e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk\" (UID: \"eaf9ef50-d192-4249-afe8-5b687be822f3\") " pod="openshift-ingress/e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk" Jul 08 13:47:42.616451 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:42.616263 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eaf9ef50-d192-4249-afe8-5b687be822f3-istio-token\") pod \"e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk\" (UID: \"eaf9ef50-d192-4249-afe8-5b687be822f3\") " pod="openshift-ingress/e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk" Jul 08 13:47:42.616451 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:42.616304 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eaf9ef50-d192-4249-afe8-5b687be822f3-istio-data\") pod \"e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk\" (UID: \"eaf9ef50-d192-4249-afe8-5b687be822f3\") " pod="openshift-ingress/e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk" Jul 08 13:47:42.616451 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:42.616344 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eaf9ef50-d192-4249-afe8-5b687be822f3-workload-certs\") pod \"e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk\" (UID: \"eaf9ef50-d192-4249-afe8-5b687be822f3\") " pod="openshift-ingress/e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk" Jul 08 13:47:42.616451 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:42.616370 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stj6t\" (UniqueName: \"kubernetes.io/projected/eaf9ef50-d192-4249-afe8-5b687be822f3-kube-api-access-stj6t\") pod \"e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk\" (UID: \"eaf9ef50-d192-4249-afe8-5b687be822f3\") " pod="openshift-ingress/e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk" Jul 08 13:47:42.616451 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:42.616407 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eaf9ef50-d192-4249-afe8-5b687be822f3-istio-podinfo\") pod \"e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk\" (UID: \"eaf9ef50-d192-4249-afe8-5b687be822f3\") " pod="openshift-ingress/e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk" Jul 08 13:47:42.616451 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:42.616440 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eaf9ef50-d192-4249-afe8-5b687be822f3-istio-envoy\") pod \"e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk\" (UID: \"eaf9ef50-d192-4249-afe8-5b687be822f3\") " pod="openshift-ingress/e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk" Jul 08 13:47:42.616817 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:42.616623 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eaf9ef50-d192-4249-afe8-5b687be822f3-workload-socket\") pod \"e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk\" (UID: \"eaf9ef50-d192-4249-afe8-5b687be822f3\") " pod="openshift-ingress/e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk" Jul 08 13:47:42.616817 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:42.616739 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eaf9ef50-d192-4249-afe8-5b687be822f3-credential-socket\") pod \"e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk\" (UID: \"eaf9ef50-d192-4249-afe8-5b687be822f3\") " pod="openshift-ingress/e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk" Jul 08 13:47:42.616913 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:42.616816 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eaf9ef50-d192-4249-afe8-5b687be822f3-workload-certs\") pod \"e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk\" (UID: \"eaf9ef50-d192-4249-afe8-5b687be822f3\") " pod="openshift-ingress/e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk" Jul 08 13:47:42.616913 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:42.616838 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eaf9ef50-d192-4249-afe8-5b687be822f3-istio-data\") pod \"e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk\" (UID: \"eaf9ef50-d192-4249-afe8-5b687be822f3\") " pod="openshift-ingress/e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk" Jul 08 13:47:42.617062 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:42.617044 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eaf9ef50-d192-4249-afe8-5b687be822f3-istiod-ca-cert\") pod \"e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk\" (UID: \"eaf9ef50-d192-4249-afe8-5b687be822f3\") " pod="openshift-ingress/e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk" Jul 08 13:47:42.618745 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:42.618724 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eaf9ef50-d192-4249-afe8-5b687be822f3-istio-envoy\") pod \"e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk\" (UID: \"eaf9ef50-d192-4249-afe8-5b687be822f3\") " pod="openshift-ingress/e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk" Jul 08 13:47:42.618834 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:42.618787 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eaf9ef50-d192-4249-afe8-5b687be822f3-istio-podinfo\") pod \"e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk\" (UID: \"eaf9ef50-d192-4249-afe8-5b687be822f3\") " pod="openshift-ingress/e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk" Jul 08 13:47:42.624570 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:42.624542 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eaf9ef50-d192-4249-afe8-5b687be822f3-istio-token\") pod \"e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk\" (UID: \"eaf9ef50-d192-4249-afe8-5b687be822f3\") " pod="openshift-ingress/e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk" Jul 08 13:47:42.624673 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:42.624655 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-stj6t\" (UniqueName: \"kubernetes.io/projected/eaf9ef50-d192-4249-afe8-5b687be822f3-kube-api-access-stj6t\") pod \"e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk\" (UID: \"eaf9ef50-d192-4249-afe8-5b687be822f3\") " pod="openshift-ingress/e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk" Jul 08 13:47:42.788005 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:42.787915 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk" Jul 08 13:47:42.923465 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:42.923443 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk"] Jul 08 13:47:42.925802 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:47:42.925772 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeaf9ef50_d192_4249_afe8_5b687be822f3.slice/crio-7069d8eedb5e77629b2a9b8f90766ed850ae505d1dc438332e71df0d623d3289 WatchSource:0}: Error finding container 7069d8eedb5e77629b2a9b8f90766ed850ae505d1dc438332e71df0d623d3289: Status 404 returned error can't find the container with id 7069d8eedb5e77629b2a9b8f90766ed850ae505d1dc438332e71df0d623d3289 Jul 08 13:47:42.927917 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:42.927886 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:47:42.928006 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:42.927958 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:47:42.928006 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:42.927986 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:47:43.732141 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:43.732097 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk" event={"ID":"eaf9ef50-d192-4249-afe8-5b687be822f3","Type":"ContainerStarted","Data":"44db773adefdb4357f08a0654b6ce19dc552472c1b86511bac696ea34232f214"} Jul 08 13:47:43.732534 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:43.732200 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk" event={"ID":"eaf9ef50-d192-4249-afe8-5b687be822f3","Type":"ContainerStarted","Data":"7069d8eedb5e77629b2a9b8f90766ed850ae505d1dc438332e71df0d623d3289"} Jul 08 13:47:43.754685 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:43.754642 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk" podStartSLOduration=1.754628361 podStartE2EDuration="1.754628361s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:47:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:47:43.753633977 +0000 UTC m=+1472.550574634" watchObservedRunningTime="2026-07-08 13:47:43.754628361 +0000 UTC m=+1472.551569025" Jul 08 13:47:43.788896 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:43.788861 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk" Jul 08 13:47:43.795645 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:43.795622 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk" Jul 08 13:47:44.737800 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:44.737770 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk" Jul 08 13:47:44.738604 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:44.738587 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk" Jul 08 13:47:46.391553 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:46.391515 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:47:48.906778 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:48.906734 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk"] Jul 08 13:47:48.907284 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:48.906999 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk" podUID="eaf9ef50-d192-4249-afe8-5b687be822f3" containerName="istio-proxy" containerID="cri-o://44db773adefdb4357f08a0654b6ce19dc552472c1b86511bac696ea34232f214" gracePeriod=30 Jul 08 13:47:54.155228 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:54.155169 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk" Jul 08 13:47:54.224394 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:54.224319 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eaf9ef50-d192-4249-afe8-5b687be822f3-istio-data\") pod \"eaf9ef50-d192-4249-afe8-5b687be822f3\" (UID: \"eaf9ef50-d192-4249-afe8-5b687be822f3\") " Jul 08 13:47:54.224394 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:54.224361 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eaf9ef50-d192-4249-afe8-5b687be822f3-istio-token\") pod \"eaf9ef50-d192-4249-afe8-5b687be822f3\" (UID: \"eaf9ef50-d192-4249-afe8-5b687be822f3\") " Jul 08 13:47:54.224394 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:54.224392 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eaf9ef50-d192-4249-afe8-5b687be822f3-workload-certs\") pod \"eaf9ef50-d192-4249-afe8-5b687be822f3\" (UID: \"eaf9ef50-d192-4249-afe8-5b687be822f3\") " Jul 08 13:47:54.224653 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:54.224440 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eaf9ef50-d192-4249-afe8-5b687be822f3-workload-socket\") pod \"eaf9ef50-d192-4249-afe8-5b687be822f3\" (UID: \"eaf9ef50-d192-4249-afe8-5b687be822f3\") " Jul 08 13:47:54.224653 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:54.224477 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-stj6t\" (UniqueName: \"kubernetes.io/projected/eaf9ef50-d192-4249-afe8-5b687be822f3-kube-api-access-stj6t\") pod \"eaf9ef50-d192-4249-afe8-5b687be822f3\" (UID: \"eaf9ef50-d192-4249-afe8-5b687be822f3\") " Jul 08 13:47:54.224653 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:54.224514 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eaf9ef50-d192-4249-afe8-5b687be822f3-credential-socket\") pod \"eaf9ef50-d192-4249-afe8-5b687be822f3\" (UID: \"eaf9ef50-d192-4249-afe8-5b687be822f3\") " Jul 08 13:47:54.224653 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:54.224543 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eaf9ef50-d192-4249-afe8-5b687be822f3-istio-envoy\") pod \"eaf9ef50-d192-4249-afe8-5b687be822f3\" (UID: \"eaf9ef50-d192-4249-afe8-5b687be822f3\") " Jul 08 13:47:54.224653 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:54.224574 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eaf9ef50-d192-4249-afe8-5b687be822f3-istiod-ca-cert\") pod \"eaf9ef50-d192-4249-afe8-5b687be822f3\" (UID: \"eaf9ef50-d192-4249-afe8-5b687be822f3\") " Jul 08 13:47:54.224653 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:54.224616 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eaf9ef50-d192-4249-afe8-5b687be822f3-istio-podinfo\") pod \"eaf9ef50-d192-4249-afe8-5b687be822f3\" (UID: \"eaf9ef50-d192-4249-afe8-5b687be822f3\") " Jul 08 13:47:54.224937 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:54.224732 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eaf9ef50-d192-4249-afe8-5b687be822f3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "eaf9ef50-d192-4249-afe8-5b687be822f3" (UID: "eaf9ef50-d192-4249-afe8-5b687be822f3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:47:54.224937 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:54.224746 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eaf9ef50-d192-4249-afe8-5b687be822f3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "eaf9ef50-d192-4249-afe8-5b687be822f3" (UID: "eaf9ef50-d192-4249-afe8-5b687be822f3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:47:54.225052 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:54.224936 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eaf9ef50-d192-4249-afe8-5b687be822f3-workload-certs\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:47:54.225052 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:54.224954 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eaf9ef50-d192-4249-afe8-5b687be822f3-workload-socket\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:47:54.225183 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:54.225058 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eaf9ef50-d192-4249-afe8-5b687be822f3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "eaf9ef50-d192-4249-afe8-5b687be822f3" (UID: "eaf9ef50-d192-4249-afe8-5b687be822f3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:47:54.225183 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:54.225063 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eaf9ef50-d192-4249-afe8-5b687be822f3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "eaf9ef50-d192-4249-afe8-5b687be822f3" (UID: "eaf9ef50-d192-4249-afe8-5b687be822f3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 13:47:54.225183 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:54.225168 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eaf9ef50-d192-4249-afe8-5b687be822f3-istio-data" (OuterVolumeSpecName: "istio-data") pod "eaf9ef50-d192-4249-afe8-5b687be822f3" (UID: "eaf9ef50-d192-4249-afe8-5b687be822f3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:47:54.226914 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:54.226890 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eaf9ef50-d192-4249-afe8-5b687be822f3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "eaf9ef50-d192-4249-afe8-5b687be822f3" (UID: "eaf9ef50-d192-4249-afe8-5b687be822f3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 13:47:54.227031 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:54.226894 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eaf9ef50-d192-4249-afe8-5b687be822f3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "eaf9ef50-d192-4249-afe8-5b687be822f3" (UID: "eaf9ef50-d192-4249-afe8-5b687be822f3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:47:54.227117 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:54.227034 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eaf9ef50-d192-4249-afe8-5b687be822f3-istio-token" (OuterVolumeSpecName: "istio-token") pod "eaf9ef50-d192-4249-afe8-5b687be822f3" (UID: "eaf9ef50-d192-4249-afe8-5b687be822f3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 13:47:54.227117 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:54.227046 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eaf9ef50-d192-4249-afe8-5b687be822f3-kube-api-access-stj6t" (OuterVolumeSpecName: "kube-api-access-stj6t") pod "eaf9ef50-d192-4249-afe8-5b687be822f3" (UID: "eaf9ef50-d192-4249-afe8-5b687be822f3"). InnerVolumeSpecName "kube-api-access-stj6t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 13:47:54.325666 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:54.325631 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eaf9ef50-d192-4249-afe8-5b687be822f3-istiod-ca-cert\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:47:54.325666 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:54.325659 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eaf9ef50-d192-4249-afe8-5b687be822f3-istio-podinfo\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:47:54.325666 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:54.325668 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eaf9ef50-d192-4249-afe8-5b687be822f3-istio-data\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:47:54.325666 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:54.325678 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eaf9ef50-d192-4249-afe8-5b687be822f3-istio-token\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:47:54.325926 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:54.325686 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-stj6t\" (UniqueName: \"kubernetes.io/projected/eaf9ef50-d192-4249-afe8-5b687be822f3-kube-api-access-stj6t\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:47:54.325926 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:54.325696 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eaf9ef50-d192-4249-afe8-5b687be822f3-credential-socket\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:47:54.325926 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:54.325704 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eaf9ef50-d192-4249-afe8-5b687be822f3-istio-envoy\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:47:54.796099 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:54.796049 2567 generic.go:356] "Generic (PLEG): container finished" podID="eaf9ef50-d192-4249-afe8-5b687be822f3" containerID="44db773adefdb4357f08a0654b6ce19dc552472c1b86511bac696ea34232f214" exitCode=0 Jul 08 13:47:54.796271 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:54.796118 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk" Jul 08 13:47:54.796271 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:54.796137 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk" event={"ID":"eaf9ef50-d192-4249-afe8-5b687be822f3","Type":"ContainerDied","Data":"44db773adefdb4357f08a0654b6ce19dc552472c1b86511bac696ea34232f214"} Jul 08 13:47:54.796271 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:54.796187 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk" event={"ID":"eaf9ef50-d192-4249-afe8-5b687be822f3","Type":"ContainerDied","Data":"7069d8eedb5e77629b2a9b8f90766ed850ae505d1dc438332e71df0d623d3289"} Jul 08 13:47:54.796271 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:54.796209 2567 scope.go:117] "RemoveContainer" containerID="44db773adefdb4357f08a0654b6ce19dc552472c1b86511bac696ea34232f214" Jul 08 13:47:54.805532 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:54.805513 2567 scope.go:117] "RemoveContainer" containerID="44db773adefdb4357f08a0654b6ce19dc552472c1b86511bac696ea34232f214" Jul 08 13:47:54.805788 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:47:54.805767 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44db773adefdb4357f08a0654b6ce19dc552472c1b86511bac696ea34232f214\": container with ID starting with 44db773adefdb4357f08a0654b6ce19dc552472c1b86511bac696ea34232f214 not found: ID does not exist" containerID="44db773adefdb4357f08a0654b6ce19dc552472c1b86511bac696ea34232f214" Jul 08 13:47:54.805855 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:54.805801 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44db773adefdb4357f08a0654b6ce19dc552472c1b86511bac696ea34232f214"} err="failed to get container status \"44db773adefdb4357f08a0654b6ce19dc552472c1b86511bac696ea34232f214\": rpc error: code = NotFound desc = could not find container \"44db773adefdb4357f08a0654b6ce19dc552472c1b86511bac696ea34232f214\": container with ID starting with 44db773adefdb4357f08a0654b6ce19dc552472c1b86511bac696ea34232f214 not found: ID does not exist" Jul 08 13:47:54.822142 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:54.822011 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk"] Jul 08 13:47:54.824131 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:54.824106 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-72d0489f-openshift-default-78cdccf7b7-8kzpk"] Jul 08 13:47:54.899421 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:54.899388 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:47:54.991326 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:54.991295 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7"] Jul 08 13:47:54.991951 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:54.991919 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eaf9ef50-d192-4249-afe8-5b687be822f3" containerName="istio-proxy" Jul 08 13:47:54.991951 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:54.991949 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="eaf9ef50-d192-4249-afe8-5b687be822f3" containerName="istio-proxy" Jul 08 13:47:54.992204 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:54.992047 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="eaf9ef50-d192-4249-afe8-5b687be822f3" containerName="istio-proxy" Jul 08 13:47:54.995195 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:54.995174 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7" Jul 08 13:47:54.997801 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:54.997783 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-bf8c325b-openshift-default-dockercfg-7ghth\"" Jul 08 13:47:55.012032 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:55.012011 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7"] Jul 08 13:47:55.132564 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:55.132473 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/701bc622-deaa-41d9-bf99-49c19e47d695-istio-token\") pod \"e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7\" (UID: \"701bc622-deaa-41d9-bf99-49c19e47d695\") " pod="openshift-ingress/e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7" Jul 08 13:47:55.132564 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:55.132539 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/701bc622-deaa-41d9-bf99-49c19e47d695-workload-certs\") pod \"e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7\" (UID: \"701bc622-deaa-41d9-bf99-49c19e47d695\") " pod="openshift-ingress/e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7" Jul 08 13:47:55.132778 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:55.132613 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/701bc622-deaa-41d9-bf99-49c19e47d695-istio-podinfo\") pod \"e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7\" (UID: \"701bc622-deaa-41d9-bf99-49c19e47d695\") " pod="openshift-ingress/e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7" Jul 08 13:47:55.132778 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:55.132650 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/701bc622-deaa-41d9-bf99-49c19e47d695-istio-envoy\") pod \"e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7\" (UID: \"701bc622-deaa-41d9-bf99-49c19e47d695\") " pod="openshift-ingress/e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7" Jul 08 13:47:55.132778 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:55.132678 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/701bc622-deaa-41d9-bf99-49c19e47d695-credential-socket\") pod \"e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7\" (UID: \"701bc622-deaa-41d9-bf99-49c19e47d695\") " pod="openshift-ingress/e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7" Jul 08 13:47:55.132778 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:55.132709 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hg2p4\" (UniqueName: \"kubernetes.io/projected/701bc622-deaa-41d9-bf99-49c19e47d695-kube-api-access-hg2p4\") pod \"e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7\" (UID: \"701bc622-deaa-41d9-bf99-49c19e47d695\") " pod="openshift-ingress/e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7" Jul 08 13:47:55.132778 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:55.132755 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/701bc622-deaa-41d9-bf99-49c19e47d695-istiod-ca-cert\") pod \"e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7\" (UID: \"701bc622-deaa-41d9-bf99-49c19e47d695\") " pod="openshift-ingress/e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7" Jul 08 13:47:55.132972 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:55.132783 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/701bc622-deaa-41d9-bf99-49c19e47d695-workload-socket\") pod \"e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7\" (UID: \"701bc622-deaa-41d9-bf99-49c19e47d695\") " pod="openshift-ingress/e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7" Jul 08 13:47:55.132972 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:55.132800 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/701bc622-deaa-41d9-bf99-49c19e47d695-istio-data\") pod \"e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7\" (UID: \"701bc622-deaa-41d9-bf99-49c19e47d695\") " pod="openshift-ingress/e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7" Jul 08 13:47:55.233387 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:55.233347 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/701bc622-deaa-41d9-bf99-49c19e47d695-credential-socket\") pod \"e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7\" (UID: \"701bc622-deaa-41d9-bf99-49c19e47d695\") " pod="openshift-ingress/e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7" Jul 08 13:47:55.233855 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:55.233399 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hg2p4\" (UniqueName: \"kubernetes.io/projected/701bc622-deaa-41d9-bf99-49c19e47d695-kube-api-access-hg2p4\") pod \"e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7\" (UID: \"701bc622-deaa-41d9-bf99-49c19e47d695\") " pod="openshift-ingress/e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7" Jul 08 13:47:55.233855 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:55.233429 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/701bc622-deaa-41d9-bf99-49c19e47d695-istiod-ca-cert\") pod \"e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7\" (UID: \"701bc622-deaa-41d9-bf99-49c19e47d695\") " pod="openshift-ingress/e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7" Jul 08 13:47:55.233855 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:55.233463 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/701bc622-deaa-41d9-bf99-49c19e47d695-workload-socket\") pod \"e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7\" (UID: \"701bc622-deaa-41d9-bf99-49c19e47d695\") " pod="openshift-ingress/e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7" Jul 08 13:47:55.233855 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:55.233487 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/701bc622-deaa-41d9-bf99-49c19e47d695-istio-data\") pod \"e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7\" (UID: \"701bc622-deaa-41d9-bf99-49c19e47d695\") " pod="openshift-ingress/e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7" Jul 08 13:47:55.233855 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:55.233641 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/701bc622-deaa-41d9-bf99-49c19e47d695-istio-token\") pod \"e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7\" (UID: \"701bc622-deaa-41d9-bf99-49c19e47d695\") " pod="openshift-ingress/e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7" Jul 08 13:47:55.233855 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:55.233728 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/701bc622-deaa-41d9-bf99-49c19e47d695-workload-certs\") pod \"e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7\" (UID: \"701bc622-deaa-41d9-bf99-49c19e47d695\") " pod="openshift-ingress/e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7" Jul 08 13:47:55.233855 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:55.233783 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/701bc622-deaa-41d9-bf99-49c19e47d695-istio-podinfo\") pod \"e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7\" (UID: \"701bc622-deaa-41d9-bf99-49c19e47d695\") " pod="openshift-ingress/e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7" Jul 08 13:47:55.233855 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:55.233811 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/701bc622-deaa-41d9-bf99-49c19e47d695-credential-socket\") pod \"e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7\" (UID: \"701bc622-deaa-41d9-bf99-49c19e47d695\") " pod="openshift-ingress/e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7" Jul 08 13:47:55.233855 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:55.233812 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/701bc622-deaa-41d9-bf99-49c19e47d695-istio-envoy\") pod \"e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7\" (UID: \"701bc622-deaa-41d9-bf99-49c19e47d695\") " pod="openshift-ingress/e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7" Jul 08 13:47:55.233855 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:55.233850 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/701bc622-deaa-41d9-bf99-49c19e47d695-istio-data\") pod \"e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7\" (UID: \"701bc622-deaa-41d9-bf99-49c19e47d695\") " pod="openshift-ingress/e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7" Jul 08 13:47:55.234308 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:55.234143 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/701bc622-deaa-41d9-bf99-49c19e47d695-workload-socket\") pod \"e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7\" (UID: \"701bc622-deaa-41d9-bf99-49c19e47d695\") " pod="openshift-ingress/e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7" Jul 08 13:47:55.234308 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:55.234168 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/701bc622-deaa-41d9-bf99-49c19e47d695-workload-certs\") pod \"e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7\" (UID: \"701bc622-deaa-41d9-bf99-49c19e47d695\") " pod="openshift-ingress/e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7" Jul 08 13:47:55.234308 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:55.234280 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/701bc622-deaa-41d9-bf99-49c19e47d695-istiod-ca-cert\") pod \"e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7\" (UID: \"701bc622-deaa-41d9-bf99-49c19e47d695\") " pod="openshift-ingress/e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7" Jul 08 13:47:55.235966 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:55.235942 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/701bc622-deaa-41d9-bf99-49c19e47d695-istio-envoy\") pod \"e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7\" (UID: \"701bc622-deaa-41d9-bf99-49c19e47d695\") " pod="openshift-ingress/e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7" Jul 08 13:47:55.236097 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:55.236064 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/701bc622-deaa-41d9-bf99-49c19e47d695-istio-podinfo\") pod \"e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7\" (UID: \"701bc622-deaa-41d9-bf99-49c19e47d695\") " pod="openshift-ingress/e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7" Jul 08 13:47:55.241224 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:55.241201 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hg2p4\" (UniqueName: \"kubernetes.io/projected/701bc622-deaa-41d9-bf99-49c19e47d695-kube-api-access-hg2p4\") pod \"e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7\" (UID: \"701bc622-deaa-41d9-bf99-49c19e47d695\") " pod="openshift-ingress/e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7" Jul 08 13:47:55.241637 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:55.241617 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/701bc622-deaa-41d9-bf99-49c19e47d695-istio-token\") pod \"e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7\" (UID: \"701bc622-deaa-41d9-bf99-49c19e47d695\") " pod="openshift-ingress/e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7" Jul 08 13:47:55.305398 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:55.305362 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7" Jul 08 13:47:55.435904 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:55.435879 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7"] Jul 08 13:47:55.437706 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:47:55.437680 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod701bc622_deaa_41d9_bf99_49c19e47d695.slice/crio-5789bcf4ad31b27f955e83a47d3e62c91b3af48e4a1d1687c8c27a60857b576a WatchSource:0}: Error finding container 5789bcf4ad31b27f955e83a47d3e62c91b3af48e4a1d1687c8c27a60857b576a: Status 404 returned error can't find the container with id 5789bcf4ad31b27f955e83a47d3e62c91b3af48e4a1d1687c8c27a60857b576a Jul 08 13:47:55.439722 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:55.439678 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:47:55.439792 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:55.439752 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:47:55.439792 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:55.439782 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:47:55.795569 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:55.795526 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eaf9ef50-d192-4249-afe8-5b687be822f3" path="/var/lib/kubelet/pods/eaf9ef50-d192-4249-afe8-5b687be822f3/volumes" Jul 08 13:47:55.802538 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:55.802502 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7" event={"ID":"701bc622-deaa-41d9-bf99-49c19e47d695","Type":"ContainerStarted","Data":"923f2bbced7282f5a6e673888f1ece281ae949ad36e26eec670be4e2845b2a4b"} Jul 08 13:47:55.802692 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:55.802559 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7" event={"ID":"701bc622-deaa-41d9-bf99-49c19e47d695","Type":"ContainerStarted","Data":"5789bcf4ad31b27f955e83a47d3e62c91b3af48e4a1d1687c8c27a60857b576a"} Jul 08 13:47:55.823124 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:55.823059 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7" podStartSLOduration=1.823043282 podStartE2EDuration="1.823043282s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:47:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:47:55.820753372 +0000 UTC m=+1484.617694051" watchObservedRunningTime="2026-07-08 13:47:55.823043282 +0000 UTC m=+1484.619983937" Jul 08 13:47:56.306258 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:56.306221 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7" Jul 08 13:47:56.312305 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:56.312264 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7" Jul 08 13:47:56.808152 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:56.808123 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7" Jul 08 13:47:56.809194 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:47:56.809170 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7" Jul 08 13:48:07.373612 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:07.373572 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7"] Jul 08 13:48:07.374030 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:07.373799 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7" podUID="701bc622-deaa-41d9-bf99-49c19e47d695" containerName="istio-proxy" containerID="cri-o://923f2bbced7282f5a6e673888f1ece281ae949ad36e26eec670be4e2845b2a4b" gracePeriod=30 Jul 08 13:48:08.026377 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:08.026352 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7" Jul 08 13:48:08.152967 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:08.152930 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/701bc622-deaa-41d9-bf99-49c19e47d695-credential-socket\") pod \"701bc622-deaa-41d9-bf99-49c19e47d695\" (UID: \"701bc622-deaa-41d9-bf99-49c19e47d695\") " Jul 08 13:48:08.153176 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:08.153008 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/701bc622-deaa-41d9-bf99-49c19e47d695-istio-token\") pod \"701bc622-deaa-41d9-bf99-49c19e47d695\" (UID: \"701bc622-deaa-41d9-bf99-49c19e47d695\") " Jul 08 13:48:08.153176 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:08.153037 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/701bc622-deaa-41d9-bf99-49c19e47d695-istio-envoy\") pod \"701bc622-deaa-41d9-bf99-49c19e47d695\" (UID: \"701bc622-deaa-41d9-bf99-49c19e47d695\") " Jul 08 13:48:08.153176 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:08.153096 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/701bc622-deaa-41d9-bf99-49c19e47d695-workload-certs\") pod \"701bc622-deaa-41d9-bf99-49c19e47d695\" (UID: \"701bc622-deaa-41d9-bf99-49c19e47d695\") " Jul 08 13:48:08.153176 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:08.153127 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/701bc622-deaa-41d9-bf99-49c19e47d695-workload-socket\") pod \"701bc622-deaa-41d9-bf99-49c19e47d695\" (UID: \"701bc622-deaa-41d9-bf99-49c19e47d695\") " Jul 08 13:48:08.153176 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:08.153154 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hg2p4\" (UniqueName: \"kubernetes.io/projected/701bc622-deaa-41d9-bf99-49c19e47d695-kube-api-access-hg2p4\") pod \"701bc622-deaa-41d9-bf99-49c19e47d695\" (UID: \"701bc622-deaa-41d9-bf99-49c19e47d695\") " Jul 08 13:48:08.153456 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:08.153194 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/701bc622-deaa-41d9-bf99-49c19e47d695-istio-podinfo\") pod \"701bc622-deaa-41d9-bf99-49c19e47d695\" (UID: \"701bc622-deaa-41d9-bf99-49c19e47d695\") " Jul 08 13:48:08.153456 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:08.153212 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/701bc622-deaa-41d9-bf99-49c19e47d695-istio-data\") pod \"701bc622-deaa-41d9-bf99-49c19e47d695\" (UID: \"701bc622-deaa-41d9-bf99-49c19e47d695\") " Jul 08 13:48:08.153456 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:08.153249 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/701bc622-deaa-41d9-bf99-49c19e47d695-istiod-ca-cert\") pod \"701bc622-deaa-41d9-bf99-49c19e47d695\" (UID: \"701bc622-deaa-41d9-bf99-49c19e47d695\") " Jul 08 13:48:08.153621 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:08.153511 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/701bc622-deaa-41d9-bf99-49c19e47d695-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "701bc622-deaa-41d9-bf99-49c19e47d695" (UID: "701bc622-deaa-41d9-bf99-49c19e47d695"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:48:08.153621 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:08.153519 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/701bc622-deaa-41d9-bf99-49c19e47d695-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "701bc622-deaa-41d9-bf99-49c19e47d695" (UID: "701bc622-deaa-41d9-bf99-49c19e47d695"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:48:08.153719 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:08.153614 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/701bc622-deaa-41d9-bf99-49c19e47d695-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "701bc622-deaa-41d9-bf99-49c19e47d695" (UID: "701bc622-deaa-41d9-bf99-49c19e47d695"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:48:08.153887 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:08.153862 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/701bc622-deaa-41d9-bf99-49c19e47d695-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "701bc622-deaa-41d9-bf99-49c19e47d695" (UID: "701bc622-deaa-41d9-bf99-49c19e47d695"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 13:48:08.154068 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:08.154038 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/701bc622-deaa-41d9-bf99-49c19e47d695-istio-data" (OuterVolumeSpecName: "istio-data") pod "701bc622-deaa-41d9-bf99-49c19e47d695" (UID: "701bc622-deaa-41d9-bf99-49c19e47d695"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:48:08.155493 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:08.155466 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/701bc622-deaa-41d9-bf99-49c19e47d695-istio-token" (OuterVolumeSpecName: "istio-token") pod "701bc622-deaa-41d9-bf99-49c19e47d695" (UID: "701bc622-deaa-41d9-bf99-49c19e47d695"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 13:48:08.155493 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:08.155481 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/701bc622-deaa-41d9-bf99-49c19e47d695-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "701bc622-deaa-41d9-bf99-49c19e47d695" (UID: "701bc622-deaa-41d9-bf99-49c19e47d695"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:48:08.155622 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:08.155488 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/701bc622-deaa-41d9-bf99-49c19e47d695-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "701bc622-deaa-41d9-bf99-49c19e47d695" (UID: "701bc622-deaa-41d9-bf99-49c19e47d695"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 13:48:08.155622 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:08.155553 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/701bc622-deaa-41d9-bf99-49c19e47d695-kube-api-access-hg2p4" (OuterVolumeSpecName: "kube-api-access-hg2p4") pod "701bc622-deaa-41d9-bf99-49c19e47d695" (UID: "701bc622-deaa-41d9-bf99-49c19e47d695"). InnerVolumeSpecName "kube-api-access-hg2p4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 13:48:08.254209 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:08.254170 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hg2p4\" (UniqueName: \"kubernetes.io/projected/701bc622-deaa-41d9-bf99-49c19e47d695-kube-api-access-hg2p4\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:48:08.254209 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:08.254204 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/701bc622-deaa-41d9-bf99-49c19e47d695-istio-podinfo\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:48:08.254209 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:08.254213 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/701bc622-deaa-41d9-bf99-49c19e47d695-istio-data\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:48:08.254209 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:08.254221 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/701bc622-deaa-41d9-bf99-49c19e47d695-istiod-ca-cert\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:48:08.254484 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:08.254229 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/701bc622-deaa-41d9-bf99-49c19e47d695-credential-socket\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:48:08.254484 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:08.254237 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/701bc622-deaa-41d9-bf99-49c19e47d695-istio-token\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:48:08.254484 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:08.254245 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/701bc622-deaa-41d9-bf99-49c19e47d695-istio-envoy\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:48:08.254484 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:08.254254 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/701bc622-deaa-41d9-bf99-49c19e47d695-workload-certs\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:48:08.254484 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:08.254262 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/701bc622-deaa-41d9-bf99-49c19e47d695-workload-socket\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:48:08.868374 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:08.868338 2567 generic.go:356] "Generic (PLEG): container finished" podID="701bc622-deaa-41d9-bf99-49c19e47d695" containerID="923f2bbced7282f5a6e673888f1ece281ae949ad36e26eec670be4e2845b2a4b" exitCode=0 Jul 08 13:48:08.868787 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:08.868384 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7" Jul 08 13:48:08.868787 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:08.868422 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7" event={"ID":"701bc622-deaa-41d9-bf99-49c19e47d695","Type":"ContainerDied","Data":"923f2bbced7282f5a6e673888f1ece281ae949ad36e26eec670be4e2845b2a4b"} Jul 08 13:48:08.868787 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:08.868480 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7" event={"ID":"701bc622-deaa-41d9-bf99-49c19e47d695","Type":"ContainerDied","Data":"5789bcf4ad31b27f955e83a47d3e62c91b3af48e4a1d1687c8c27a60857b576a"} Jul 08 13:48:08.868787 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:08.868502 2567 scope.go:117] "RemoveContainer" containerID="923f2bbced7282f5a6e673888f1ece281ae949ad36e26eec670be4e2845b2a4b" Jul 08 13:48:08.877730 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:08.877713 2567 scope.go:117] "RemoveContainer" containerID="923f2bbced7282f5a6e673888f1ece281ae949ad36e26eec670be4e2845b2a4b" Jul 08 13:48:08.877973 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:48:08.877954 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"923f2bbced7282f5a6e673888f1ece281ae949ad36e26eec670be4e2845b2a4b\": container with ID starting with 923f2bbced7282f5a6e673888f1ece281ae949ad36e26eec670be4e2845b2a4b not found: ID does not exist" containerID="923f2bbced7282f5a6e673888f1ece281ae949ad36e26eec670be4e2845b2a4b" Jul 08 13:48:08.878034 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:08.877985 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"923f2bbced7282f5a6e673888f1ece281ae949ad36e26eec670be4e2845b2a4b"} err="failed to get container status \"923f2bbced7282f5a6e673888f1ece281ae949ad36e26eec670be4e2845b2a4b\": rpc error: code = NotFound desc = could not find container \"923f2bbced7282f5a6e673888f1ece281ae949ad36e26eec670be4e2845b2a4b\": container with ID starting with 923f2bbced7282f5a6e673888f1ece281ae949ad36e26eec670be4e2845b2a4b not found: ID does not exist" Jul 08 13:48:08.891999 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:08.891970 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7"] Jul 08 13:48:08.895638 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:08.895612 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-bf8c325b-openshift-default-779d854c9c-wjwm7"] Jul 08 13:48:09.795653 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:09.795613 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="701bc622-deaa-41d9-bf99-49c19e47d695" path="/var/lib/kubelet/pods/701bc622-deaa-41d9-bf99-49c19e47d695/volumes" Jul 08 13:48:13.319796 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:13.319760 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd"] Jul 08 13:48:13.320369 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:13.320185 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="701bc622-deaa-41d9-bf99-49c19e47d695" containerName="istio-proxy" Jul 08 13:48:13.320369 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:13.320201 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="701bc622-deaa-41d9-bf99-49c19e47d695" containerName="istio-proxy" Jul 08 13:48:13.320369 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:13.320274 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="701bc622-deaa-41d9-bf99-49c19e47d695" containerName="istio-proxy" Jul 08 13:48:13.322893 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:13.322870 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd" Jul 08 13:48:13.325575 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:13.325552 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-50abd907-openshift-default-dockercfg-h9vz4\"" Jul 08 13:48:13.335647 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:13.335624 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd"] Jul 08 13:48:13.499372 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:13.499339 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/845b52f7-50ef-4eb2-abfd-500fa621bf48-istio-podinfo\") pod \"e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd\" (UID: \"845b52f7-50ef-4eb2-abfd-500fa621bf48\") " pod="openshift-ingress/e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd" Jul 08 13:48:13.499372 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:13.499376 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/845b52f7-50ef-4eb2-abfd-500fa621bf48-credential-socket\") pod \"e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd\" (UID: \"845b52f7-50ef-4eb2-abfd-500fa621bf48\") " pod="openshift-ingress/e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd" Jul 08 13:48:13.499579 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:13.499402 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/845b52f7-50ef-4eb2-abfd-500fa621bf48-istio-envoy\") pod \"e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd\" (UID: \"845b52f7-50ef-4eb2-abfd-500fa621bf48\") " pod="openshift-ingress/e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd" Jul 08 13:48:13.499579 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:13.499476 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/845b52f7-50ef-4eb2-abfd-500fa621bf48-workload-certs\") pod \"e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd\" (UID: \"845b52f7-50ef-4eb2-abfd-500fa621bf48\") " pod="openshift-ingress/e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd" Jul 08 13:48:13.499579 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:13.499525 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/845b52f7-50ef-4eb2-abfd-500fa621bf48-istiod-ca-cert\") pod \"e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd\" (UID: \"845b52f7-50ef-4eb2-abfd-500fa621bf48\") " pod="openshift-ingress/e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd" Jul 08 13:48:13.499579 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:13.499558 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fc5p\" (UniqueName: \"kubernetes.io/projected/845b52f7-50ef-4eb2-abfd-500fa621bf48-kube-api-access-8fc5p\") pod \"e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd\" (UID: \"845b52f7-50ef-4eb2-abfd-500fa621bf48\") " pod="openshift-ingress/e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd" Jul 08 13:48:13.499707 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:13.499592 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/845b52f7-50ef-4eb2-abfd-500fa621bf48-workload-socket\") pod \"e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd\" (UID: \"845b52f7-50ef-4eb2-abfd-500fa621bf48\") " pod="openshift-ingress/e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd" Jul 08 13:48:13.499707 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:13.499639 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/845b52f7-50ef-4eb2-abfd-500fa621bf48-istio-data\") pod \"e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd\" (UID: \"845b52f7-50ef-4eb2-abfd-500fa621bf48\") " pod="openshift-ingress/e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd" Jul 08 13:48:13.499707 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:13.499660 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/845b52f7-50ef-4eb2-abfd-500fa621bf48-istio-token\") pod \"e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd\" (UID: \"845b52f7-50ef-4eb2-abfd-500fa621bf48\") " pod="openshift-ingress/e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd" Jul 08 13:48:13.600555 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:13.600479 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/845b52f7-50ef-4eb2-abfd-500fa621bf48-credential-socket\") pod \"e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd\" (UID: \"845b52f7-50ef-4eb2-abfd-500fa621bf48\") " pod="openshift-ingress/e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd" Jul 08 13:48:13.600555 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:13.600522 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/845b52f7-50ef-4eb2-abfd-500fa621bf48-istio-envoy\") pod \"e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd\" (UID: \"845b52f7-50ef-4eb2-abfd-500fa621bf48\") " pod="openshift-ingress/e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd" Jul 08 13:48:13.600555 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:13.600543 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/845b52f7-50ef-4eb2-abfd-500fa621bf48-workload-certs\") pod \"e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd\" (UID: \"845b52f7-50ef-4eb2-abfd-500fa621bf48\") " pod="openshift-ingress/e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd" Jul 08 13:48:13.600760 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:13.600580 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/845b52f7-50ef-4eb2-abfd-500fa621bf48-istiod-ca-cert\") pod \"e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd\" (UID: \"845b52f7-50ef-4eb2-abfd-500fa621bf48\") " pod="openshift-ingress/e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd" Jul 08 13:48:13.600760 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:13.600624 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fc5p\" (UniqueName: \"kubernetes.io/projected/845b52f7-50ef-4eb2-abfd-500fa621bf48-kube-api-access-8fc5p\") pod \"e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd\" (UID: \"845b52f7-50ef-4eb2-abfd-500fa621bf48\") " pod="openshift-ingress/e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd" Jul 08 13:48:13.600864 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:13.600780 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/845b52f7-50ef-4eb2-abfd-500fa621bf48-workload-socket\") pod \"e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd\" (UID: \"845b52f7-50ef-4eb2-abfd-500fa621bf48\") " pod="openshift-ingress/e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd" Jul 08 13:48:13.600864 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:13.600848 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/845b52f7-50ef-4eb2-abfd-500fa621bf48-istio-data\") pod \"e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd\" (UID: \"845b52f7-50ef-4eb2-abfd-500fa621bf48\") " pod="openshift-ingress/e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd" Jul 08 13:48:13.600962 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:13.600878 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/845b52f7-50ef-4eb2-abfd-500fa621bf48-istio-token\") pod \"e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd\" (UID: \"845b52f7-50ef-4eb2-abfd-500fa621bf48\") " pod="openshift-ingress/e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd" Jul 08 13:48:13.600962 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:13.600905 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/845b52f7-50ef-4eb2-abfd-500fa621bf48-credential-socket\") pod \"e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd\" (UID: \"845b52f7-50ef-4eb2-abfd-500fa621bf48\") " pod="openshift-ingress/e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd" Jul 08 13:48:13.600962 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:13.600930 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/845b52f7-50ef-4eb2-abfd-500fa621bf48-istio-podinfo\") pod \"e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd\" (UID: \"845b52f7-50ef-4eb2-abfd-500fa621bf48\") " pod="openshift-ingress/e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd" Jul 08 13:48:13.601155 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:13.600957 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/845b52f7-50ef-4eb2-abfd-500fa621bf48-workload-certs\") pod \"e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd\" (UID: \"845b52f7-50ef-4eb2-abfd-500fa621bf48\") " pod="openshift-ingress/e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd" Jul 08 13:48:13.601155 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:13.601121 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/845b52f7-50ef-4eb2-abfd-500fa621bf48-workload-socket\") pod \"e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd\" (UID: \"845b52f7-50ef-4eb2-abfd-500fa621bf48\") " pod="openshift-ingress/e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd" Jul 08 13:48:13.601269 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:13.601169 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/845b52f7-50ef-4eb2-abfd-500fa621bf48-istio-data\") pod \"e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd\" (UID: \"845b52f7-50ef-4eb2-abfd-500fa621bf48\") " pod="openshift-ingress/e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd" Jul 08 13:48:13.601347 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:13.601327 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/845b52f7-50ef-4eb2-abfd-500fa621bf48-istiod-ca-cert\") pod \"e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd\" (UID: \"845b52f7-50ef-4eb2-abfd-500fa621bf48\") " pod="openshift-ingress/e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd" Jul 08 13:48:13.602814 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:13.602790 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/845b52f7-50ef-4eb2-abfd-500fa621bf48-istio-envoy\") pod \"e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd\" (UID: \"845b52f7-50ef-4eb2-abfd-500fa621bf48\") " pod="openshift-ingress/e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd" Jul 08 13:48:13.603283 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:13.603261 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/845b52f7-50ef-4eb2-abfd-500fa621bf48-istio-podinfo\") pod \"e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd\" (UID: \"845b52f7-50ef-4eb2-abfd-500fa621bf48\") " pod="openshift-ingress/e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd" Jul 08 13:48:13.608686 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:13.608661 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fc5p\" (UniqueName: \"kubernetes.io/projected/845b52f7-50ef-4eb2-abfd-500fa621bf48-kube-api-access-8fc5p\") pod \"e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd\" (UID: \"845b52f7-50ef-4eb2-abfd-500fa621bf48\") " pod="openshift-ingress/e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd" Jul 08 13:48:13.609149 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:13.609129 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/845b52f7-50ef-4eb2-abfd-500fa621bf48-istio-token\") pod \"e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd\" (UID: \"845b52f7-50ef-4eb2-abfd-500fa621bf48\") " pod="openshift-ingress/e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd" Jul 08 13:48:13.634172 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:13.634139 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd" Jul 08 13:48:13.765959 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:13.765934 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd"] Jul 08 13:48:13.768061 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:48:13.768038 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod845b52f7_50ef_4eb2_abfd_500fa621bf48.slice/crio-d8d148a2c56a986849c773850056ec0877fdf6e7324bea4f1f30e99b99c49677 WatchSource:0}: Error finding container d8d148a2c56a986849c773850056ec0877fdf6e7324bea4f1f30e99b99c49677: Status 404 returned error can't find the container with id d8d148a2c56a986849c773850056ec0877fdf6e7324bea4f1f30e99b99c49677 Jul 08 13:48:13.770347 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:13.770308 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:48:13.770418 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:13.770396 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:48:13.770454 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:13.770435 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:48:13.897113 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:13.897002 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd" event={"ID":"845b52f7-50ef-4eb2-abfd-500fa621bf48","Type":"ContainerStarted","Data":"05873dd4f46bf206a6840268eda587c6b49522c22e3075557df4504a0040e639"} Jul 08 13:48:13.897113 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:13.897061 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd" event={"ID":"845b52f7-50ef-4eb2-abfd-500fa621bf48","Type":"ContainerStarted","Data":"d8d148a2c56a986849c773850056ec0877fdf6e7324bea4f1f30e99b99c49677"} Jul 08 13:48:13.921247 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:13.921193 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd" podStartSLOduration=0.921175548 podStartE2EDuration="921.175548ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:48:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:48:13.918277933 +0000 UTC m=+1502.715218601" watchObservedRunningTime="2026-07-08 13:48:13.921175548 +0000 UTC m=+1502.718116204" Jul 08 13:48:14.635328 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:14.635279 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd" Jul 08 13:48:14.650854 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:14.650822 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd" Jul 08 13:48:14.902053 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:14.901976 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd" Jul 08 13:48:14.902931 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:14.902912 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd" Jul 08 13:48:17.394355 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:17.394320 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:48:19.073679 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:19.073643 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd"] Jul 08 13:48:19.074136 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:19.073878 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd" podUID="845b52f7-50ef-4eb2-abfd-500fa621bf48" containerName="istio-proxy" containerID="cri-o://05873dd4f46bf206a6840268eda587c6b49522c22e3075557df4504a0040e639" gracePeriod=30 Jul 08 13:48:25.215469 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:25.215443 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd" Jul 08 13:48:25.301376 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:25.301297 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8fc5p\" (UniqueName: \"kubernetes.io/projected/845b52f7-50ef-4eb2-abfd-500fa621bf48-kube-api-access-8fc5p\") pod \"845b52f7-50ef-4eb2-abfd-500fa621bf48\" (UID: \"845b52f7-50ef-4eb2-abfd-500fa621bf48\") " Jul 08 13:48:25.301376 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:25.301336 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/845b52f7-50ef-4eb2-abfd-500fa621bf48-workload-certs\") pod \"845b52f7-50ef-4eb2-abfd-500fa621bf48\" (UID: \"845b52f7-50ef-4eb2-abfd-500fa621bf48\") " Jul 08 13:48:25.301376 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:25.301370 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/845b52f7-50ef-4eb2-abfd-500fa621bf48-istiod-ca-cert\") pod \"845b52f7-50ef-4eb2-abfd-500fa621bf48\" (UID: \"845b52f7-50ef-4eb2-abfd-500fa621bf48\") " Jul 08 13:48:25.301640 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:25.301391 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/845b52f7-50ef-4eb2-abfd-500fa621bf48-credential-socket\") pod \"845b52f7-50ef-4eb2-abfd-500fa621bf48\" (UID: \"845b52f7-50ef-4eb2-abfd-500fa621bf48\") " Jul 08 13:48:25.301640 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:25.301520 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/845b52f7-50ef-4eb2-abfd-500fa621bf48-istio-token\") pod \"845b52f7-50ef-4eb2-abfd-500fa621bf48\" (UID: \"845b52f7-50ef-4eb2-abfd-500fa621bf48\") " Jul 08 13:48:25.301640 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:25.301566 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/845b52f7-50ef-4eb2-abfd-500fa621bf48-istio-envoy\") pod \"845b52f7-50ef-4eb2-abfd-500fa621bf48\" (UID: \"845b52f7-50ef-4eb2-abfd-500fa621bf48\") " Jul 08 13:48:25.301640 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:25.301592 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/845b52f7-50ef-4eb2-abfd-500fa621bf48-istio-podinfo\") pod \"845b52f7-50ef-4eb2-abfd-500fa621bf48\" (UID: \"845b52f7-50ef-4eb2-abfd-500fa621bf48\") " Jul 08 13:48:25.301640 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:25.301603 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/845b52f7-50ef-4eb2-abfd-500fa621bf48-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "845b52f7-50ef-4eb2-abfd-500fa621bf48" (UID: "845b52f7-50ef-4eb2-abfd-500fa621bf48"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:48:25.301899 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:25.301666 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/845b52f7-50ef-4eb2-abfd-500fa621bf48-workload-socket\") pod \"845b52f7-50ef-4eb2-abfd-500fa621bf48\" (UID: \"845b52f7-50ef-4eb2-abfd-500fa621bf48\") " Jul 08 13:48:25.301899 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:25.301688 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/845b52f7-50ef-4eb2-abfd-500fa621bf48-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "845b52f7-50ef-4eb2-abfd-500fa621bf48" (UID: "845b52f7-50ef-4eb2-abfd-500fa621bf48"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:48:25.301899 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:25.301707 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/845b52f7-50ef-4eb2-abfd-500fa621bf48-istio-data\") pod \"845b52f7-50ef-4eb2-abfd-500fa621bf48\" (UID: \"845b52f7-50ef-4eb2-abfd-500fa621bf48\") " Jul 08 13:48:25.301899 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:25.301783 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/845b52f7-50ef-4eb2-abfd-500fa621bf48-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "845b52f7-50ef-4eb2-abfd-500fa621bf48" (UID: "845b52f7-50ef-4eb2-abfd-500fa621bf48"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 13:48:25.302113 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:25.301902 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/845b52f7-50ef-4eb2-abfd-500fa621bf48-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "845b52f7-50ef-4eb2-abfd-500fa621bf48" (UID: "845b52f7-50ef-4eb2-abfd-500fa621bf48"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:48:25.302177 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:25.302111 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/845b52f7-50ef-4eb2-abfd-500fa621bf48-workload-certs\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:48:25.302177 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:25.302132 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/845b52f7-50ef-4eb2-abfd-500fa621bf48-istiod-ca-cert\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:48:25.302177 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:25.302144 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/845b52f7-50ef-4eb2-abfd-500fa621bf48-credential-socket\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:48:25.302177 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:25.302157 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/845b52f7-50ef-4eb2-abfd-500fa621bf48-workload-socket\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:48:25.302469 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:25.302445 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/845b52f7-50ef-4eb2-abfd-500fa621bf48-istio-data" (OuterVolumeSpecName: "istio-data") pod "845b52f7-50ef-4eb2-abfd-500fa621bf48" (UID: "845b52f7-50ef-4eb2-abfd-500fa621bf48"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:48:25.303767 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:25.303739 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/845b52f7-50ef-4eb2-abfd-500fa621bf48-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "845b52f7-50ef-4eb2-abfd-500fa621bf48" (UID: "845b52f7-50ef-4eb2-abfd-500fa621bf48"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 13:48:25.303767 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:25.303749 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/845b52f7-50ef-4eb2-abfd-500fa621bf48-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "845b52f7-50ef-4eb2-abfd-500fa621bf48" (UID: "845b52f7-50ef-4eb2-abfd-500fa621bf48"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:48:25.303998 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:25.303974 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/845b52f7-50ef-4eb2-abfd-500fa621bf48-istio-token" (OuterVolumeSpecName: "istio-token") pod "845b52f7-50ef-4eb2-abfd-500fa621bf48" (UID: "845b52f7-50ef-4eb2-abfd-500fa621bf48"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 13:48:25.304102 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:25.303999 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/845b52f7-50ef-4eb2-abfd-500fa621bf48-kube-api-access-8fc5p" (OuterVolumeSpecName: "kube-api-access-8fc5p") pod "845b52f7-50ef-4eb2-abfd-500fa621bf48" (UID: "845b52f7-50ef-4eb2-abfd-500fa621bf48"). InnerVolumeSpecName "kube-api-access-8fc5p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 13:48:25.402529 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:25.402498 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/845b52f7-50ef-4eb2-abfd-500fa621bf48-istio-data\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:48:25.402529 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:25.402523 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8fc5p\" (UniqueName: \"kubernetes.io/projected/845b52f7-50ef-4eb2-abfd-500fa621bf48-kube-api-access-8fc5p\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:48:25.402529 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:25.402533 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/845b52f7-50ef-4eb2-abfd-500fa621bf48-istio-token\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:48:25.402745 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:25.402542 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/845b52f7-50ef-4eb2-abfd-500fa621bf48-istio-envoy\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:48:25.402745 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:25.402550 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/845b52f7-50ef-4eb2-abfd-500fa621bf48-istio-podinfo\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:48:25.957425 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:25.957391 2567 generic.go:356] "Generic (PLEG): container finished" podID="845b52f7-50ef-4eb2-abfd-500fa621bf48" containerID="05873dd4f46bf206a6840268eda587c6b49522c22e3075557df4504a0040e639" exitCode=0 Jul 08 13:48:25.957629 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:25.957444 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd" event={"ID":"845b52f7-50ef-4eb2-abfd-500fa621bf48","Type":"ContainerDied","Data":"05873dd4f46bf206a6840268eda587c6b49522c22e3075557df4504a0040e639"} Jul 08 13:48:25.957629 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:25.957457 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd" Jul 08 13:48:25.957629 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:25.957490 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd" event={"ID":"845b52f7-50ef-4eb2-abfd-500fa621bf48","Type":"ContainerDied","Data":"d8d148a2c56a986849c773850056ec0877fdf6e7324bea4f1f30e99b99c49677"} Jul 08 13:48:25.957629 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:25.957515 2567 scope.go:117] "RemoveContainer" containerID="05873dd4f46bf206a6840268eda587c6b49522c22e3075557df4504a0040e639" Jul 08 13:48:25.966478 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:25.966459 2567 scope.go:117] "RemoveContainer" containerID="05873dd4f46bf206a6840268eda587c6b49522c22e3075557df4504a0040e639" Jul 08 13:48:25.966741 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:48:25.966724 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05873dd4f46bf206a6840268eda587c6b49522c22e3075557df4504a0040e639\": container with ID starting with 05873dd4f46bf206a6840268eda587c6b49522c22e3075557df4504a0040e639 not found: ID does not exist" containerID="05873dd4f46bf206a6840268eda587c6b49522c22e3075557df4504a0040e639" Jul 08 13:48:25.966841 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:25.966748 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05873dd4f46bf206a6840268eda587c6b49522c22e3075557df4504a0040e639"} err="failed to get container status \"05873dd4f46bf206a6840268eda587c6b49522c22e3075557df4504a0040e639\": rpc error: code = NotFound desc = could not find container \"05873dd4f46bf206a6840268eda587c6b49522c22e3075557df4504a0040e639\": container with ID starting with 05873dd4f46bf206a6840268eda587c6b49522c22e3075557df4504a0040e639 not found: ID does not exist" Jul 08 13:48:25.975666 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:25.975641 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd"] Jul 08 13:48:25.981508 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:25.981480 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-50abd907-openshift-default-6c94444c87-2ncfd"] Jul 08 13:48:26.491992 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:26.491954 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:48:27.796109 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:48:27.796058 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="845b52f7-50ef-4eb2-abfd-500fa621bf48" path="/var/lib/kubelet/pods/845b52f7-50ef-4eb2-abfd-500fa621bf48/volumes" Jul 08 13:49:47.795418 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:49:47.795387 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:49:56.698040 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:49:56.698004 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:50:00.050226 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:00.050192 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68"] Jul 08 13:50:00.050582 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:00.050527 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="845b52f7-50ef-4eb2-abfd-500fa621bf48" containerName="istio-proxy" Jul 08 13:50:00.050582 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:00.050537 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="845b52f7-50ef-4eb2-abfd-500fa621bf48" containerName="istio-proxy" Jul 08 13:50:00.050651 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:00.050593 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="845b52f7-50ef-4eb2-abfd-500fa621bf48" containerName="istio-proxy" Jul 08 13:50:00.052530 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:00.052510 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68" Jul 08 13:50:00.054949 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:00.054930 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-e91b2474-openshift-default-dockercfg-kwdx8\"" Jul 08 13:50:00.074754 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:00.074728 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68"] Jul 08 13:50:00.183693 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:00.183651 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9883b4c8-1458-4a36-b425-fcff3e855eeb-istio-data\") pod \"e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68\" (UID: \"9883b4c8-1458-4a36-b425-fcff3e855eeb\") " pod="openshift-ingress/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68" Jul 08 13:50:00.183693 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:00.183696 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9883b4c8-1458-4a36-b425-fcff3e855eeb-workload-socket\") pod \"e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68\" (UID: \"9883b4c8-1458-4a36-b425-fcff3e855eeb\") " pod="openshift-ingress/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68" Jul 08 13:50:00.183894 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:00.183720 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9883b4c8-1458-4a36-b425-fcff3e855eeb-istiod-ca-cert\") pod \"e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68\" (UID: \"9883b4c8-1458-4a36-b425-fcff3e855eeb\") " pod="openshift-ingress/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68" Jul 08 13:50:00.183894 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:00.183802 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9883b4c8-1458-4a36-b425-fcff3e855eeb-istio-envoy\") pod \"e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68\" (UID: \"9883b4c8-1458-4a36-b425-fcff3e855eeb\") " pod="openshift-ingress/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68" Jul 08 13:50:00.183894 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:00.183836 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9883b4c8-1458-4a36-b425-fcff3e855eeb-istio-podinfo\") pod \"e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68\" (UID: \"9883b4c8-1458-4a36-b425-fcff3e855eeb\") " pod="openshift-ingress/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68" Jul 08 13:50:00.183894 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:00.183877 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68vtk\" (UniqueName: \"kubernetes.io/projected/9883b4c8-1458-4a36-b425-fcff3e855eeb-kube-api-access-68vtk\") pod \"e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68\" (UID: \"9883b4c8-1458-4a36-b425-fcff3e855eeb\") " pod="openshift-ingress/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68" Jul 08 13:50:00.184055 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:00.183919 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9883b4c8-1458-4a36-b425-fcff3e855eeb-credential-socket\") pod \"e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68\" (UID: \"9883b4c8-1458-4a36-b425-fcff3e855eeb\") " pod="openshift-ingress/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68" Jul 08 13:50:00.184055 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:00.183938 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9883b4c8-1458-4a36-b425-fcff3e855eeb-workload-certs\") pod \"e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68\" (UID: \"9883b4c8-1458-4a36-b425-fcff3e855eeb\") " pod="openshift-ingress/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68" Jul 08 13:50:00.184055 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:00.183962 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9883b4c8-1458-4a36-b425-fcff3e855eeb-istio-token\") pod \"e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68\" (UID: \"9883b4c8-1458-4a36-b425-fcff3e855eeb\") " pod="openshift-ingress/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68" Jul 08 13:50:00.285181 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:00.285142 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68vtk\" (UniqueName: \"kubernetes.io/projected/9883b4c8-1458-4a36-b425-fcff3e855eeb-kube-api-access-68vtk\") pod \"e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68\" (UID: \"9883b4c8-1458-4a36-b425-fcff3e855eeb\") " pod="openshift-ingress/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68" Jul 08 13:50:00.285351 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:00.285189 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9883b4c8-1458-4a36-b425-fcff3e855eeb-credential-socket\") pod \"e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68\" (UID: \"9883b4c8-1458-4a36-b425-fcff3e855eeb\") " pod="openshift-ingress/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68" Jul 08 13:50:00.285351 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:00.285205 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9883b4c8-1458-4a36-b425-fcff3e855eeb-workload-certs\") pod \"e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68\" (UID: \"9883b4c8-1458-4a36-b425-fcff3e855eeb\") " pod="openshift-ingress/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68" Jul 08 13:50:00.285351 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:00.285227 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9883b4c8-1458-4a36-b425-fcff3e855eeb-istio-token\") pod \"e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68\" (UID: \"9883b4c8-1458-4a36-b425-fcff3e855eeb\") " pod="openshift-ingress/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68" Jul 08 13:50:00.285351 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:00.285261 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9883b4c8-1458-4a36-b425-fcff3e855eeb-istio-data\") pod \"e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68\" (UID: \"9883b4c8-1458-4a36-b425-fcff3e855eeb\") " pod="openshift-ingress/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68" Jul 08 13:50:00.285351 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:00.285284 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9883b4c8-1458-4a36-b425-fcff3e855eeb-workload-socket\") pod \"e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68\" (UID: \"9883b4c8-1458-4a36-b425-fcff3e855eeb\") " pod="openshift-ingress/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68" Jul 08 13:50:00.285351 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:00.285309 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9883b4c8-1458-4a36-b425-fcff3e855eeb-istiod-ca-cert\") pod \"e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68\" (UID: \"9883b4c8-1458-4a36-b425-fcff3e855eeb\") " pod="openshift-ingress/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68" Jul 08 13:50:00.285351 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:00.285340 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9883b4c8-1458-4a36-b425-fcff3e855eeb-istio-envoy\") pod \"e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68\" (UID: \"9883b4c8-1458-4a36-b425-fcff3e855eeb\") " pod="openshift-ingress/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68" Jul 08 13:50:00.285714 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:00.285365 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9883b4c8-1458-4a36-b425-fcff3e855eeb-istio-podinfo\") pod \"e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68\" (UID: \"9883b4c8-1458-4a36-b425-fcff3e855eeb\") " pod="openshift-ingress/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68" Jul 08 13:50:00.285714 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:00.285629 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9883b4c8-1458-4a36-b425-fcff3e855eeb-credential-socket\") pod \"e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68\" (UID: \"9883b4c8-1458-4a36-b425-fcff3e855eeb\") " pod="openshift-ingress/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68" Jul 08 13:50:00.285714 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:00.285639 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9883b4c8-1458-4a36-b425-fcff3e855eeb-workload-certs\") pod \"e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68\" (UID: \"9883b4c8-1458-4a36-b425-fcff3e855eeb\") " pod="openshift-ingress/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68" Jul 08 13:50:00.285870 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:00.285769 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9883b4c8-1458-4a36-b425-fcff3e855eeb-istio-data\") pod \"e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68\" (UID: \"9883b4c8-1458-4a36-b425-fcff3e855eeb\") " pod="openshift-ingress/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68" Jul 08 13:50:00.286021 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:00.285992 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9883b4c8-1458-4a36-b425-fcff3e855eeb-istiod-ca-cert\") pod \"e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68\" (UID: \"9883b4c8-1458-4a36-b425-fcff3e855eeb\") " pod="openshift-ingress/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68" Jul 08 13:50:00.286179 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:00.286000 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9883b4c8-1458-4a36-b425-fcff3e855eeb-workload-socket\") pod \"e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68\" (UID: \"9883b4c8-1458-4a36-b425-fcff3e855eeb\") " pod="openshift-ingress/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68" Jul 08 13:50:00.287736 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:00.287713 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9883b4c8-1458-4a36-b425-fcff3e855eeb-istio-envoy\") pod \"e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68\" (UID: \"9883b4c8-1458-4a36-b425-fcff3e855eeb\") " pod="openshift-ingress/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68" Jul 08 13:50:00.287816 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:00.287790 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9883b4c8-1458-4a36-b425-fcff3e855eeb-istio-podinfo\") pod \"e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68\" (UID: \"9883b4c8-1458-4a36-b425-fcff3e855eeb\") " pod="openshift-ingress/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68" Jul 08 13:50:00.293869 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:00.293843 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9883b4c8-1458-4a36-b425-fcff3e855eeb-istio-token\") pod \"e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68\" (UID: \"9883b4c8-1458-4a36-b425-fcff3e855eeb\") " pod="openshift-ingress/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68" Jul 08 13:50:00.293973 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:00.293945 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-68vtk\" (UniqueName: \"kubernetes.io/projected/9883b4c8-1458-4a36-b425-fcff3e855eeb-kube-api-access-68vtk\") pod \"e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68\" (UID: \"9883b4c8-1458-4a36-b425-fcff3e855eeb\") " pod="openshift-ingress/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68" Jul 08 13:50:00.363189 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:00.363102 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68" Jul 08 13:50:00.500008 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:00.499985 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68"] Jul 08 13:50:00.501939 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:50:00.501909 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9883b4c8_1458_4a36_b425_fcff3e855eeb.slice/crio-14020babb0e683905954dd057aa86386db413cc199a9fa6ddd95f5bbb8e42bfd WatchSource:0}: Error finding container 14020babb0e683905954dd057aa86386db413cc199a9fa6ddd95f5bbb8e42bfd: Status 404 returned error can't find the container with id 14020babb0e683905954dd057aa86386db413cc199a9fa6ddd95f5bbb8e42bfd Jul 08 13:50:00.504473 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:00.504432 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:50:00.504551 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:00.504506 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:50:00.504551 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:00.504536 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:50:01.422654 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:01.422616 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68" event={"ID":"9883b4c8-1458-4a36-b425-fcff3e855eeb","Type":"ContainerStarted","Data":"0efb0174a78684d3d108e269e36b9c4bb8ffd04c22493c07e314c8d1ff5c8a91"} Jul 08 13:50:01.423034 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:01.422675 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68" event={"ID":"9883b4c8-1458-4a36-b425-fcff3e855eeb","Type":"ContainerStarted","Data":"14020babb0e683905954dd057aa86386db413cc199a9fa6ddd95f5bbb8e42bfd"} Jul 08 13:50:01.443030 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:01.442981 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68" podStartSLOduration=1.442965848 podStartE2EDuration="1.442965848s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:50:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:50:01.441544744 +0000 UTC m=+1610.238485547" watchObservedRunningTime="2026-07-08 13:50:01.442965848 +0000 UTC m=+1610.239906503" Jul 08 13:50:02.364049 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:02.364008 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68" Jul 08 13:50:02.369112 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:02.369068 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68" Jul 08 13:50:02.427699 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:02.427667 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68" Jul 08 13:50:02.428698 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:02.428676 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68" Jul 08 13:50:08.696182 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:08.696107 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:50:17.429136 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:17.429070 2567 patch_prober.go:28] interesting pod/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body= Jul 08 13:50:17.429559 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:17.429171 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68" podUID="9883b4c8-1458-4a36-b425-fcff3e855eeb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" Jul 08 13:50:17.503933 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:17.503899 2567 generic.go:356] "Generic (PLEG): container finished" podID="9883b4c8-1458-4a36-b425-fcff3e855eeb" containerID="0efb0174a78684d3d108e269e36b9c4bb8ffd04c22493c07e314c8d1ff5c8a91" exitCode=0 Jul 08 13:50:17.504119 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:17.503959 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68" event={"ID":"9883b4c8-1458-4a36-b425-fcff3e855eeb","Type":"ContainerDied","Data":"0efb0174a78684d3d108e269e36b9c4bb8ffd04c22493c07e314c8d1ff5c8a91"} Jul 08 13:50:17.504436 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:17.504423 2567 scope.go:117] "RemoveContainer" containerID="0efb0174a78684d3d108e269e36b9c4bb8ffd04c22493c07e314c8d1ff5c8a91" Jul 08 13:50:17.505386 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:17.505362 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:50:17.505455 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:17.505441 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:50:17.505508 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:17.505496 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:50:18.512589 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:18.512552 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68" event={"ID":"9883b4c8-1458-4a36-b425-fcff3e855eeb","Type":"ContainerStarted","Data":"bc29400e07ee6613540285b0804ce5e33366b0b5f1ce3348ef3eac77002f3592"} Jul 08 13:50:19.363866 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:19.363828 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68" Jul 08 13:50:19.364161 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:19.364135 2567 patch_prober.go:28] interesting pod/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body= Jul 08 13:50:19.364218 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:19.364187 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68" podUID="9883b4c8-1458-4a36-b425-fcff3e855eeb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" Jul 08 13:50:20.364540 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:20.364504 2567 patch_prober.go:28] interesting pod/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body= Jul 08 13:50:20.364950 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:20.364563 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68" podUID="9883b4c8-1458-4a36-b425-fcff3e855eeb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" Jul 08 13:50:20.707571 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:20.707483 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:50:21.363533 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:21.363498 2567 patch_prober.go:28] interesting pod/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body= Jul 08 13:50:21.363728 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:21.363571 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68" podUID="9883b4c8-1458-4a36-b425-fcff3e855eeb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" Jul 08 13:50:22.363788 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:22.363749 2567 patch_prober.go:28] interesting pod/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body= Jul 08 13:50:22.364224 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:22.363829 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68" podUID="9883b4c8-1458-4a36-b425-fcff3e855eeb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" Jul 08 13:50:22.422726 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:22.422692 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68"] Jul 08 13:50:22.422960 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:22.422918 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68" podUID="9883b4c8-1458-4a36-b425-fcff3e855eeb" containerName="istio-proxy" containerID="cri-o://bc29400e07ee6613540285b0804ce5e33366b0b5f1ce3348ef3eac77002f3592" gracePeriod=30 Jul 08 13:50:27.975485 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:27.975454 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68" Jul 08 13:50:28.040789 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:28.040707 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9883b4c8-1458-4a36-b425-fcff3e855eeb-istiod-ca-cert\") pod \"9883b4c8-1458-4a36-b425-fcff3e855eeb\" (UID: \"9883b4c8-1458-4a36-b425-fcff3e855eeb\") " Jul 08 13:50:28.040789 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:28.040748 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9883b4c8-1458-4a36-b425-fcff3e855eeb-istio-podinfo\") pod \"9883b4c8-1458-4a36-b425-fcff3e855eeb\" (UID: \"9883b4c8-1458-4a36-b425-fcff3e855eeb\") " Jul 08 13:50:28.040789 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:28.040779 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9883b4c8-1458-4a36-b425-fcff3e855eeb-istio-data\") pod \"9883b4c8-1458-4a36-b425-fcff3e855eeb\" (UID: \"9883b4c8-1458-4a36-b425-fcff3e855eeb\") " Jul 08 13:50:28.041057 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:28.040805 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9883b4c8-1458-4a36-b425-fcff3e855eeb-istio-envoy\") pod \"9883b4c8-1458-4a36-b425-fcff3e855eeb\" (UID: \"9883b4c8-1458-4a36-b425-fcff3e855eeb\") " Jul 08 13:50:28.041057 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:28.040977 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9883b4c8-1458-4a36-b425-fcff3e855eeb-credential-socket\") pod \"9883b4c8-1458-4a36-b425-fcff3e855eeb\" (UID: \"9883b4c8-1458-4a36-b425-fcff3e855eeb\") " Jul 08 13:50:28.041057 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:28.041034 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-68vtk\" (UniqueName: \"kubernetes.io/projected/9883b4c8-1458-4a36-b425-fcff3e855eeb-kube-api-access-68vtk\") pod \"9883b4c8-1458-4a36-b425-fcff3e855eeb\" (UID: \"9883b4c8-1458-4a36-b425-fcff3e855eeb\") " Jul 08 13:50:28.041252 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:28.041124 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9883b4c8-1458-4a36-b425-fcff3e855eeb-istio-token\") pod \"9883b4c8-1458-4a36-b425-fcff3e855eeb\" (UID: \"9883b4c8-1458-4a36-b425-fcff3e855eeb\") " Jul 08 13:50:28.041252 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:28.041179 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9883b4c8-1458-4a36-b425-fcff3e855eeb-workload-certs\") pod \"9883b4c8-1458-4a36-b425-fcff3e855eeb\" (UID: \"9883b4c8-1458-4a36-b425-fcff3e855eeb\") " Jul 08 13:50:28.041252 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:28.041206 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9883b4c8-1458-4a36-b425-fcff3e855eeb-workload-socket\") pod \"9883b4c8-1458-4a36-b425-fcff3e855eeb\" (UID: \"9883b4c8-1458-4a36-b425-fcff3e855eeb\") " Jul 08 13:50:28.041396 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:28.041253 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9883b4c8-1458-4a36-b425-fcff3e855eeb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9883b4c8-1458-4a36-b425-fcff3e855eeb" (UID: "9883b4c8-1458-4a36-b425-fcff3e855eeb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:50:28.041396 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:28.041282 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9883b4c8-1458-4a36-b425-fcff3e855eeb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9883b4c8-1458-4a36-b425-fcff3e855eeb" (UID: "9883b4c8-1458-4a36-b425-fcff3e855eeb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 13:50:28.041518 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:28.041496 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9883b4c8-1458-4a36-b425-fcff3e855eeb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9883b4c8-1458-4a36-b425-fcff3e855eeb" (UID: "9883b4c8-1458-4a36-b425-fcff3e855eeb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:50:28.041634 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:28.041602 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9883b4c8-1458-4a36-b425-fcff3e855eeb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9883b4c8-1458-4a36-b425-fcff3e855eeb" (UID: "9883b4c8-1458-4a36-b425-fcff3e855eeb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:50:28.041634 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:28.041630 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9883b4c8-1458-4a36-b425-fcff3e855eeb-workload-socket\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:50:28.041802 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:28.041647 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9883b4c8-1458-4a36-b425-fcff3e855eeb-istiod-ca-cert\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:50:28.041802 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:28.041658 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9883b4c8-1458-4a36-b425-fcff3e855eeb-credential-socket\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:50:28.041883 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:28.041836 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9883b4c8-1458-4a36-b425-fcff3e855eeb-istio-data" (OuterVolumeSpecName: "istio-data") pod "9883b4c8-1458-4a36-b425-fcff3e855eeb" (UID: "9883b4c8-1458-4a36-b425-fcff3e855eeb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:50:28.043139 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:28.043060 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9883b4c8-1458-4a36-b425-fcff3e855eeb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9883b4c8-1458-4a36-b425-fcff3e855eeb" (UID: "9883b4c8-1458-4a36-b425-fcff3e855eeb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:50:28.043254 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:28.043232 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9883b4c8-1458-4a36-b425-fcff3e855eeb-istio-token" (OuterVolumeSpecName: "istio-token") pod "9883b4c8-1458-4a36-b425-fcff3e855eeb" (UID: "9883b4c8-1458-4a36-b425-fcff3e855eeb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 13:50:28.043422 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:28.043401 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9883b4c8-1458-4a36-b425-fcff3e855eeb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9883b4c8-1458-4a36-b425-fcff3e855eeb" (UID: "9883b4c8-1458-4a36-b425-fcff3e855eeb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 13:50:28.043661 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:28.043641 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9883b4c8-1458-4a36-b425-fcff3e855eeb-kube-api-access-68vtk" (OuterVolumeSpecName: "kube-api-access-68vtk") pod "9883b4c8-1458-4a36-b425-fcff3e855eeb" (UID: "9883b4c8-1458-4a36-b425-fcff3e855eeb"). InnerVolumeSpecName "kube-api-access-68vtk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 13:50:28.142281 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:28.142238 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9883b4c8-1458-4a36-b425-fcff3e855eeb-istio-podinfo\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:50:28.142281 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:28.142271 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9883b4c8-1458-4a36-b425-fcff3e855eeb-istio-data\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:50:28.142281 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:28.142280 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9883b4c8-1458-4a36-b425-fcff3e855eeb-istio-envoy\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:50:28.142281 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:28.142290 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-68vtk\" (UniqueName: \"kubernetes.io/projected/9883b4c8-1458-4a36-b425-fcff3e855eeb-kube-api-access-68vtk\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:50:28.142545 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:28.142301 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9883b4c8-1458-4a36-b425-fcff3e855eeb-istio-token\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:50:28.142545 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:28.142309 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9883b4c8-1458-4a36-b425-fcff3e855eeb-workload-certs\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:50:28.564056 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:28.564025 2567 generic.go:356] "Generic (PLEG): container finished" podID="9883b4c8-1458-4a36-b425-fcff3e855eeb" containerID="bc29400e07ee6613540285b0804ce5e33366b0b5f1ce3348ef3eac77002f3592" exitCode=0 Jul 08 13:50:28.564237 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:28.564071 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68" Jul 08 13:50:28.564237 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:28.564116 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68" event={"ID":"9883b4c8-1458-4a36-b425-fcff3e855eeb","Type":"ContainerDied","Data":"bc29400e07ee6613540285b0804ce5e33366b0b5f1ce3348ef3eac77002f3592"} Jul 08 13:50:28.564237 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:28.564171 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68" event={"ID":"9883b4c8-1458-4a36-b425-fcff3e855eeb","Type":"ContainerDied","Data":"14020babb0e683905954dd057aa86386db413cc199a9fa6ddd95f5bbb8e42bfd"} Jul 08 13:50:28.564237 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:28.564194 2567 scope.go:117] "RemoveContainer" containerID="bc29400e07ee6613540285b0804ce5e33366b0b5f1ce3348ef3eac77002f3592" Jul 08 13:50:28.573593 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:28.573572 2567 scope.go:117] "RemoveContainer" containerID="0efb0174a78684d3d108e269e36b9c4bb8ffd04c22493c07e314c8d1ff5c8a91" Jul 08 13:50:28.581319 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:28.581304 2567 scope.go:117] "RemoveContainer" containerID="bc29400e07ee6613540285b0804ce5e33366b0b5f1ce3348ef3eac77002f3592" Jul 08 13:50:28.581569 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:50:28.581553 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc29400e07ee6613540285b0804ce5e33366b0b5f1ce3348ef3eac77002f3592\": container with ID starting with bc29400e07ee6613540285b0804ce5e33366b0b5f1ce3348ef3eac77002f3592 not found: ID does not exist" containerID="bc29400e07ee6613540285b0804ce5e33366b0b5f1ce3348ef3eac77002f3592" Jul 08 13:50:28.581620 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:28.581579 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc29400e07ee6613540285b0804ce5e33366b0b5f1ce3348ef3eac77002f3592"} err="failed to get container status \"bc29400e07ee6613540285b0804ce5e33366b0b5f1ce3348ef3eac77002f3592\": rpc error: code = NotFound desc = could not find container \"bc29400e07ee6613540285b0804ce5e33366b0b5f1ce3348ef3eac77002f3592\": container with ID starting with bc29400e07ee6613540285b0804ce5e33366b0b5f1ce3348ef3eac77002f3592 not found: ID does not exist" Jul 08 13:50:28.581620 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:28.581596 2567 scope.go:117] "RemoveContainer" containerID="0efb0174a78684d3d108e269e36b9c4bb8ffd04c22493c07e314c8d1ff5c8a91" Jul 08 13:50:28.581843 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:50:28.581823 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0efb0174a78684d3d108e269e36b9c4bb8ffd04c22493c07e314c8d1ff5c8a91\": container with ID starting with 0efb0174a78684d3d108e269e36b9c4bb8ffd04c22493c07e314c8d1ff5c8a91 not found: ID does not exist" containerID="0efb0174a78684d3d108e269e36b9c4bb8ffd04c22493c07e314c8d1ff5c8a91" Jul 08 13:50:28.581895 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:28.581854 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0efb0174a78684d3d108e269e36b9c4bb8ffd04c22493c07e314c8d1ff5c8a91"} err="failed to get container status \"0efb0174a78684d3d108e269e36b9c4bb8ffd04c22493c07e314c8d1ff5c8a91\": rpc error: code = NotFound desc = could not find container \"0efb0174a78684d3d108e269e36b9c4bb8ffd04c22493c07e314c8d1ff5c8a91\": container with ID starting with 0efb0174a78684d3d108e269e36b9c4bb8ffd04c22493c07e314c8d1ff5c8a91 not found: ID does not exist" Jul 08 13:50:28.589934 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:28.589903 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68"] Jul 08 13:50:28.594356 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:28.594332 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-e91b2474-openshift-default-db8fd67c5-ppw68"] Jul 08 13:50:29.795813 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:29.795782 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9883b4c8-1458-4a36-b425-fcff3e855eeb" path="/var/lib/kubelet/pods/9883b4c8-1458-4a36-b425-fcff3e855eeb/volumes" Jul 08 13:50:42.718861 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:42.718824 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp"] Jul 08 13:50:42.719269 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:42.719161 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9883b4c8-1458-4a36-b425-fcff3e855eeb" containerName="istio-proxy" Jul 08 13:50:42.719269 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:42.719174 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9883b4c8-1458-4a36-b425-fcff3e855eeb" containerName="istio-proxy" Jul 08 13:50:42.719269 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:42.719186 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9883b4c8-1458-4a36-b425-fcff3e855eeb" containerName="istio-proxy" Jul 08 13:50:42.719269 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:42.719191 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9883b4c8-1458-4a36-b425-fcff3e855eeb" containerName="istio-proxy" Jul 08 13:50:42.719269 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:42.719262 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9883b4c8-1458-4a36-b425-fcff3e855eeb" containerName="istio-proxy" Jul 08 13:50:42.719269 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:42.719271 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9883b4c8-1458-4a36-b425-fcff3e855eeb" containerName="istio-proxy" Jul 08 13:50:42.722113 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:42.722094 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp" Jul 08 13:50:42.727163 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:42.727136 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-54e722cc-openshift-default-dockercfg-s9hhq\"" Jul 08 13:50:42.759377 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:42.759344 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp"] Jul 08 13:50:42.867462 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:42.867427 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83719851-5080-42f7-9cc3-7b2f29e56992-workload-socket\") pod \"e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp\" (UID: \"83719851-5080-42f7-9cc3-7b2f29e56992\") " pod="openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp" Jul 08 13:50:42.867462 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:42.867467 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83719851-5080-42f7-9cc3-7b2f29e56992-credential-socket\") pod \"e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp\" (UID: \"83719851-5080-42f7-9cc3-7b2f29e56992\") " pod="openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp" Jul 08 13:50:42.867685 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:42.867541 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rx9jj\" (UniqueName: \"kubernetes.io/projected/83719851-5080-42f7-9cc3-7b2f29e56992-kube-api-access-rx9jj\") pod \"e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp\" (UID: \"83719851-5080-42f7-9cc3-7b2f29e56992\") " pod="openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp" Jul 08 13:50:42.867685 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:42.867636 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83719851-5080-42f7-9cc3-7b2f29e56992-workload-certs\") pod \"e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp\" (UID: \"83719851-5080-42f7-9cc3-7b2f29e56992\") " pod="openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp" Jul 08 13:50:42.867752 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:42.867689 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83719851-5080-42f7-9cc3-7b2f29e56992-istio-data\") pod \"e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp\" (UID: \"83719851-5080-42f7-9cc3-7b2f29e56992\") " pod="openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp" Jul 08 13:50:42.867752 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:42.867710 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83719851-5080-42f7-9cc3-7b2f29e56992-istio-token\") pod \"e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp\" (UID: \"83719851-5080-42f7-9cc3-7b2f29e56992\") " pod="openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp" Jul 08 13:50:42.867817 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:42.867767 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83719851-5080-42f7-9cc3-7b2f29e56992-istiod-ca-cert\") pod \"e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp\" (UID: \"83719851-5080-42f7-9cc3-7b2f29e56992\") " pod="openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp" Jul 08 13:50:42.867817 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:42.867794 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83719851-5080-42f7-9cc3-7b2f29e56992-istio-podinfo\") pod \"e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp\" (UID: \"83719851-5080-42f7-9cc3-7b2f29e56992\") " pod="openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp" Jul 08 13:50:42.867878 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:42.867815 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83719851-5080-42f7-9cc3-7b2f29e56992-istio-envoy\") pod \"e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp\" (UID: \"83719851-5080-42f7-9cc3-7b2f29e56992\") " pod="openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp" Jul 08 13:50:42.969089 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:42.968999 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83719851-5080-42f7-9cc3-7b2f29e56992-workload-certs\") pod \"e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp\" (UID: \"83719851-5080-42f7-9cc3-7b2f29e56992\") " pod="openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp" Jul 08 13:50:42.969089 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:42.969034 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83719851-5080-42f7-9cc3-7b2f29e56992-istio-data\") pod \"e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp\" (UID: \"83719851-5080-42f7-9cc3-7b2f29e56992\") " pod="openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp" Jul 08 13:50:42.969089 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:42.969051 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83719851-5080-42f7-9cc3-7b2f29e56992-istio-token\") pod \"e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp\" (UID: \"83719851-5080-42f7-9cc3-7b2f29e56992\") " pod="openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp" Jul 08 13:50:42.969409 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:42.969098 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83719851-5080-42f7-9cc3-7b2f29e56992-istiod-ca-cert\") pod \"e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp\" (UID: \"83719851-5080-42f7-9cc3-7b2f29e56992\") " pod="openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp" Jul 08 13:50:42.969409 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:42.969127 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83719851-5080-42f7-9cc3-7b2f29e56992-istio-podinfo\") pod \"e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp\" (UID: \"83719851-5080-42f7-9cc3-7b2f29e56992\") " pod="openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp" Jul 08 13:50:42.969409 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:42.969147 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83719851-5080-42f7-9cc3-7b2f29e56992-istio-envoy\") pod \"e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp\" (UID: \"83719851-5080-42f7-9cc3-7b2f29e56992\") " pod="openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp" Jul 08 13:50:42.969409 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:42.969203 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83719851-5080-42f7-9cc3-7b2f29e56992-workload-socket\") pod \"e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp\" (UID: \"83719851-5080-42f7-9cc3-7b2f29e56992\") " pod="openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp" Jul 08 13:50:42.969409 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:42.969235 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83719851-5080-42f7-9cc3-7b2f29e56992-credential-socket\") pod \"e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp\" (UID: \"83719851-5080-42f7-9cc3-7b2f29e56992\") " pod="openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp" Jul 08 13:50:42.969409 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:42.969282 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rx9jj\" (UniqueName: \"kubernetes.io/projected/83719851-5080-42f7-9cc3-7b2f29e56992-kube-api-access-rx9jj\") pod \"e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp\" (UID: \"83719851-5080-42f7-9cc3-7b2f29e56992\") " pod="openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp" Jul 08 13:50:42.969694 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:42.969526 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83719851-5080-42f7-9cc3-7b2f29e56992-istio-data\") pod \"e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp\" (UID: \"83719851-5080-42f7-9cc3-7b2f29e56992\") " pod="openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp" Jul 08 13:50:42.969694 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:42.969673 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83719851-5080-42f7-9cc3-7b2f29e56992-workload-socket\") pod \"e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp\" (UID: \"83719851-5080-42f7-9cc3-7b2f29e56992\") " pod="openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp" Jul 08 13:50:42.969780 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:42.969700 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83719851-5080-42f7-9cc3-7b2f29e56992-workload-certs\") pod \"e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp\" (UID: \"83719851-5080-42f7-9cc3-7b2f29e56992\") " pod="openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp" Jul 08 13:50:42.969780 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:42.969744 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83719851-5080-42f7-9cc3-7b2f29e56992-credential-socket\") pod \"e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp\" (UID: \"83719851-5080-42f7-9cc3-7b2f29e56992\") " pod="openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp" Jul 08 13:50:42.970140 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:42.970118 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83719851-5080-42f7-9cc3-7b2f29e56992-istiod-ca-cert\") pod \"e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp\" (UID: \"83719851-5080-42f7-9cc3-7b2f29e56992\") " pod="openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp" Jul 08 13:50:42.971565 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:42.971537 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83719851-5080-42f7-9cc3-7b2f29e56992-istio-envoy\") pod \"e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp\" (UID: \"83719851-5080-42f7-9cc3-7b2f29e56992\") " pod="openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp" Jul 08 13:50:42.971672 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:42.971624 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83719851-5080-42f7-9cc3-7b2f29e56992-istio-podinfo\") pod \"e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp\" (UID: \"83719851-5080-42f7-9cc3-7b2f29e56992\") " pod="openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp" Jul 08 13:50:42.984127 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:42.984067 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83719851-5080-42f7-9cc3-7b2f29e56992-istio-token\") pod \"e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp\" (UID: \"83719851-5080-42f7-9cc3-7b2f29e56992\") " pod="openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp" Jul 08 13:50:42.990623 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:42.990592 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rx9jj\" (UniqueName: \"kubernetes.io/projected/83719851-5080-42f7-9cc3-7b2f29e56992-kube-api-access-rx9jj\") pod \"e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp\" (UID: \"83719851-5080-42f7-9cc3-7b2f29e56992\") " pod="openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp" Jul 08 13:50:43.031474 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:43.031443 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp" Jul 08 13:50:43.172235 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:43.172212 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp"] Jul 08 13:50:43.173532 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:50:43.173505 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83719851_5080_42f7_9cc3_7b2f29e56992.slice/crio-704940ee4289130cfd3ee9766efe5dea230a2e93271adb5a0c92910ec47391c8 WatchSource:0}: Error finding container 704940ee4289130cfd3ee9766efe5dea230a2e93271adb5a0c92910ec47391c8: Status 404 returned error can't find the container with id 704940ee4289130cfd3ee9766efe5dea230a2e93271adb5a0c92910ec47391c8 Jul 08 13:50:43.175620 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:43.175589 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:50:43.175714 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:43.175649 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:50:43.175714 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:43.175675 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:50:43.640049 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:43.639949 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp" event={"ID":"83719851-5080-42f7-9cc3-7b2f29e56992","Type":"ContainerStarted","Data":"c9d0a5612694594b2c381c8efa861f5e2dcc24d07a820fde79d19b8bf1d16e78"} Jul 08 13:50:43.640049 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:43.640047 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp" event={"ID":"83719851-5080-42f7-9cc3-7b2f29e56992","Type":"ContainerStarted","Data":"704940ee4289130cfd3ee9766efe5dea230a2e93271adb5a0c92910ec47391c8"} Jul 08 13:50:43.663860 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:43.663810 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp" podStartSLOduration=1.663791552 podStartE2EDuration="1.663791552s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:50:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:50:43.661285426 +0000 UTC m=+1652.458226083" watchObservedRunningTime="2026-07-08 13:50:43.663791552 +0000 UTC m=+1652.460732207" Jul 08 13:50:44.032352 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:44.032320 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp" Jul 08 13:50:44.037240 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:44.037215 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp" Jul 08 13:50:44.645215 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:44.645190 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp" Jul 08 13:50:44.646302 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:44.646282 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp" Jul 08 13:50:50.902495 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:50.902459 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:50:53.811867 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:53.811830 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-54e722cc/e2e-lifecycle-model-54e722cc-kserve-74d8796b8c-xdlbc"] Jul 08 13:50:53.815154 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:53.815134 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-54e722cc/e2e-lifecycle-model-54e722cc-kserve-74d8796b8c-xdlbc" Jul 08 13:50:53.820758 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:53.820716 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-54e722cc\"/\"e2e-lifecycle-model-54e722cc-kserve-self-signed-certs\"" Jul 08 13:50:53.820758 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:53.820714 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-54e722cc\"/\"default-dockercfg-vn8pk\"" Jul 08 13:50:53.821115 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:53.821059 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-54e722cc\"/\"openshift-service-ca.crt\"" Jul 08 13:50:53.821222 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:53.821064 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-54e722cc\"/\"kube-root-ca.crt\"" Jul 08 13:50:53.837355 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:53.837330 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-54e722cc/e2e-lifecycle-model-54e722cc-kserve-74d8796b8c-xdlbc"] Jul 08 13:50:53.970761 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:53.970726 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c02acb4-0e96-43cc-a96d-ee1a6c9e382f-dshm\") pod \"e2e-lifecycle-model-54e722cc-kserve-74d8796b8c-xdlbc\" (UID: \"0c02acb4-0e96-43cc-a96d-ee1a6c9e382f\") " pod="ai-tenant-e2e-mt-54e722cc/e2e-lifecycle-model-54e722cc-kserve-74d8796b8c-xdlbc" Jul 08 13:50:53.970933 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:53.970769 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c02acb4-0e96-43cc-a96d-ee1a6c9e382f-tmp-dir\") pod \"e2e-lifecycle-model-54e722cc-kserve-74d8796b8c-xdlbc\" (UID: \"0c02acb4-0e96-43cc-a96d-ee1a6c9e382f\") " pod="ai-tenant-e2e-mt-54e722cc/e2e-lifecycle-model-54e722cc-kserve-74d8796b8c-xdlbc" Jul 08 13:50:53.970933 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:53.970833 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c02acb4-0e96-43cc-a96d-ee1a6c9e382f-tls-certs\") pod \"e2e-lifecycle-model-54e722cc-kserve-74d8796b8c-xdlbc\" (UID: \"0c02acb4-0e96-43cc-a96d-ee1a6c9e382f\") " pod="ai-tenant-e2e-mt-54e722cc/e2e-lifecycle-model-54e722cc-kserve-74d8796b8c-xdlbc" Jul 08 13:50:53.970933 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:53.970880 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c02acb4-0e96-43cc-a96d-ee1a6c9e382f-model-cache\") pod \"e2e-lifecycle-model-54e722cc-kserve-74d8796b8c-xdlbc\" (UID: \"0c02acb4-0e96-43cc-a96d-ee1a6c9e382f\") " pod="ai-tenant-e2e-mt-54e722cc/e2e-lifecycle-model-54e722cc-kserve-74d8796b8c-xdlbc" Jul 08 13:50:53.970933 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:53.970926 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c02acb4-0e96-43cc-a96d-ee1a6c9e382f-home\") pod \"e2e-lifecycle-model-54e722cc-kserve-74d8796b8c-xdlbc\" (UID: \"0c02acb4-0e96-43cc-a96d-ee1a6c9e382f\") " pod="ai-tenant-e2e-mt-54e722cc/e2e-lifecycle-model-54e722cc-kserve-74d8796b8c-xdlbc" Jul 08 13:50:53.971065 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:53.970965 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxc6m\" (UniqueName: \"kubernetes.io/projected/0c02acb4-0e96-43cc-a96d-ee1a6c9e382f-kube-api-access-jxc6m\") pod \"e2e-lifecycle-model-54e722cc-kserve-74d8796b8c-xdlbc\" (UID: \"0c02acb4-0e96-43cc-a96d-ee1a6c9e382f\") " pod="ai-tenant-e2e-mt-54e722cc/e2e-lifecycle-model-54e722cc-kserve-74d8796b8c-xdlbc" Jul 08 13:50:54.071651 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:54.071566 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxc6m\" (UniqueName: \"kubernetes.io/projected/0c02acb4-0e96-43cc-a96d-ee1a6c9e382f-kube-api-access-jxc6m\") pod \"e2e-lifecycle-model-54e722cc-kserve-74d8796b8c-xdlbc\" (UID: \"0c02acb4-0e96-43cc-a96d-ee1a6c9e382f\") " pod="ai-tenant-e2e-mt-54e722cc/e2e-lifecycle-model-54e722cc-kserve-74d8796b8c-xdlbc" Jul 08 13:50:54.071651 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:54.071611 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c02acb4-0e96-43cc-a96d-ee1a6c9e382f-dshm\") pod \"e2e-lifecycle-model-54e722cc-kserve-74d8796b8c-xdlbc\" (UID: \"0c02acb4-0e96-43cc-a96d-ee1a6c9e382f\") " pod="ai-tenant-e2e-mt-54e722cc/e2e-lifecycle-model-54e722cc-kserve-74d8796b8c-xdlbc" Jul 08 13:50:54.071651 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:54.071638 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c02acb4-0e96-43cc-a96d-ee1a6c9e382f-tmp-dir\") pod \"e2e-lifecycle-model-54e722cc-kserve-74d8796b8c-xdlbc\" (UID: \"0c02acb4-0e96-43cc-a96d-ee1a6c9e382f\") " pod="ai-tenant-e2e-mt-54e722cc/e2e-lifecycle-model-54e722cc-kserve-74d8796b8c-xdlbc" Jul 08 13:50:54.071925 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:54.071667 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c02acb4-0e96-43cc-a96d-ee1a6c9e382f-tls-certs\") pod \"e2e-lifecycle-model-54e722cc-kserve-74d8796b8c-xdlbc\" (UID: \"0c02acb4-0e96-43cc-a96d-ee1a6c9e382f\") " pod="ai-tenant-e2e-mt-54e722cc/e2e-lifecycle-model-54e722cc-kserve-74d8796b8c-xdlbc" Jul 08 13:50:54.071925 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:54.071712 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c02acb4-0e96-43cc-a96d-ee1a6c9e382f-model-cache\") pod \"e2e-lifecycle-model-54e722cc-kserve-74d8796b8c-xdlbc\" (UID: \"0c02acb4-0e96-43cc-a96d-ee1a6c9e382f\") " pod="ai-tenant-e2e-mt-54e722cc/e2e-lifecycle-model-54e722cc-kserve-74d8796b8c-xdlbc" Jul 08 13:50:54.071925 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:54.071735 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c02acb4-0e96-43cc-a96d-ee1a6c9e382f-home\") pod \"e2e-lifecycle-model-54e722cc-kserve-74d8796b8c-xdlbc\" (UID: \"0c02acb4-0e96-43cc-a96d-ee1a6c9e382f\") " pod="ai-tenant-e2e-mt-54e722cc/e2e-lifecycle-model-54e722cc-kserve-74d8796b8c-xdlbc" Jul 08 13:50:54.072192 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:54.072171 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c02acb4-0e96-43cc-a96d-ee1a6c9e382f-model-cache\") pod \"e2e-lifecycle-model-54e722cc-kserve-74d8796b8c-xdlbc\" (UID: \"0c02acb4-0e96-43cc-a96d-ee1a6c9e382f\") " pod="ai-tenant-e2e-mt-54e722cc/e2e-lifecycle-model-54e722cc-kserve-74d8796b8c-xdlbc" Jul 08 13:50:54.072281 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:54.072221 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c02acb4-0e96-43cc-a96d-ee1a6c9e382f-home\") pod \"e2e-lifecycle-model-54e722cc-kserve-74d8796b8c-xdlbc\" (UID: \"0c02acb4-0e96-43cc-a96d-ee1a6c9e382f\") " pod="ai-tenant-e2e-mt-54e722cc/e2e-lifecycle-model-54e722cc-kserve-74d8796b8c-xdlbc" Jul 08 13:50:54.072334 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:54.072284 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c02acb4-0e96-43cc-a96d-ee1a6c9e382f-tmp-dir\") pod \"e2e-lifecycle-model-54e722cc-kserve-74d8796b8c-xdlbc\" (UID: \"0c02acb4-0e96-43cc-a96d-ee1a6c9e382f\") " pod="ai-tenant-e2e-mt-54e722cc/e2e-lifecycle-model-54e722cc-kserve-74d8796b8c-xdlbc" Jul 08 13:50:54.073844 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:54.073826 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c02acb4-0e96-43cc-a96d-ee1a6c9e382f-dshm\") pod \"e2e-lifecycle-model-54e722cc-kserve-74d8796b8c-xdlbc\" (UID: \"0c02acb4-0e96-43cc-a96d-ee1a6c9e382f\") " pod="ai-tenant-e2e-mt-54e722cc/e2e-lifecycle-model-54e722cc-kserve-74d8796b8c-xdlbc" Jul 08 13:50:54.074113 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:54.074095 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c02acb4-0e96-43cc-a96d-ee1a6c9e382f-tls-certs\") pod \"e2e-lifecycle-model-54e722cc-kserve-74d8796b8c-xdlbc\" (UID: \"0c02acb4-0e96-43cc-a96d-ee1a6c9e382f\") " pod="ai-tenant-e2e-mt-54e722cc/e2e-lifecycle-model-54e722cc-kserve-74d8796b8c-xdlbc" Jul 08 13:50:54.081541 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:54.081508 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxc6m\" (UniqueName: \"kubernetes.io/projected/0c02acb4-0e96-43cc-a96d-ee1a6c9e382f-kube-api-access-jxc6m\") pod \"e2e-lifecycle-model-54e722cc-kserve-74d8796b8c-xdlbc\" (UID: \"0c02acb4-0e96-43cc-a96d-ee1a6c9e382f\") " pod="ai-tenant-e2e-mt-54e722cc/e2e-lifecycle-model-54e722cc-kserve-74d8796b8c-xdlbc" Jul 08 13:50:54.126323 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:54.126289 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-54e722cc/e2e-lifecycle-model-54e722cc-kserve-74d8796b8c-xdlbc" Jul 08 13:50:54.279551 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:54.279519 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-54e722cc/e2e-lifecycle-model-54e722cc-kserve-74d8796b8c-xdlbc"] Jul 08 13:50:54.280697 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:50:54.280669 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0c02acb4_0e96_43cc_a96d_ee1a6c9e382f.slice/crio-df5b0e7368ace37467acb38a9cc06095df7e1954f9850c8d4d89eef8b26acf85 WatchSource:0}: Error finding container df5b0e7368ace37467acb38a9cc06095df7e1954f9850c8d4d89eef8b26acf85: Status 404 returned error can't find the container with id df5b0e7368ace37467acb38a9cc06095df7e1954f9850c8d4d89eef8b26acf85 Jul 08 13:50:54.709293 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:54.709191 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-54e722cc/e2e-lifecycle-model-54e722cc-kserve-74d8796b8c-xdlbc" event={"ID":"0c02acb4-0e96-43cc-a96d-ee1a6c9e382f","Type":"ContainerStarted","Data":"a527daa799c48f8480437bd5bb66758862acd435cec4843601cdbc6f0c77f5e6"} Jul 08 13:50:54.709465 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:54.709311 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-54e722cc/e2e-lifecycle-model-54e722cc-kserve-74d8796b8c-xdlbc" event={"ID":"0c02acb4-0e96-43cc-a96d-ee1a6c9e382f","Type":"ContainerStarted","Data":"df5b0e7368ace37467acb38a9cc06095df7e1954f9850c8d4d89eef8b26acf85"} Jul 08 13:50:54.728945 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:54.728891 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-54e722cc/e2e-lifecycle-model-54e722cc-kserve-74d8796b8c-xdlbc" podStartSLOduration=1.7288755070000001 podStartE2EDuration="1.728875507s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:50:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:50:54.727202625 +0000 UTC m=+1663.524143281" watchObservedRunningTime="2026-07-08 13:50:54.728875507 +0000 UTC m=+1663.525816160" Jul 08 13:50:58.731561 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:58.731480 2567 generic.go:356] "Generic (PLEG): container finished" podID="83719851-5080-42f7-9cc3-7b2f29e56992" containerID="c9d0a5612694594b2c381c8efa861f5e2dcc24d07a820fde79d19b8bf1d16e78" exitCode=0 Jul 08 13:50:58.731561 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:58.731548 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp" event={"ID":"83719851-5080-42f7-9cc3-7b2f29e56992","Type":"ContainerDied","Data":"c9d0a5612694594b2c381c8efa861f5e2dcc24d07a820fde79d19b8bf1d16e78"} Jul 08 13:50:58.731988 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:58.731914 2567 scope.go:117] "RemoveContainer" containerID="c9d0a5612694594b2c381c8efa861f5e2dcc24d07a820fde79d19b8bf1d16e78" Jul 08 13:50:58.732967 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:58.732933 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:50:58.733039 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:58.733021 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:50:58.733093 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:58.733062 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:50:59.645525 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:59.645483 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp" Jul 08 13:50:59.737836 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:50:59.737796 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp" event={"ID":"83719851-5080-42f7-9cc3-7b2f29e56992","Type":"ContainerStarted","Data":"7dfb9f1f35bb0093283edfd56dd0409f3d8fbbedce02f334fbfddd371552a76e"} Jul 08 13:51:00.032296 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:51:00.032264 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp" Jul 08 13:51:00.033357 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:51:00.033332 2567 patch_prober.go:28] interesting pod/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body= Jul 08 13:51:00.033476 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:51:00.033379 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp" podUID="83719851-5080-42f7-9cc3-7b2f29e56992" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" Jul 08 13:51:01.031901 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:51:01.031858 2567 patch_prober.go:28] interesting pod/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body= Jul 08 13:51:01.032322 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:51:01.031921 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp" podUID="83719851-5080-42f7-9cc3-7b2f29e56992" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" Jul 08 13:51:02.032264 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:51:02.032226 2567 patch_prober.go:28] interesting pod/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body= Jul 08 13:51:02.032665 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:51:02.032282 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp" podUID="83719851-5080-42f7-9cc3-7b2f29e56992" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" Jul 08 13:51:03.031830 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:51:03.031797 2567 patch_prober.go:28] interesting pod/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body= Jul 08 13:51:03.032014 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:51:03.031851 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp" podUID="83719851-5080-42f7-9cc3-7b2f29e56992" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" Jul 08 13:51:03.410519 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:51:03.410428 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:51:04.035956 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:51:04.035925 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp" Jul 08 13:51:04.036341 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:51:04.036320 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp" Jul 08 13:51:04.036888 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:51:04.036870 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp" Jul 08 13:51:04.127455 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:51:04.127425 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-54e722cc/e2e-lifecycle-model-54e722cc-kserve-74d8796b8c-xdlbc" Jul 08 13:51:04.127609 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:51:04.127466 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-54e722cc/e2e-lifecycle-model-54e722cc-kserve-74d8796b8c-xdlbc" Jul 08 13:51:04.140391 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:51:04.140371 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-54e722cc/e2e-lifecycle-model-54e722cc-kserve-74d8796b8c-xdlbc" Jul 08 13:51:04.773304 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:51:04.773267 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-54e722cc/e2e-lifecycle-model-54e722cc-kserve-74d8796b8c-xdlbc" Jul 08 13:51:10.803346 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:51:10.803312 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:51:23.555880 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:51:23.555840 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-54e722cc/e2e-lifecycle-model-54e722cc-kserve-74d8796b8c-xdlbc"] Jul 08 13:51:24.108421 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:51:24.108382 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:51:38.575981 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:51:38.575871 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-54e722cc/e2e-lifecycle-model-54e722cc-kserve-74d8796b8c-xdlbc" podUID="0c02acb4-0e96-43cc-a96d-ee1a6c9e382f" containerName="main" containerID="cri-o://a527daa799c48f8480437bd5bb66758862acd435cec4843601cdbc6f0c77f5e6" gracePeriod=45 Jul 08 13:51:38.815967 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:51:38.815941 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-54e722cc/e2e-lifecycle-model-54e722cc-kserve-74d8796b8c-xdlbc" Jul 08 13:51:38.876157 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:51:38.875996 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c02acb4-0e96-43cc-a96d-ee1a6c9e382f-home\") pod \"0c02acb4-0e96-43cc-a96d-ee1a6c9e382f\" (UID: \"0c02acb4-0e96-43cc-a96d-ee1a6c9e382f\") " Jul 08 13:51:38.876157 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:51:38.876033 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c02acb4-0e96-43cc-a96d-ee1a6c9e382f-tls-certs\") pod \"0c02acb4-0e96-43cc-a96d-ee1a6c9e382f\" (UID: \"0c02acb4-0e96-43cc-a96d-ee1a6c9e382f\") " Jul 08 13:51:38.876157 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:51:38.876063 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jxc6m\" (UniqueName: \"kubernetes.io/projected/0c02acb4-0e96-43cc-a96d-ee1a6c9e382f-kube-api-access-jxc6m\") pod \"0c02acb4-0e96-43cc-a96d-ee1a6c9e382f\" (UID: \"0c02acb4-0e96-43cc-a96d-ee1a6c9e382f\") " Jul 08 13:51:38.876157 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:51:38.876123 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c02acb4-0e96-43cc-a96d-ee1a6c9e382f-tmp-dir\") pod \"0c02acb4-0e96-43cc-a96d-ee1a6c9e382f\" (UID: \"0c02acb4-0e96-43cc-a96d-ee1a6c9e382f\") " Jul 08 13:51:38.876157 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:51:38.876148 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c02acb4-0e96-43cc-a96d-ee1a6c9e382f-model-cache\") pod \"0c02acb4-0e96-43cc-a96d-ee1a6c9e382f\" (UID: \"0c02acb4-0e96-43cc-a96d-ee1a6c9e382f\") " Jul 08 13:51:38.876506 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:51:38.876168 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c02acb4-0e96-43cc-a96d-ee1a6c9e382f-dshm\") pod \"0c02acb4-0e96-43cc-a96d-ee1a6c9e382f\" (UID: \"0c02acb4-0e96-43cc-a96d-ee1a6c9e382f\") " Jul 08 13:51:38.876506 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:51:38.876314 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c02acb4-0e96-43cc-a96d-ee1a6c9e382f-home" (OuterVolumeSpecName: "home") pod "0c02acb4-0e96-43cc-a96d-ee1a6c9e382f" (UID: "0c02acb4-0e96-43cc-a96d-ee1a6c9e382f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:51:38.876506 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:51:38.876401 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c02acb4-0e96-43cc-a96d-ee1a6c9e382f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0c02acb4-0e96-43cc-a96d-ee1a6c9e382f" (UID: "0c02acb4-0e96-43cc-a96d-ee1a6c9e382f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:51:38.876506 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:51:38.876421 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c02acb4-0e96-43cc-a96d-ee1a6c9e382f-model-cache" (OuterVolumeSpecName: "model-cache") pod "0c02acb4-0e96-43cc-a96d-ee1a6c9e382f" (UID: "0c02acb4-0e96-43cc-a96d-ee1a6c9e382f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:51:38.876506 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:51:38.876479 2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c02acb4-0e96-43cc-a96d-ee1a6c9e382f-home\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:51:38.876506 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:51:38.876495 2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c02acb4-0e96-43cc-a96d-ee1a6c9e382f-tmp-dir\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:51:38.876506 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:51:38.876508 2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c02acb4-0e96-43cc-a96d-ee1a6c9e382f-model-cache\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:51:38.878356 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:51:38.878325 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c02acb4-0e96-43cc-a96d-ee1a6c9e382f-dshm" (OuterVolumeSpecName: "dshm") pod "0c02acb4-0e96-43cc-a96d-ee1a6c9e382f" (UID: "0c02acb4-0e96-43cc-a96d-ee1a6c9e382f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:51:38.878461 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:51:38.878383 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c02acb4-0e96-43cc-a96d-ee1a6c9e382f-kube-api-access-jxc6m" (OuterVolumeSpecName: "kube-api-access-jxc6m") pod "0c02acb4-0e96-43cc-a96d-ee1a6c9e382f" (UID: "0c02acb4-0e96-43cc-a96d-ee1a6c9e382f"). InnerVolumeSpecName "kube-api-access-jxc6m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 13:51:38.878461 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:51:38.878385 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c02acb4-0e96-43cc-a96d-ee1a6c9e382f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0c02acb4-0e96-43cc-a96d-ee1a6c9e382f" (UID: "0c02acb4-0e96-43cc-a96d-ee1a6c9e382f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 13:51:38.923851 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:51:38.923823 2567 generic.go:356] "Generic (PLEG): container finished" podID="0c02acb4-0e96-43cc-a96d-ee1a6c9e382f" containerID="a527daa799c48f8480437bd5bb66758862acd435cec4843601cdbc6f0c77f5e6" exitCode=0 Jul 08 13:51:38.923994 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:51:38.923863 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-54e722cc/e2e-lifecycle-model-54e722cc-kserve-74d8796b8c-xdlbc" Jul 08 13:51:38.923994 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:51:38.923886 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-54e722cc/e2e-lifecycle-model-54e722cc-kserve-74d8796b8c-xdlbc" event={"ID":"0c02acb4-0e96-43cc-a96d-ee1a6c9e382f","Type":"ContainerDied","Data":"a527daa799c48f8480437bd5bb66758862acd435cec4843601cdbc6f0c77f5e6"} Jul 08 13:51:38.923994 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:51:38.923931 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-54e722cc/e2e-lifecycle-model-54e722cc-kserve-74d8796b8c-xdlbc" event={"ID":"0c02acb4-0e96-43cc-a96d-ee1a6c9e382f","Type":"ContainerDied","Data":"df5b0e7368ace37467acb38a9cc06095df7e1954f9850c8d4d89eef8b26acf85"} Jul 08 13:51:38.923994 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:51:38.923948 2567 scope.go:117] "RemoveContainer" containerID="a527daa799c48f8480437bd5bb66758862acd435cec4843601cdbc6f0c77f5e6" Jul 08 13:51:38.932430 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:51:38.932414 2567 scope.go:117] "RemoveContainer" containerID="a527daa799c48f8480437bd5bb66758862acd435cec4843601cdbc6f0c77f5e6" Jul 08 13:51:38.932690 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:51:38.932672 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a527daa799c48f8480437bd5bb66758862acd435cec4843601cdbc6f0c77f5e6\": container with ID starting with a527daa799c48f8480437bd5bb66758862acd435cec4843601cdbc6f0c77f5e6 not found: ID does not exist" containerID="a527daa799c48f8480437bd5bb66758862acd435cec4843601cdbc6f0c77f5e6" Jul 08 13:51:38.932745 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:51:38.932700 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a527daa799c48f8480437bd5bb66758862acd435cec4843601cdbc6f0c77f5e6"} err="failed to get container status \"a527daa799c48f8480437bd5bb66758862acd435cec4843601cdbc6f0c77f5e6\": rpc error: code = NotFound desc = could not find container \"a527daa799c48f8480437bd5bb66758862acd435cec4843601cdbc6f0c77f5e6\": container with ID starting with a527daa799c48f8480437bd5bb66758862acd435cec4843601cdbc6f0c77f5e6 not found: ID does not exist" Jul 08 13:51:38.944373 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:51:38.944351 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-54e722cc/e2e-lifecycle-model-54e722cc-kserve-74d8796b8c-xdlbc"] Jul 08 13:51:38.948688 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:51:38.948669 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-54e722cc/e2e-lifecycle-model-54e722cc-kserve-74d8796b8c-xdlbc"] Jul 08 13:51:38.976936 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:51:38.976911 2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c02acb4-0e96-43cc-a96d-ee1a6c9e382f-tls-certs\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:51:38.976936 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:51:38.976933 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jxc6m\" (UniqueName: \"kubernetes.io/projected/0c02acb4-0e96-43cc-a96d-ee1a6c9e382f-kube-api-access-jxc6m\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:51:38.977131 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:51:38.976943 2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c02acb4-0e96-43cc-a96d-ee1a6c9e382f-dshm\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:51:39.797018 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:51:39.796988 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c02acb4-0e96-43cc-a96d-ee1a6c9e382f" path="/var/lib/kubelet/pods/0c02acb4-0e96-43cc-a96d-ee1a6c9e382f/volumes" Jul 08 13:52:03.294420 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:52:03.294382 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:52:12.655973 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:52:12.655936 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp"] Jul 08 13:52:12.656392 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:52:12.656212 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp" podUID="83719851-5080-42f7-9cc3-7b2f29e56992" containerName="istio-proxy" containerID="cri-o://7dfb9f1f35bb0093283edfd56dd0409f3d8fbbedce02f334fbfddd371552a76e" gracePeriod=30 Jul 08 13:52:17.303778 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:52:17.303740 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:52:17.919008 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:52:17.918934 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp" Jul 08 13:52:17.989965 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:52:17.989933 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83719851-5080-42f7-9cc3-7b2f29e56992-istio-data\") pod \"83719851-5080-42f7-9cc3-7b2f29e56992\" (UID: \"83719851-5080-42f7-9cc3-7b2f29e56992\") " Jul 08 13:52:17.989965 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:52:17.989972 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83719851-5080-42f7-9cc3-7b2f29e56992-credential-socket\") pod \"83719851-5080-42f7-9cc3-7b2f29e56992\" (UID: \"83719851-5080-42f7-9cc3-7b2f29e56992\") " Jul 08 13:52:17.990237 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:52:17.990000 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83719851-5080-42f7-9cc3-7b2f29e56992-istio-podinfo\") pod \"83719851-5080-42f7-9cc3-7b2f29e56992\" (UID: \"83719851-5080-42f7-9cc3-7b2f29e56992\") " Jul 08 13:52:17.990237 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:52:17.990034 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83719851-5080-42f7-9cc3-7b2f29e56992-istiod-ca-cert\") pod \"83719851-5080-42f7-9cc3-7b2f29e56992\" (UID: \"83719851-5080-42f7-9cc3-7b2f29e56992\") " Jul 08 13:52:17.990237 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:52:17.990061 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rx9jj\" (UniqueName: \"kubernetes.io/projected/83719851-5080-42f7-9cc3-7b2f29e56992-kube-api-access-rx9jj\") pod \"83719851-5080-42f7-9cc3-7b2f29e56992\" (UID: \"83719851-5080-42f7-9cc3-7b2f29e56992\") " Jul 08 13:52:17.990237 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:52:17.990116 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83719851-5080-42f7-9cc3-7b2f29e56992-istio-token\") pod \"83719851-5080-42f7-9cc3-7b2f29e56992\" (UID: \"83719851-5080-42f7-9cc3-7b2f29e56992\") " Jul 08 13:52:17.990237 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:52:17.990177 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83719851-5080-42f7-9cc3-7b2f29e56992-workload-socket\") pod \"83719851-5080-42f7-9cc3-7b2f29e56992\" (UID: \"83719851-5080-42f7-9cc3-7b2f29e56992\") " Jul 08 13:52:17.990237 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:52:17.990203 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83719851-5080-42f7-9cc3-7b2f29e56992-istio-envoy\") pod \"83719851-5080-42f7-9cc3-7b2f29e56992\" (UID: \"83719851-5080-42f7-9cc3-7b2f29e56992\") " Jul 08 13:52:17.990534 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:52:17.990254 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83719851-5080-42f7-9cc3-7b2f29e56992-workload-certs\") pod \"83719851-5080-42f7-9cc3-7b2f29e56992\" (UID: \"83719851-5080-42f7-9cc3-7b2f29e56992\") " Jul 08 13:52:17.990534 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:52:17.990325 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83719851-5080-42f7-9cc3-7b2f29e56992-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "83719851-5080-42f7-9cc3-7b2f29e56992" (UID: "83719851-5080-42f7-9cc3-7b2f29e56992"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:52:17.990691 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:52:17.990551 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83719851-5080-42f7-9cc3-7b2f29e56992-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "83719851-5080-42f7-9cc3-7b2f29e56992" (UID: "83719851-5080-42f7-9cc3-7b2f29e56992"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:52:17.990691 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:52:17.990555 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83719851-5080-42f7-9cc3-7b2f29e56992-credential-socket\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:52:17.990691 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:52:17.990575 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83719851-5080-42f7-9cc3-7b2f29e56992-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "83719851-5080-42f7-9cc3-7b2f29e56992" (UID: "83719851-5080-42f7-9cc3-7b2f29e56992"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 13:52:17.990691 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:52:17.990661 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83719851-5080-42f7-9cc3-7b2f29e56992-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "83719851-5080-42f7-9cc3-7b2f29e56992" (UID: "83719851-5080-42f7-9cc3-7b2f29e56992"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:52:17.990892 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:52:17.990821 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83719851-5080-42f7-9cc3-7b2f29e56992-istio-data" (OuterVolumeSpecName: "istio-data") pod "83719851-5080-42f7-9cc3-7b2f29e56992" (UID: "83719851-5080-42f7-9cc3-7b2f29e56992"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:52:17.992469 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:52:17.992424 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83719851-5080-42f7-9cc3-7b2f29e56992-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "83719851-5080-42f7-9cc3-7b2f29e56992" (UID: "83719851-5080-42f7-9cc3-7b2f29e56992"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:52:17.992594 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:52:17.992481 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/83719851-5080-42f7-9cc3-7b2f29e56992-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "83719851-5080-42f7-9cc3-7b2f29e56992" (UID: "83719851-5080-42f7-9cc3-7b2f29e56992"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 13:52:17.992751 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:52:17.992726 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83719851-5080-42f7-9cc3-7b2f29e56992-istio-token" (OuterVolumeSpecName: "istio-token") pod "83719851-5080-42f7-9cc3-7b2f29e56992" (UID: "83719851-5080-42f7-9cc3-7b2f29e56992"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 13:52:17.992823 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:52:17.992743 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83719851-5080-42f7-9cc3-7b2f29e56992-kube-api-access-rx9jj" (OuterVolumeSpecName: "kube-api-access-rx9jj") pod "83719851-5080-42f7-9cc3-7b2f29e56992" (UID: "83719851-5080-42f7-9cc3-7b2f29e56992"). InnerVolumeSpecName "kube-api-access-rx9jj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 13:52:18.091765 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:52:18.091729 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83719851-5080-42f7-9cc3-7b2f29e56992-workload-certs\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:52:18.091765 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:52:18.091758 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83719851-5080-42f7-9cc3-7b2f29e56992-istio-data\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:52:18.091765 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:52:18.091767 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83719851-5080-42f7-9cc3-7b2f29e56992-istio-podinfo\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:52:18.091765 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:52:18.091776 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83719851-5080-42f7-9cc3-7b2f29e56992-istiod-ca-cert\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:52:18.092155 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:52:18.091786 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rx9jj\" (UniqueName: \"kubernetes.io/projected/83719851-5080-42f7-9cc3-7b2f29e56992-kube-api-access-rx9jj\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:52:18.092155 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:52:18.091795 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83719851-5080-42f7-9cc3-7b2f29e56992-istio-token\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:52:18.092155 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:52:18.091804 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83719851-5080-42f7-9cc3-7b2f29e56992-workload-socket\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:52:18.092155 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:52:18.091812 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83719851-5080-42f7-9cc3-7b2f29e56992-istio-envoy\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:52:18.115834 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:52:18.115796 2567 generic.go:356] "Generic (PLEG): container finished" podID="83719851-5080-42f7-9cc3-7b2f29e56992" containerID="7dfb9f1f35bb0093283edfd56dd0409f3d8fbbedce02f334fbfddd371552a76e" exitCode=0 Jul 08 13:52:18.116010 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:52:18.115840 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp" Jul 08 13:52:18.116010 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:52:18.115878 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp" event={"ID":"83719851-5080-42f7-9cc3-7b2f29e56992","Type":"ContainerDied","Data":"7dfb9f1f35bb0093283edfd56dd0409f3d8fbbedce02f334fbfddd371552a76e"} Jul 08 13:52:18.116010 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:52:18.115932 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp" event={"ID":"83719851-5080-42f7-9cc3-7b2f29e56992","Type":"ContainerDied","Data":"704940ee4289130cfd3ee9766efe5dea230a2e93271adb5a0c92910ec47391c8"} Jul 08 13:52:18.116010 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:52:18.115956 2567 scope.go:117] "RemoveContainer" containerID="7dfb9f1f35bb0093283edfd56dd0409f3d8fbbedce02f334fbfddd371552a76e" Jul 08 13:52:18.125193 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:52:18.125171 2567 scope.go:117] "RemoveContainer" containerID="c9d0a5612694594b2c381c8efa861f5e2dcc24d07a820fde79d19b8bf1d16e78" Jul 08 13:52:18.133333 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:52:18.133313 2567 scope.go:117] "RemoveContainer" containerID="7dfb9f1f35bb0093283edfd56dd0409f3d8fbbedce02f334fbfddd371552a76e" Jul 08 13:52:18.133617 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:52:18.133598 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7dfb9f1f35bb0093283edfd56dd0409f3d8fbbedce02f334fbfddd371552a76e\": container with ID starting with 7dfb9f1f35bb0093283edfd56dd0409f3d8fbbedce02f334fbfddd371552a76e not found: ID does not exist" containerID="7dfb9f1f35bb0093283edfd56dd0409f3d8fbbedce02f334fbfddd371552a76e" Jul 08 13:52:18.133664 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:52:18.133626 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7dfb9f1f35bb0093283edfd56dd0409f3d8fbbedce02f334fbfddd371552a76e"} err="failed to get container status \"7dfb9f1f35bb0093283edfd56dd0409f3d8fbbedce02f334fbfddd371552a76e\": rpc error: code = NotFound desc = could not find container \"7dfb9f1f35bb0093283edfd56dd0409f3d8fbbedce02f334fbfddd371552a76e\": container with ID starting with 7dfb9f1f35bb0093283edfd56dd0409f3d8fbbedce02f334fbfddd371552a76e not found: ID does not exist" Jul 08 13:52:18.133664 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:52:18.133644 2567 scope.go:117] "RemoveContainer" containerID="c9d0a5612694594b2c381c8efa861f5e2dcc24d07a820fde79d19b8bf1d16e78" Jul 08 13:52:18.133876 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:52:18.133857 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9d0a5612694594b2c381c8efa861f5e2dcc24d07a820fde79d19b8bf1d16e78\": container with ID starting with c9d0a5612694594b2c381c8efa861f5e2dcc24d07a820fde79d19b8bf1d16e78 not found: ID does not exist" containerID="c9d0a5612694594b2c381c8efa861f5e2dcc24d07a820fde79d19b8bf1d16e78" Jul 08 13:52:18.133963 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:52:18.133882 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9d0a5612694594b2c381c8efa861f5e2dcc24d07a820fde79d19b8bf1d16e78"} err="failed to get container status \"c9d0a5612694594b2c381c8efa861f5e2dcc24d07a820fde79d19b8bf1d16e78\": rpc error: code = NotFound desc = could not find container \"c9d0a5612694594b2c381c8efa861f5e2dcc24d07a820fde79d19b8bf1d16e78\": container with ID starting with c9d0a5612694594b2c381c8efa861f5e2dcc24d07a820fde79d19b8bf1d16e78 not found: ID does not exist" Jul 08 13:52:18.147631 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:52:18.147602 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp"] Jul 08 13:52:18.155590 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:52:18.155562 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-54e722cc-openshift-default-d59bb4b6c-9rmjp"] Jul 08 13:52:19.796133 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:52:19.796097 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83719851-5080-42f7-9cc3-7b2f29e56992" path="/var/lib/kubelet/pods/83719851-5080-42f7-9cc3-7b2f29e56992/volumes" Jul 08 13:52:30.694295 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:52:30.694261 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:53:07.846349 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:07.846269 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz"] Jul 08 13:53:07.846842 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:07.846620 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83719851-5080-42f7-9cc3-7b2f29e56992" containerName="istio-proxy" Jul 08 13:53:07.846842 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:07.846637 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="83719851-5080-42f7-9cc3-7b2f29e56992" containerName="istio-proxy" Jul 08 13:53:07.846842 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:07.846664 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c02acb4-0e96-43cc-a96d-ee1a6c9e382f" containerName="main" Jul 08 13:53:07.846842 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:07.846669 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c02acb4-0e96-43cc-a96d-ee1a6c9e382f" containerName="main" Jul 08 13:53:07.846842 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:07.846682 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83719851-5080-42f7-9cc3-7b2f29e56992" containerName="istio-proxy" Jul 08 13:53:07.846842 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:07.846688 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="83719851-5080-42f7-9cc3-7b2f29e56992" containerName="istio-proxy" Jul 08 13:53:07.846842 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:07.846740 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c02acb4-0e96-43cc-a96d-ee1a6c9e382f" containerName="main" Jul 08 13:53:07.846842 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:07.846747 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="83719851-5080-42f7-9cc3-7b2f29e56992" containerName="istio-proxy" Jul 08 13:53:07.846842 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:07.846754 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="83719851-5080-42f7-9cc3-7b2f29e56992" containerName="istio-proxy" Jul 08 13:53:07.849759 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:07.849737 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz" Jul 08 13:53:07.852986 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:07.852963 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-ac32a0-openshift-default-dockercfg-svsnn\"" Jul 08 13:53:07.862799 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:07.862773 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz"] Jul 08 13:53:07.917643 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:07.917600 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb4bd630-5b36-435f-8d68-87876a2f5a99-istiod-ca-cert\") pod \"e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz\" (UID: \"eb4bd630-5b36-435f-8d68-87876a2f5a99\") " pod="openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz" Jul 08 13:53:07.917643 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:07.917649 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb4bd630-5b36-435f-8d68-87876a2f5a99-istio-envoy\") pod \"e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz\" (UID: \"eb4bd630-5b36-435f-8d68-87876a2f5a99\") " pod="openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz" Jul 08 13:53:07.917866 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:07.917666 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb4bd630-5b36-435f-8d68-87876a2f5a99-istio-podinfo\") pod \"e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz\" (UID: \"eb4bd630-5b36-435f-8d68-87876a2f5a99\") " pod="openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz" Jul 08 13:53:07.917866 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:07.917719 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb4bd630-5b36-435f-8d68-87876a2f5a99-credential-socket\") pod \"e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz\" (UID: \"eb4bd630-5b36-435f-8d68-87876a2f5a99\") " pod="openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz" Jul 08 13:53:07.917866 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:07.917758 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb4bd630-5b36-435f-8d68-87876a2f5a99-workload-socket\") pod \"e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz\" (UID: \"eb4bd630-5b36-435f-8d68-87876a2f5a99\") " pod="openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz" Jul 08 13:53:07.917866 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:07.917780 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb4bd630-5b36-435f-8d68-87876a2f5a99-workload-certs\") pod \"e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz\" (UID: \"eb4bd630-5b36-435f-8d68-87876a2f5a99\") " pod="openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz" Jul 08 13:53:07.917866 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:07.917843 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb4bd630-5b36-435f-8d68-87876a2f5a99-istio-data\") pod \"e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz\" (UID: \"eb4bd630-5b36-435f-8d68-87876a2f5a99\") " pod="openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz" Jul 08 13:53:07.917866 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:07.917867 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb4bd630-5b36-435f-8d68-87876a2f5a99-istio-token\") pod \"e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz\" (UID: \"eb4bd630-5b36-435f-8d68-87876a2f5a99\") " pod="openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz" Jul 08 13:53:07.918067 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:07.917885 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7zcn\" (UniqueName: \"kubernetes.io/projected/eb4bd630-5b36-435f-8d68-87876a2f5a99-kube-api-access-r7zcn\") pod \"e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz\" (UID: \"eb4bd630-5b36-435f-8d68-87876a2f5a99\") " pod="openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz" Jul 08 13:53:08.018508 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:08.018476 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7zcn\" (UniqueName: \"kubernetes.io/projected/eb4bd630-5b36-435f-8d68-87876a2f5a99-kube-api-access-r7zcn\") pod \"e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz\" (UID: \"eb4bd630-5b36-435f-8d68-87876a2f5a99\") " pod="openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz" Jul 08 13:53:08.018707 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:08.018522 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb4bd630-5b36-435f-8d68-87876a2f5a99-istiod-ca-cert\") pod \"e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz\" (UID: \"eb4bd630-5b36-435f-8d68-87876a2f5a99\") " pod="openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz" Jul 08 13:53:08.018707 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:08.018550 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb4bd630-5b36-435f-8d68-87876a2f5a99-istio-envoy\") pod \"e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz\" (UID: \"eb4bd630-5b36-435f-8d68-87876a2f5a99\") " pod="openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz" Jul 08 13:53:08.018707 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:08.018575 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb4bd630-5b36-435f-8d68-87876a2f5a99-istio-podinfo\") pod \"e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz\" (UID: \"eb4bd630-5b36-435f-8d68-87876a2f5a99\") " pod="openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz" Jul 08 13:53:08.018707 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:08.018599 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb4bd630-5b36-435f-8d68-87876a2f5a99-credential-socket\") pod \"e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz\" (UID: \"eb4bd630-5b36-435f-8d68-87876a2f5a99\") " pod="openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz" Jul 08 13:53:08.018707 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:08.018665 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb4bd630-5b36-435f-8d68-87876a2f5a99-workload-socket\") pod \"e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz\" (UID: \"eb4bd630-5b36-435f-8d68-87876a2f5a99\") " pod="openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz" Jul 08 13:53:08.018987 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:08.018716 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb4bd630-5b36-435f-8d68-87876a2f5a99-workload-certs\") pod \"e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz\" (UID: \"eb4bd630-5b36-435f-8d68-87876a2f5a99\") " pod="openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz" Jul 08 13:53:08.018987 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:08.018771 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb4bd630-5b36-435f-8d68-87876a2f5a99-istio-data\") pod \"e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz\" (UID: \"eb4bd630-5b36-435f-8d68-87876a2f5a99\") " pod="openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz" Jul 08 13:53:08.018987 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:08.018816 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb4bd630-5b36-435f-8d68-87876a2f5a99-istio-token\") pod \"e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz\" (UID: \"eb4bd630-5b36-435f-8d68-87876a2f5a99\") " pod="openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz" Jul 08 13:53:08.019183 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:08.019035 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb4bd630-5b36-435f-8d68-87876a2f5a99-credential-socket\") pod \"e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz\" (UID: \"eb4bd630-5b36-435f-8d68-87876a2f5a99\") " pod="openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz" Jul 08 13:53:08.019183 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:08.019130 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb4bd630-5b36-435f-8d68-87876a2f5a99-workload-certs\") pod \"e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz\" (UID: \"eb4bd630-5b36-435f-8d68-87876a2f5a99\") " pod="openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz" Jul 08 13:53:08.019326 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:08.019300 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb4bd630-5b36-435f-8d68-87876a2f5a99-workload-socket\") pod \"e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz\" (UID: \"eb4bd630-5b36-435f-8d68-87876a2f5a99\") " pod="openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz" Jul 08 13:53:08.019580 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:08.019475 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb4bd630-5b36-435f-8d68-87876a2f5a99-istiod-ca-cert\") pod \"e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz\" (UID: \"eb4bd630-5b36-435f-8d68-87876a2f5a99\") " pod="openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz" Jul 08 13:53:08.019580 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:08.019508 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb4bd630-5b36-435f-8d68-87876a2f5a99-istio-data\") pod \"e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz\" (UID: \"eb4bd630-5b36-435f-8d68-87876a2f5a99\") " pod="openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz" Jul 08 13:53:08.021414 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:08.021387 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb4bd630-5b36-435f-8d68-87876a2f5a99-istio-envoy\") pod \"e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz\" (UID: \"eb4bd630-5b36-435f-8d68-87876a2f5a99\") " pod="openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz" Jul 08 13:53:08.021605 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:08.021584 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb4bd630-5b36-435f-8d68-87876a2f5a99-istio-podinfo\") pod \"e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz\" (UID: \"eb4bd630-5b36-435f-8d68-87876a2f5a99\") " pod="openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz" Jul 08 13:53:08.037041 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:08.037014 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb4bd630-5b36-435f-8d68-87876a2f5a99-istio-token\") pod \"e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz\" (UID: \"eb4bd630-5b36-435f-8d68-87876a2f5a99\") " pod="openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz" Jul 08 13:53:08.038275 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:08.038253 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7zcn\" (UniqueName: \"kubernetes.io/projected/eb4bd630-5b36-435f-8d68-87876a2f5a99-kube-api-access-r7zcn\") pod \"e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz\" (UID: \"eb4bd630-5b36-435f-8d68-87876a2f5a99\") " pod="openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz" Jul 08 13:53:08.160048 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:08.159942 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz" Jul 08 13:53:08.293558 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:08.293530 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz"] Jul 08 13:53:08.294878 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:53:08.294852 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb4bd630_5b36_435f_8d68_87876a2f5a99.slice/crio-026f025a0595ea385ce3564c109826fbe0b0f72ad19e7106d9f39bd97f23ee62 WatchSource:0}: Error finding container 026f025a0595ea385ce3564c109826fbe0b0f72ad19e7106d9f39bd97f23ee62: Status 404 returned error can't find the container with id 026f025a0595ea385ce3564c109826fbe0b0f72ad19e7106d9f39bd97f23ee62 Jul 08 13:53:08.297000 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:08.296972 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:53:08.297061 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:08.297033 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:53:08.297133 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:08.297062 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:53:08.369518 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:08.369493 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz" event={"ID":"eb4bd630-5b36-435f-8d68-87876a2f5a99","Type":"ContainerStarted","Data":"c7c4ca29089f88d9a096efe443ffbe546ff14ddfa5843fd205700bbc8bbd4d2e"} Jul 08 13:53:08.369634 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:08.369549 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz" event={"ID":"eb4bd630-5b36-435f-8d68-87876a2f5a99","Type":"ContainerStarted","Data":"026f025a0595ea385ce3564c109826fbe0b0f72ad19e7106d9f39bd97f23ee62"} Jul 08 13:53:08.388647 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:08.388587 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz" podStartSLOduration=1.388565349 podStartE2EDuration="1.388565349s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:53:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:53:08.38723919 +0000 UTC m=+1797.184179850" watchObservedRunningTime="2026-07-08 13:53:08.388565349 +0000 UTC m=+1797.185506007" Jul 08 13:53:09.160900 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:09.160871 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz" Jul 08 13:53:09.165927 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:09.165906 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz" Jul 08 13:53:09.375224 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:09.375182 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz" Jul 08 13:53:09.376059 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:09.376039 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz" Jul 08 13:53:15.451126 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:15.451093 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl"] Jul 08 13:53:15.454562 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:15.454539 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl" Jul 08 13:53:15.457600 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:15.457578 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-8f6d76-openshift-default-dockercfg-tnr9p\"" Jul 08 13:53:15.469201 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:15.469175 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl"] Jul 08 13:53:15.480344 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:15.480321 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86d728fd-be08-46cc-8d15-7135f6ae561a-istio-envoy\") pod \"e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl\" (UID: \"86d728fd-be08-46cc-8d15-7135f6ae561a\") " pod="openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl" Jul 08 13:53:15.480465 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:15.480380 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86d728fd-be08-46cc-8d15-7135f6ae561a-istio-token\") pod \"e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl\" (UID: \"86d728fd-be08-46cc-8d15-7135f6ae561a\") " pod="openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl" Jul 08 13:53:15.480515 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:15.480463 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86d728fd-be08-46cc-8d15-7135f6ae561a-workload-socket\") pod \"e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl\" (UID: \"86d728fd-be08-46cc-8d15-7135f6ae561a\") " pod="openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl" Jul 08 13:53:15.480562 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:15.480507 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86d728fd-be08-46cc-8d15-7135f6ae561a-credential-socket\") pod \"e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl\" (UID: \"86d728fd-be08-46cc-8d15-7135f6ae561a\") " pod="openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl" Jul 08 13:53:15.480607 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:15.480566 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86d728fd-be08-46cc-8d15-7135f6ae561a-istio-podinfo\") pod \"e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl\" (UID: \"86d728fd-be08-46cc-8d15-7135f6ae561a\") " pod="openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl" Jul 08 13:53:15.480607 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:15.480592 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86d728fd-be08-46cc-8d15-7135f6ae561a-workload-certs\") pod \"e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl\" (UID: \"86d728fd-be08-46cc-8d15-7135f6ae561a\") " pod="openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl" Jul 08 13:53:15.480696 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:15.480607 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86d728fd-be08-46cc-8d15-7135f6ae561a-istiod-ca-cert\") pod \"e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl\" (UID: \"86d728fd-be08-46cc-8d15-7135f6ae561a\") " pod="openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl" Jul 08 13:53:15.480696 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:15.480634 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tc6hz\" (UniqueName: \"kubernetes.io/projected/86d728fd-be08-46cc-8d15-7135f6ae561a-kube-api-access-tc6hz\") pod \"e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl\" (UID: \"86d728fd-be08-46cc-8d15-7135f6ae561a\") " pod="openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl" Jul 08 13:53:15.480781 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:15.480721 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86d728fd-be08-46cc-8d15-7135f6ae561a-istio-data\") pod \"e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl\" (UID: \"86d728fd-be08-46cc-8d15-7135f6ae561a\") " pod="openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl" Jul 08 13:53:15.581288 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:15.581252 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86d728fd-be08-46cc-8d15-7135f6ae561a-istio-data\") pod \"e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl\" (UID: \"86d728fd-be08-46cc-8d15-7135f6ae561a\") " pod="openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl" Jul 08 13:53:15.581480 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:15.581299 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86d728fd-be08-46cc-8d15-7135f6ae561a-istio-envoy\") pod \"e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl\" (UID: \"86d728fd-be08-46cc-8d15-7135f6ae561a\") " pod="openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl" Jul 08 13:53:15.581480 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:15.581324 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86d728fd-be08-46cc-8d15-7135f6ae561a-istio-token\") pod \"e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl\" (UID: \"86d728fd-be08-46cc-8d15-7135f6ae561a\") " pod="openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl" Jul 08 13:53:15.581480 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:15.581359 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86d728fd-be08-46cc-8d15-7135f6ae561a-workload-socket\") pod \"e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl\" (UID: \"86d728fd-be08-46cc-8d15-7135f6ae561a\") " pod="openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl" Jul 08 13:53:15.581480 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:15.581377 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86d728fd-be08-46cc-8d15-7135f6ae561a-credential-socket\") pod \"e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl\" (UID: \"86d728fd-be08-46cc-8d15-7135f6ae561a\") " pod="openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl" Jul 08 13:53:15.581480 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:15.581394 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86d728fd-be08-46cc-8d15-7135f6ae561a-istio-podinfo\") pod \"e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl\" (UID: \"86d728fd-be08-46cc-8d15-7135f6ae561a\") " pod="openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl" Jul 08 13:53:15.581480 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:15.581419 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86d728fd-be08-46cc-8d15-7135f6ae561a-workload-certs\") pod \"e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl\" (UID: \"86d728fd-be08-46cc-8d15-7135f6ae561a\") " pod="openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl" Jul 08 13:53:15.581480 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:15.581443 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86d728fd-be08-46cc-8d15-7135f6ae561a-istiod-ca-cert\") pod \"e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl\" (UID: \"86d728fd-be08-46cc-8d15-7135f6ae561a\") " pod="openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl" Jul 08 13:53:15.581823 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:15.581659 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tc6hz\" (UniqueName: \"kubernetes.io/projected/86d728fd-be08-46cc-8d15-7135f6ae561a-kube-api-access-tc6hz\") pod \"e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl\" (UID: \"86d728fd-be08-46cc-8d15-7135f6ae561a\") " pod="openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl" Jul 08 13:53:15.581823 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:15.581758 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86d728fd-be08-46cc-8d15-7135f6ae561a-istio-data\") pod \"e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl\" (UID: \"86d728fd-be08-46cc-8d15-7135f6ae561a\") " pod="openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl" Jul 08 13:53:15.581942 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:15.581831 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86d728fd-be08-46cc-8d15-7135f6ae561a-workload-socket\") pod \"e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl\" (UID: \"86d728fd-be08-46cc-8d15-7135f6ae561a\") " pod="openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl" Jul 08 13:53:15.581942 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:15.581851 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86d728fd-be08-46cc-8d15-7135f6ae561a-workload-certs\") pod \"e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl\" (UID: \"86d728fd-be08-46cc-8d15-7135f6ae561a\") " pod="openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl" Jul 08 13:53:15.582034 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:15.581955 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86d728fd-be08-46cc-8d15-7135f6ae561a-credential-socket\") pod \"e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl\" (UID: \"86d728fd-be08-46cc-8d15-7135f6ae561a\") " pod="openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl" Jul 08 13:53:15.582375 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:15.582351 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86d728fd-be08-46cc-8d15-7135f6ae561a-istiod-ca-cert\") pod \"e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl\" (UID: \"86d728fd-be08-46cc-8d15-7135f6ae561a\") " pod="openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl" Jul 08 13:53:15.583885 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:15.583868 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86d728fd-be08-46cc-8d15-7135f6ae561a-istio-envoy\") pod \"e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl\" (UID: \"86d728fd-be08-46cc-8d15-7135f6ae561a\") " pod="openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl" Jul 08 13:53:15.583999 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:15.583980 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86d728fd-be08-46cc-8d15-7135f6ae561a-istio-podinfo\") pod \"e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl\" (UID: \"86d728fd-be08-46cc-8d15-7135f6ae561a\") " pod="openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl" Jul 08 13:53:15.590908 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:15.590883 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86d728fd-be08-46cc-8d15-7135f6ae561a-istio-token\") pod \"e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl\" (UID: \"86d728fd-be08-46cc-8d15-7135f6ae561a\") " pod="openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl" Jul 08 13:53:15.590908 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:15.590899 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tc6hz\" (UniqueName: \"kubernetes.io/projected/86d728fd-be08-46cc-8d15-7135f6ae561a-kube-api-access-tc6hz\") pod \"e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl\" (UID: \"86d728fd-be08-46cc-8d15-7135f6ae561a\") " pod="openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl" Jul 08 13:53:15.764670 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:15.764637 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl" Jul 08 13:53:15.904654 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:15.904628 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl"] Jul 08 13:53:15.906331 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:53:15.906301 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86d728fd_be08_46cc_8d15_7135f6ae561a.slice/crio-cd0d64eb8e789973bb450735be5e3bb74de14eaa85443f53b09dc6e3b22570cd WatchSource:0}: Error finding container cd0d64eb8e789973bb450735be5e3bb74de14eaa85443f53b09dc6e3b22570cd: Status 404 returned error can't find the container with id cd0d64eb8e789973bb450735be5e3bb74de14eaa85443f53b09dc6e3b22570cd Jul 08 13:53:15.908657 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:15.908622 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:53:15.908738 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:15.908720 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:53:15.908776 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:15.908762 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:53:16.410418 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:16.410380 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl" event={"ID":"86d728fd-be08-46cc-8d15-7135f6ae561a","Type":"ContainerStarted","Data":"73490c35996b638e4abbfeee26e04a840b19702e9bfac61cdb8e3e6e094a9426"} Jul 08 13:53:16.410622 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:16.410433 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl" event={"ID":"86d728fd-be08-46cc-8d15-7135f6ae561a","Type":"ContainerStarted","Data":"cd0d64eb8e789973bb450735be5e3bb74de14eaa85443f53b09dc6e3b22570cd"} Jul 08 13:53:16.431351 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:16.431292 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl" podStartSLOduration=1.431274167 podStartE2EDuration="1.431274167s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:53:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:53:16.430410423 +0000 UTC m=+1805.227351080" watchObservedRunningTime="2026-07-08 13:53:16.431274167 +0000 UTC m=+1805.228214842" Jul 08 13:53:16.765546 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:16.765514 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl" Jul 08 13:53:16.770380 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:16.770359 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl" Jul 08 13:53:17.415388 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:17.415355 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl" Jul 08 13:53:17.416392 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:17.416375 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl" Jul 08 13:53:17.695340 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:17.695254 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:53:25.417287 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:25.417244 2567 patch_prober.go:28] interesting pod/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 08 13:53:25.417696 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:25.417319 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz" podUID="eb4bd630-5b36-435f-8d68-87876a2f5a99" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 08 13:53:26.464579 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:26.464547 2567 generic.go:356] "Generic (PLEG): container finished" podID="eb4bd630-5b36-435f-8d68-87876a2f5a99" containerID="c7c4ca29089f88d9a096efe443ffbe546ff14ddfa5843fd205700bbc8bbd4d2e" exitCode=0 Jul 08 13:53:26.465013 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:26.464627 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz" event={"ID":"eb4bd630-5b36-435f-8d68-87876a2f5a99","Type":"ContainerDied","Data":"c7c4ca29089f88d9a096efe443ffbe546ff14ddfa5843fd205700bbc8bbd4d2e"} Jul 08 13:53:26.465055 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:26.465013 2567 scope.go:117] "RemoveContainer" containerID="c7c4ca29089f88d9a096efe443ffbe546ff14ddfa5843fd205700bbc8bbd4d2e" Jul 08 13:53:26.466017 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:26.465991 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:53:26.466105 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:26.466057 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:53:26.466148 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:26.466112 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:53:27.471198 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:27.471165 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz" event={"ID":"eb4bd630-5b36-435f-8d68-87876a2f5a99","Type":"ContainerStarted","Data":"c1589ad7118f1569a3590c4244a3e41d32994c95bb551a7c415f955c5e19c82d"} Jul 08 13:53:28.161108 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:28.161039 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz" Jul 08 13:53:28.161313 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:28.161203 2567 patch_prober.go:28] interesting pod/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Jul 08 13:53:28.161313 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:28.161252 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz" podUID="eb4bd630-5b36-435f-8d68-87876a2f5a99" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Jul 08 13:53:29.160909 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:29.160870 2567 patch_prober.go:28] interesting pod/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Jul 08 13:53:29.161331 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:29.160928 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz" podUID="eb4bd630-5b36-435f-8d68-87876a2f5a99" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Jul 08 13:53:29.393804 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:29.393763 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:53:30.160776 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:30.160740 2567 patch_prober.go:28] interesting pod/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Jul 08 13:53:30.160970 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:30.160791 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz" podUID="eb4bd630-5b36-435f-8d68-87876a2f5a99" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Jul 08 13:53:31.160885 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:31.160848 2567 patch_prober.go:28] interesting pod/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Jul 08 13:53:31.161098 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:31.160898 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz" podUID="eb4bd630-5b36-435f-8d68-87876a2f5a99" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Jul 08 13:53:32.164596 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:32.164567 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz" Jul 08 13:53:32.165094 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:32.164829 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz" Jul 08 13:53:32.165674 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:32.165652 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz" Jul 08 13:53:33.457285 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:33.457240 2567 patch_prober.go:28] interesting pod/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 08 13:53:33.457662 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:33.457329 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl" podUID="86d728fd-be08-46cc-8d15-7135f6ae561a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 08 13:53:34.515993 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:34.515963 2567 generic.go:356] "Generic (PLEG): container finished" podID="86d728fd-be08-46cc-8d15-7135f6ae561a" containerID="73490c35996b638e4abbfeee26e04a840b19702e9bfac61cdb8e3e6e094a9426" exitCode=0 Jul 08 13:53:34.516428 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:34.516040 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl" event={"ID":"86d728fd-be08-46cc-8d15-7135f6ae561a","Type":"ContainerDied","Data":"73490c35996b638e4abbfeee26e04a840b19702e9bfac61cdb8e3e6e094a9426"} Jul 08 13:53:34.516484 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:34.516437 2567 scope.go:117] "RemoveContainer" containerID="73490c35996b638e4abbfeee26e04a840b19702e9bfac61cdb8e3e6e094a9426" Jul 08 13:53:34.517509 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:34.517481 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:53:34.517557 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:34.517548 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:53:34.517592 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:34.517584 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:53:35.523818 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:35.523785 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl" event={"ID":"86d728fd-be08-46cc-8d15-7135f6ae561a","Type":"ContainerStarted","Data":"b4e37f975c923fb69f7b7bb068279b1aa432f8d461dd3b4ead4fca7cdbd0da99"} Jul 08 13:53:35.765511 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:35.765473 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl" Jul 08 13:53:35.765733 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:35.765712 2567 patch_prober.go:28] interesting pod/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body= Jul 08 13:53:35.765824 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:35.765766 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl" podUID="86d728fd-be08-46cc-8d15-7135f6ae561a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" Jul 08 13:53:36.765955 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:36.765918 2567 patch_prober.go:28] interesting pod/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body= Jul 08 13:53:36.766451 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:36.765974 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl" podUID="86d728fd-be08-46cc-8d15-7135f6ae561a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" Jul 08 13:53:37.765537 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:37.765491 2567 patch_prober.go:28] interesting pod/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body= Jul 08 13:53:37.765714 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:37.765613 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl" podUID="86d728fd-be08-46cc-8d15-7135f6ae561a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" Jul 08 13:53:38.765706 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:38.765670 2567 patch_prober.go:28] interesting pod/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body= Jul 08 13:53:38.766192 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:38.765737 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl" podUID="86d728fd-be08-46cc-8d15-7135f6ae561a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" Jul 08 13:53:39.769031 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:39.769002 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl" Jul 08 13:53:39.769445 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:39.769389 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl" Jul 08 13:53:39.770196 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:39.770173 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl" Jul 08 13:53:40.093643 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:40.093541 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:53:58.797109 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:53:58.797058 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:54:46.984862 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:54:46.984770 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:55:25.622848 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:25.622814 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz"] Jul 08 13:55:25.623541 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:25.623020 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz" podUID="eb4bd630-5b36-435f-8d68-87876a2f5a99" containerName="istio-proxy" containerID="cri-o://c1589ad7118f1569a3590c4244a3e41d32994c95bb551a7c415f955c5e19c82d" gracePeriod=30 Jul 08 13:55:29.596585 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:29.596548 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:55:30.878977 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:30.878952 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz" Jul 08 13:55:30.908742 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:30.908673 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb4bd630-5b36-435f-8d68-87876a2f5a99-workload-certs\") pod \"eb4bd630-5b36-435f-8d68-87876a2f5a99\" (UID: \"eb4bd630-5b36-435f-8d68-87876a2f5a99\") " Jul 08 13:55:30.908742 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:30.908716 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb4bd630-5b36-435f-8d68-87876a2f5a99-istio-envoy\") pod \"eb4bd630-5b36-435f-8d68-87876a2f5a99\" (UID: \"eb4bd630-5b36-435f-8d68-87876a2f5a99\") " Jul 08 13:55:30.908921 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:30.908755 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r7zcn\" (UniqueName: \"kubernetes.io/projected/eb4bd630-5b36-435f-8d68-87876a2f5a99-kube-api-access-r7zcn\") pod \"eb4bd630-5b36-435f-8d68-87876a2f5a99\" (UID: \"eb4bd630-5b36-435f-8d68-87876a2f5a99\") " Jul 08 13:55:30.908921 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:30.908785 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb4bd630-5b36-435f-8d68-87876a2f5a99-istio-data\") pod \"eb4bd630-5b36-435f-8d68-87876a2f5a99\" (UID: \"eb4bd630-5b36-435f-8d68-87876a2f5a99\") " Jul 08 13:55:30.908921 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:30.908837 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb4bd630-5b36-435f-8d68-87876a2f5a99-istiod-ca-cert\") pod \"eb4bd630-5b36-435f-8d68-87876a2f5a99\" (UID: \"eb4bd630-5b36-435f-8d68-87876a2f5a99\") " Jul 08 13:55:30.908921 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:30.908864 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb4bd630-5b36-435f-8d68-87876a2f5a99-credential-socket\") pod \"eb4bd630-5b36-435f-8d68-87876a2f5a99\" (UID: \"eb4bd630-5b36-435f-8d68-87876a2f5a99\") " Jul 08 13:55:30.908921 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:30.908903 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb4bd630-5b36-435f-8d68-87876a2f5a99-istio-token\") pod \"eb4bd630-5b36-435f-8d68-87876a2f5a99\" (UID: \"eb4bd630-5b36-435f-8d68-87876a2f5a99\") " Jul 08 13:55:30.909215 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:30.908952 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb4bd630-5b36-435f-8d68-87876a2f5a99-istio-podinfo\") pod \"eb4bd630-5b36-435f-8d68-87876a2f5a99\" (UID: \"eb4bd630-5b36-435f-8d68-87876a2f5a99\") " Jul 08 13:55:30.909215 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:30.908976 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb4bd630-5b36-435f-8d68-87876a2f5a99-workload-socket\") pod \"eb4bd630-5b36-435f-8d68-87876a2f5a99\" (UID: \"eb4bd630-5b36-435f-8d68-87876a2f5a99\") " Jul 08 13:55:30.909215 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:30.908998 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb4bd630-5b36-435f-8d68-87876a2f5a99-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "eb4bd630-5b36-435f-8d68-87876a2f5a99" (UID: "eb4bd630-5b36-435f-8d68-87876a2f5a99"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:55:30.909380 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:30.909213 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb4bd630-5b36-435f-8d68-87876a2f5a99-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "eb4bd630-5b36-435f-8d68-87876a2f5a99" (UID: "eb4bd630-5b36-435f-8d68-87876a2f5a99"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:55:30.909380 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:30.909315 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb4bd630-5b36-435f-8d68-87876a2f5a99-workload-certs\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:55:30.909380 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:30.909335 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb4bd630-5b36-435f-8d68-87876a2f5a99-credential-socket\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:55:30.909538 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:30.909403 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb4bd630-5b36-435f-8d68-87876a2f5a99-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "eb4bd630-5b36-435f-8d68-87876a2f5a99" (UID: "eb4bd630-5b36-435f-8d68-87876a2f5a99"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:55:30.909818 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:30.909787 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb4bd630-5b36-435f-8d68-87876a2f5a99-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "eb4bd630-5b36-435f-8d68-87876a2f5a99" (UID: "eb4bd630-5b36-435f-8d68-87876a2f5a99"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 13:55:30.910562 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:30.910525 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb4bd630-5b36-435f-8d68-87876a2f5a99-istio-data" (OuterVolumeSpecName: "istio-data") pod "eb4bd630-5b36-435f-8d68-87876a2f5a99" (UID: "eb4bd630-5b36-435f-8d68-87876a2f5a99"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:55:30.911159 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:30.911131 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eb4bd630-5b36-435f-8d68-87876a2f5a99-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "eb4bd630-5b36-435f-8d68-87876a2f5a99" (UID: "eb4bd630-5b36-435f-8d68-87876a2f5a99"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 13:55:30.911283 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:30.911254 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb4bd630-5b36-435f-8d68-87876a2f5a99-istio-token" (OuterVolumeSpecName: "istio-token") pod "eb4bd630-5b36-435f-8d68-87876a2f5a99" (UID: "eb4bd630-5b36-435f-8d68-87876a2f5a99"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 13:55:30.911432 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:30.911403 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb4bd630-5b36-435f-8d68-87876a2f5a99-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "eb4bd630-5b36-435f-8d68-87876a2f5a99" (UID: "eb4bd630-5b36-435f-8d68-87876a2f5a99"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:55:30.911432 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:30.911420 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb4bd630-5b36-435f-8d68-87876a2f5a99-kube-api-access-r7zcn" (OuterVolumeSpecName: "kube-api-access-r7zcn") pod "eb4bd630-5b36-435f-8d68-87876a2f5a99" (UID: "eb4bd630-5b36-435f-8d68-87876a2f5a99"). InnerVolumeSpecName "kube-api-access-r7zcn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 13:55:31.010613 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:31.010576 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb4bd630-5b36-435f-8d68-87876a2f5a99-istio-token\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:55:31.010613 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:31.010605 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb4bd630-5b36-435f-8d68-87876a2f5a99-istio-podinfo\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:55:31.010613 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:31.010615 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb4bd630-5b36-435f-8d68-87876a2f5a99-workload-socket\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:55:31.010613 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:31.010624 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb4bd630-5b36-435f-8d68-87876a2f5a99-istio-envoy\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:55:31.010883 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:31.010633 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r7zcn\" (UniqueName: \"kubernetes.io/projected/eb4bd630-5b36-435f-8d68-87876a2f5a99-kube-api-access-r7zcn\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:55:31.010883 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:31.010643 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb4bd630-5b36-435f-8d68-87876a2f5a99-istio-data\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:55:31.010883 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:31.010651 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb4bd630-5b36-435f-8d68-87876a2f5a99-istiod-ca-cert\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:55:31.107925 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:31.107889 2567 generic.go:356] "Generic (PLEG): container finished" podID="eb4bd630-5b36-435f-8d68-87876a2f5a99" containerID="c1589ad7118f1569a3590c4244a3e41d32994c95bb551a7c415f955c5e19c82d" exitCode=0 Jul 08 13:55:31.108102 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:31.107938 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz" Jul 08 13:55:31.108102 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:31.107968 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz" event={"ID":"eb4bd630-5b36-435f-8d68-87876a2f5a99","Type":"ContainerDied","Data":"c1589ad7118f1569a3590c4244a3e41d32994c95bb551a7c415f955c5e19c82d"} Jul 08 13:55:31.108102 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:31.108055 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz" event={"ID":"eb4bd630-5b36-435f-8d68-87876a2f5a99","Type":"ContainerDied","Data":"026f025a0595ea385ce3564c109826fbe0b0f72ad19e7106d9f39bd97f23ee62"} Jul 08 13:55:31.108102 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:31.108096 2567 scope.go:117] "RemoveContainer" containerID="c1589ad7118f1569a3590c4244a3e41d32994c95bb551a7c415f955c5e19c82d" Jul 08 13:55:31.117235 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:31.117220 2567 scope.go:117] "RemoveContainer" containerID="c7c4ca29089f88d9a096efe443ffbe546ff14ddfa5843fd205700bbc8bbd4d2e" Jul 08 13:55:31.124948 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:31.124933 2567 scope.go:117] "RemoveContainer" containerID="c1589ad7118f1569a3590c4244a3e41d32994c95bb551a7c415f955c5e19c82d" Jul 08 13:55:31.125244 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:55:31.125219 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1589ad7118f1569a3590c4244a3e41d32994c95bb551a7c415f955c5e19c82d\": container with ID starting with c1589ad7118f1569a3590c4244a3e41d32994c95bb551a7c415f955c5e19c82d not found: ID does not exist" containerID="c1589ad7118f1569a3590c4244a3e41d32994c95bb551a7c415f955c5e19c82d" Jul 08 13:55:31.125300 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:31.125253 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1589ad7118f1569a3590c4244a3e41d32994c95bb551a7c415f955c5e19c82d"} err="failed to get container status \"c1589ad7118f1569a3590c4244a3e41d32994c95bb551a7c415f955c5e19c82d\": rpc error: code = NotFound desc = could not find container \"c1589ad7118f1569a3590c4244a3e41d32994c95bb551a7c415f955c5e19c82d\": container with ID starting with c1589ad7118f1569a3590c4244a3e41d32994c95bb551a7c415f955c5e19c82d not found: ID does not exist" Jul 08 13:55:31.125300 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:31.125274 2567 scope.go:117] "RemoveContainer" containerID="c7c4ca29089f88d9a096efe443ffbe546ff14ddfa5843fd205700bbc8bbd4d2e" Jul 08 13:55:31.125513 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:55:31.125494 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7c4ca29089f88d9a096efe443ffbe546ff14ddfa5843fd205700bbc8bbd4d2e\": container with ID starting with c7c4ca29089f88d9a096efe443ffbe546ff14ddfa5843fd205700bbc8bbd4d2e not found: ID does not exist" containerID="c7c4ca29089f88d9a096efe443ffbe546ff14ddfa5843fd205700bbc8bbd4d2e" Jul 08 13:55:31.125571 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:31.125523 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7c4ca29089f88d9a096efe443ffbe546ff14ddfa5843fd205700bbc8bbd4d2e"} err="failed to get container status \"c7c4ca29089f88d9a096efe443ffbe546ff14ddfa5843fd205700bbc8bbd4d2e\": rpc error: code = NotFound desc = could not find container \"c7c4ca29089f88d9a096efe443ffbe546ff14ddfa5843fd205700bbc8bbd4d2e\": container with ID starting with c7c4ca29089f88d9a096efe443ffbe546ff14ddfa5843fd205700bbc8bbd4d2e not found: ID does not exist" Jul 08 13:55:31.130133 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:31.130109 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz"] Jul 08 13:55:31.134212 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:31.134193 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-ac32a0-openshift-default-65df6cc945-hnbjz"] Jul 08 13:55:31.795814 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:31.795783 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb4bd630-5b36-435f-8d68-87876a2f5a99" path="/var/lib/kubelet/pods/eb4bd630-5b36-435f-8d68-87876a2f5a99/volumes" Jul 08 13:55:32.710038 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:32.710007 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl"] Jul 08 13:55:32.710470 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:32.710219 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl" podUID="86d728fd-be08-46cc-8d15-7135f6ae561a" containerName="istio-proxy" containerID="cri-o://b4e37f975c923fb69f7b7bb068279b1aa432f8d461dd3b4ead4fca7cdbd0da99" gracePeriod=30 Jul 08 13:55:33.604907 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:33.604866 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt"] Jul 08 13:55:33.605237 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:33.605221 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb4bd630-5b36-435f-8d68-87876a2f5a99" containerName="istio-proxy" Jul 08 13:55:33.605237 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:33.605237 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb4bd630-5b36-435f-8d68-87876a2f5a99" containerName="istio-proxy" Jul 08 13:55:33.605363 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:33.605313 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb4bd630-5b36-435f-8d68-87876a2f5a99" containerName="istio-proxy" Jul 08 13:55:33.605363 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:33.605324 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb4bd630-5b36-435f-8d68-87876a2f5a99" containerName="istio-proxy" Jul 08 13:55:33.609963 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:33.609944 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt" Jul 08 13:55:33.612390 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:33.612368 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-8dd341-openshift-default-dockercfg-s7jcv\"" Jul 08 13:55:33.619947 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:33.619921 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt"] Jul 08 13:55:33.734209 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:33.734174 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70a84f6f-0e62-4cb5-883c-b5b0b7adf253-istio-token\") pod \"e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt\" (UID: \"70a84f6f-0e62-4cb5-883c-b5b0b7adf253\") " pod="openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt" Jul 08 13:55:33.734598 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:33.734226 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70a84f6f-0e62-4cb5-883c-b5b0b7adf253-workload-socket\") pod \"e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt\" (UID: \"70a84f6f-0e62-4cb5-883c-b5b0b7adf253\") " pod="openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt" Jul 08 13:55:33.734598 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:33.734246 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70a84f6f-0e62-4cb5-883c-b5b0b7adf253-credential-socket\") pod \"e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt\" (UID: \"70a84f6f-0e62-4cb5-883c-b5b0b7adf253\") " pod="openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt" Jul 08 13:55:33.734598 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:33.734321 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70a84f6f-0e62-4cb5-883c-b5b0b7adf253-istio-envoy\") pod \"e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt\" (UID: \"70a84f6f-0e62-4cb5-883c-b5b0b7adf253\") " pod="openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt" Jul 08 13:55:33.734598 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:33.734345 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70a84f6f-0e62-4cb5-883c-b5b0b7adf253-istio-podinfo\") pod \"e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt\" (UID: \"70a84f6f-0e62-4cb5-883c-b5b0b7adf253\") " pod="openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt" Jul 08 13:55:33.734598 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:33.734365 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70a84f6f-0e62-4cb5-883c-b5b0b7adf253-workload-certs\") pod \"e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt\" (UID: \"70a84f6f-0e62-4cb5-883c-b5b0b7adf253\") " pod="openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt" Jul 08 13:55:33.734598 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:33.734404 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70a84f6f-0e62-4cb5-883c-b5b0b7adf253-istiod-ca-cert\") pod \"e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt\" (UID: \"70a84f6f-0e62-4cb5-883c-b5b0b7adf253\") " pod="openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt" Jul 08 13:55:33.734598 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:33.734425 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hf2bd\" (UniqueName: \"kubernetes.io/projected/70a84f6f-0e62-4cb5-883c-b5b0b7adf253-kube-api-access-hf2bd\") pod \"e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt\" (UID: \"70a84f6f-0e62-4cb5-883c-b5b0b7adf253\") " pod="openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt" Jul 08 13:55:33.734598 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:33.734453 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70a84f6f-0e62-4cb5-883c-b5b0b7adf253-istio-data\") pod \"e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt\" (UID: \"70a84f6f-0e62-4cb5-883c-b5b0b7adf253\") " pod="openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt" Jul 08 13:55:33.834956 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:33.834921 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70a84f6f-0e62-4cb5-883c-b5b0b7adf253-istio-envoy\") pod \"e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt\" (UID: \"70a84f6f-0e62-4cb5-883c-b5b0b7adf253\") " pod="openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt" Jul 08 13:55:33.834956 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:33.834956 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70a84f6f-0e62-4cb5-883c-b5b0b7adf253-istio-podinfo\") pod \"e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt\" (UID: \"70a84f6f-0e62-4cb5-883c-b5b0b7adf253\") " pod="openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt" Jul 08 13:55:33.835236 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:33.834977 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70a84f6f-0e62-4cb5-883c-b5b0b7adf253-workload-certs\") pod \"e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt\" (UID: \"70a84f6f-0e62-4cb5-883c-b5b0b7adf253\") " pod="openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt" Jul 08 13:55:33.835236 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:33.835012 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70a84f6f-0e62-4cb5-883c-b5b0b7adf253-istiod-ca-cert\") pod \"e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt\" (UID: \"70a84f6f-0e62-4cb5-883c-b5b0b7adf253\") " pod="openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt" Jul 08 13:55:33.835236 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:33.835029 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hf2bd\" (UniqueName: \"kubernetes.io/projected/70a84f6f-0e62-4cb5-883c-b5b0b7adf253-kube-api-access-hf2bd\") pod \"e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt\" (UID: \"70a84f6f-0e62-4cb5-883c-b5b0b7adf253\") " pod="openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt" Jul 08 13:55:33.835236 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:33.835050 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70a84f6f-0e62-4cb5-883c-b5b0b7adf253-istio-data\") pod \"e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt\" (UID: \"70a84f6f-0e62-4cb5-883c-b5b0b7adf253\") " pod="openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt" Jul 08 13:55:33.835236 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:33.835119 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70a84f6f-0e62-4cb5-883c-b5b0b7adf253-istio-token\") pod \"e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt\" (UID: \"70a84f6f-0e62-4cb5-883c-b5b0b7adf253\") " pod="openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt" Jul 08 13:55:33.835236 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:33.835155 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70a84f6f-0e62-4cb5-883c-b5b0b7adf253-workload-socket\") pod \"e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt\" (UID: \"70a84f6f-0e62-4cb5-883c-b5b0b7adf253\") " pod="openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt" Jul 08 13:55:33.835236 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:33.835180 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70a84f6f-0e62-4cb5-883c-b5b0b7adf253-credential-socket\") pod \"e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt\" (UID: \"70a84f6f-0e62-4cb5-883c-b5b0b7adf253\") " pod="openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt" Jul 08 13:55:33.835591 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:33.835493 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70a84f6f-0e62-4cb5-883c-b5b0b7adf253-workload-certs\") pod \"e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt\" (UID: \"70a84f6f-0e62-4cb5-883c-b5b0b7adf253\") " pod="openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt" Jul 08 13:55:33.835591 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:33.835535 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70a84f6f-0e62-4cb5-883c-b5b0b7adf253-credential-socket\") pod \"e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt\" (UID: \"70a84f6f-0e62-4cb5-883c-b5b0b7adf253\") " pod="openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt" Jul 08 13:55:33.835707 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:33.835594 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70a84f6f-0e62-4cb5-883c-b5b0b7adf253-workload-socket\") pod \"e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt\" (UID: \"70a84f6f-0e62-4cb5-883c-b5b0b7adf253\") " pod="openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt" Jul 08 13:55:33.835707 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:33.835621 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70a84f6f-0e62-4cb5-883c-b5b0b7adf253-istio-data\") pod \"e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt\" (UID: \"70a84f6f-0e62-4cb5-883c-b5b0b7adf253\") " pod="openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt" Jul 08 13:55:33.835862 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:33.835842 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70a84f6f-0e62-4cb5-883c-b5b0b7adf253-istiod-ca-cert\") pod \"e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt\" (UID: \"70a84f6f-0e62-4cb5-883c-b5b0b7adf253\") " pod="openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt" Jul 08 13:55:33.837276 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:33.837258 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70a84f6f-0e62-4cb5-883c-b5b0b7adf253-istio-envoy\") pod \"e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt\" (UID: \"70a84f6f-0e62-4cb5-883c-b5b0b7adf253\") " pod="openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt" Jul 08 13:55:33.837512 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:33.837495 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70a84f6f-0e62-4cb5-883c-b5b0b7adf253-istio-podinfo\") pod \"e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt\" (UID: \"70a84f6f-0e62-4cb5-883c-b5b0b7adf253\") " pod="openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt" Jul 08 13:55:33.842935 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:33.842916 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70a84f6f-0e62-4cb5-883c-b5b0b7adf253-istio-token\") pod \"e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt\" (UID: \"70a84f6f-0e62-4cb5-883c-b5b0b7adf253\") " pod="openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt" Jul 08 13:55:33.843059 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:33.843044 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hf2bd\" (UniqueName: \"kubernetes.io/projected/70a84f6f-0e62-4cb5-883c-b5b0b7adf253-kube-api-access-hf2bd\") pod \"e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt\" (UID: \"70a84f6f-0e62-4cb5-883c-b5b0b7adf253\") " pod="openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt" Jul 08 13:55:33.921644 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:33.921567 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt" Jul 08 13:55:34.060628 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:34.060602 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt"] Jul 08 13:55:34.061839 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:55:34.061813 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70a84f6f_0e62_4cb5_883c_b5b0b7adf253.slice/crio-e8092b14fcb29b6df47c2ea7e7a75a25b160d79f09f0d34d85e0e8758afd9228 WatchSource:0}: Error finding container e8092b14fcb29b6df47c2ea7e7a75a25b160d79f09f0d34d85e0e8758afd9228: Status 404 returned error can't find the container with id e8092b14fcb29b6df47c2ea7e7a75a25b160d79f09f0d34d85e0e8758afd9228 Jul 08 13:55:34.064011 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:34.063981 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:55:34.064138 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:34.064055 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:55:34.064138 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:34.064116 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:55:34.125644 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:34.125612 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt" event={"ID":"70a84f6f-0e62-4cb5-883c-b5b0b7adf253","Type":"ContainerStarted","Data":"e8092b14fcb29b6df47c2ea7e7a75a25b160d79f09f0d34d85e0e8758afd9228"} Jul 08 13:55:35.132198 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:35.132156 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt" event={"ID":"70a84f6f-0e62-4cb5-883c-b5b0b7adf253","Type":"ContainerStarted","Data":"6e973a41d58cf5d572dbc6de7021d10c94a4af5399b42cf86dd356f5ab0fbc3b"} Jul 08 13:55:35.154452 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:35.154401 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt" podStartSLOduration=2.15438525 podStartE2EDuration="2.15438525s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:55:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:55:35.151762132 +0000 UTC m=+1943.948702789" watchObservedRunningTime="2026-07-08 13:55:35.15438525 +0000 UTC m=+1943.951325906" Jul 08 13:55:35.921918 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:35.921879 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt" Jul 08 13:55:35.926960 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:35.926934 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt" Jul 08 13:55:36.136793 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:36.136765 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt" Jul 08 13:55:36.137772 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:36.137752 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt" Jul 08 13:55:37.962260 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:37.962235 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl" Jul 08 13:55:38.075738 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:38.075643 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86d728fd-be08-46cc-8d15-7135f6ae561a-workload-certs\") pod \"86d728fd-be08-46cc-8d15-7135f6ae561a\" (UID: \"86d728fd-be08-46cc-8d15-7135f6ae561a\") " Jul 08 13:55:38.075738 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:38.075700 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86d728fd-be08-46cc-8d15-7135f6ae561a-workload-socket\") pod \"86d728fd-be08-46cc-8d15-7135f6ae561a\" (UID: \"86d728fd-be08-46cc-8d15-7135f6ae561a\") " Jul 08 13:55:38.075738 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:38.075734 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86d728fd-be08-46cc-8d15-7135f6ae561a-istio-envoy\") pod \"86d728fd-be08-46cc-8d15-7135f6ae561a\" (UID: \"86d728fd-be08-46cc-8d15-7135f6ae561a\") " Jul 08 13:55:38.076023 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:38.075763 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tc6hz\" (UniqueName: \"kubernetes.io/projected/86d728fd-be08-46cc-8d15-7135f6ae561a-kube-api-access-tc6hz\") pod \"86d728fd-be08-46cc-8d15-7135f6ae561a\" (UID: \"86d728fd-be08-46cc-8d15-7135f6ae561a\") " Jul 08 13:55:38.076023 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:38.075792 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86d728fd-be08-46cc-8d15-7135f6ae561a-istiod-ca-cert\") pod \"86d728fd-be08-46cc-8d15-7135f6ae561a\" (UID: \"86d728fd-be08-46cc-8d15-7135f6ae561a\") " Jul 08 13:55:38.076023 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:38.075873 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86d728fd-be08-46cc-8d15-7135f6ae561a-istio-podinfo\") pod \"86d728fd-be08-46cc-8d15-7135f6ae561a\" (UID: \"86d728fd-be08-46cc-8d15-7135f6ae561a\") " Jul 08 13:55:38.076023 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:38.075951 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86d728fd-be08-46cc-8d15-7135f6ae561a-istio-token\") pod \"86d728fd-be08-46cc-8d15-7135f6ae561a\" (UID: \"86d728fd-be08-46cc-8d15-7135f6ae561a\") " Jul 08 13:55:38.076023 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:38.075985 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86d728fd-be08-46cc-8d15-7135f6ae561a-credential-socket\") pod \"86d728fd-be08-46cc-8d15-7135f6ae561a\" (UID: \"86d728fd-be08-46cc-8d15-7135f6ae561a\") " Jul 08 13:55:38.076023 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:38.075987 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86d728fd-be08-46cc-8d15-7135f6ae561a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "86d728fd-be08-46cc-8d15-7135f6ae561a" (UID: "86d728fd-be08-46cc-8d15-7135f6ae561a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:55:38.076023 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:38.076005 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86d728fd-be08-46cc-8d15-7135f6ae561a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "86d728fd-be08-46cc-8d15-7135f6ae561a" (UID: "86d728fd-be08-46cc-8d15-7135f6ae561a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:55:38.076023 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:38.076010 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86d728fd-be08-46cc-8d15-7135f6ae561a-istio-data\") pod \"86d728fd-be08-46cc-8d15-7135f6ae561a\" (UID: \"86d728fd-be08-46cc-8d15-7135f6ae561a\") " Jul 08 13:55:38.076746 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:38.076311 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86d728fd-be08-46cc-8d15-7135f6ae561a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "86d728fd-be08-46cc-8d15-7135f6ae561a" (UID: "86d728fd-be08-46cc-8d15-7135f6ae561a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:55:38.076746 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:38.076333 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86d728fd-be08-46cc-8d15-7135f6ae561a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "86d728fd-be08-46cc-8d15-7135f6ae561a" (UID: "86d728fd-be08-46cc-8d15-7135f6ae561a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 13:55:38.076746 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:38.076412 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86d728fd-be08-46cc-8d15-7135f6ae561a-workload-certs\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:55:38.076746 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:38.076431 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86d728fd-be08-46cc-8d15-7135f6ae561a-workload-socket\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:55:38.077014 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:38.076993 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86d728fd-be08-46cc-8d15-7135f6ae561a-istio-data" (OuterVolumeSpecName: "istio-data") pod "86d728fd-be08-46cc-8d15-7135f6ae561a" (UID: "86d728fd-be08-46cc-8d15-7135f6ae561a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:55:38.078054 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:38.078027 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86d728fd-be08-46cc-8d15-7135f6ae561a-istio-token" (OuterVolumeSpecName: "istio-token") pod "86d728fd-be08-46cc-8d15-7135f6ae561a" (UID: "86d728fd-be08-46cc-8d15-7135f6ae561a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 13:55:38.078198 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:38.078135 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/86d728fd-be08-46cc-8d15-7135f6ae561a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "86d728fd-be08-46cc-8d15-7135f6ae561a" (UID: "86d728fd-be08-46cc-8d15-7135f6ae561a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 13:55:38.078269 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:38.078233 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86d728fd-be08-46cc-8d15-7135f6ae561a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "86d728fd-be08-46cc-8d15-7135f6ae561a" (UID: "86d728fd-be08-46cc-8d15-7135f6ae561a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:55:38.078381 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:38.078360 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86d728fd-be08-46cc-8d15-7135f6ae561a-kube-api-access-tc6hz" (OuterVolumeSpecName: "kube-api-access-tc6hz") pod "86d728fd-be08-46cc-8d15-7135f6ae561a" (UID: "86d728fd-be08-46cc-8d15-7135f6ae561a"). InnerVolumeSpecName "kube-api-access-tc6hz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 13:55:38.154956 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:38.154914 2567 generic.go:356] "Generic (PLEG): container finished" podID="86d728fd-be08-46cc-8d15-7135f6ae561a" containerID="b4e37f975c923fb69f7b7bb068279b1aa432f8d461dd3b4ead4fca7cdbd0da99" exitCode=0 Jul 08 13:55:38.155159 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:38.154988 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl" Jul 08 13:55:38.155159 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:38.155000 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl" event={"ID":"86d728fd-be08-46cc-8d15-7135f6ae561a","Type":"ContainerDied","Data":"b4e37f975c923fb69f7b7bb068279b1aa432f8d461dd3b4ead4fca7cdbd0da99"} Jul 08 13:55:38.155159 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:38.155098 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl" event={"ID":"86d728fd-be08-46cc-8d15-7135f6ae561a","Type":"ContainerDied","Data":"cd0d64eb8e789973bb450735be5e3bb74de14eaa85443f53b09dc6e3b22570cd"} Jul 08 13:55:38.155159 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:38.155124 2567 scope.go:117] "RemoveContainer" containerID="b4e37f975c923fb69f7b7bb068279b1aa432f8d461dd3b4ead4fca7cdbd0da99" Jul 08 13:55:38.164384 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:38.164367 2567 scope.go:117] "RemoveContainer" containerID="73490c35996b638e4abbfeee26e04a840b19702e9bfac61cdb8e3e6e094a9426" Jul 08 13:55:38.172737 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:38.172721 2567 scope.go:117] "RemoveContainer" containerID="b4e37f975c923fb69f7b7bb068279b1aa432f8d461dd3b4ead4fca7cdbd0da99" Jul 08 13:55:38.172994 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:55:38.172977 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4e37f975c923fb69f7b7bb068279b1aa432f8d461dd3b4ead4fca7cdbd0da99\": container with ID starting with b4e37f975c923fb69f7b7bb068279b1aa432f8d461dd3b4ead4fca7cdbd0da99 not found: ID does not exist" containerID="b4e37f975c923fb69f7b7bb068279b1aa432f8d461dd3b4ead4fca7cdbd0da99" Jul 08 13:55:38.173044 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:38.173003 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4e37f975c923fb69f7b7bb068279b1aa432f8d461dd3b4ead4fca7cdbd0da99"} err="failed to get container status \"b4e37f975c923fb69f7b7bb068279b1aa432f8d461dd3b4ead4fca7cdbd0da99\": rpc error: code = NotFound desc = could not find container \"b4e37f975c923fb69f7b7bb068279b1aa432f8d461dd3b4ead4fca7cdbd0da99\": container with ID starting with b4e37f975c923fb69f7b7bb068279b1aa432f8d461dd3b4ead4fca7cdbd0da99 not found: ID does not exist" Jul 08 13:55:38.173044 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:38.173020 2567 scope.go:117] "RemoveContainer" containerID="73490c35996b638e4abbfeee26e04a840b19702e9bfac61cdb8e3e6e094a9426" Jul 08 13:55:38.173273 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:55:38.173258 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73490c35996b638e4abbfeee26e04a840b19702e9bfac61cdb8e3e6e094a9426\": container with ID starting with 73490c35996b638e4abbfeee26e04a840b19702e9bfac61cdb8e3e6e094a9426 not found: ID does not exist" containerID="73490c35996b638e4abbfeee26e04a840b19702e9bfac61cdb8e3e6e094a9426" Jul 08 13:55:38.173321 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:38.173276 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73490c35996b638e4abbfeee26e04a840b19702e9bfac61cdb8e3e6e094a9426"} err="failed to get container status \"73490c35996b638e4abbfeee26e04a840b19702e9bfac61cdb8e3e6e094a9426\": rpc error: code = NotFound desc = could not find container \"73490c35996b638e4abbfeee26e04a840b19702e9bfac61cdb8e3e6e094a9426\": container with ID starting with 73490c35996b638e4abbfeee26e04a840b19702e9bfac61cdb8e3e6e094a9426 not found: ID does not exist" Jul 08 13:55:38.177714 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:38.177694 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86d728fd-be08-46cc-8d15-7135f6ae561a-istio-token\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:55:38.177810 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:38.177716 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86d728fd-be08-46cc-8d15-7135f6ae561a-credential-socket\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:55:38.177810 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:38.177730 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86d728fd-be08-46cc-8d15-7135f6ae561a-istio-data\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:55:38.177810 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:38.177742 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86d728fd-be08-46cc-8d15-7135f6ae561a-istio-envoy\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:55:38.177810 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:38.177754 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tc6hz\" (UniqueName: \"kubernetes.io/projected/86d728fd-be08-46cc-8d15-7135f6ae561a-kube-api-access-tc6hz\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:55:38.177810 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:38.177767 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86d728fd-be08-46cc-8d15-7135f6ae561a-istiod-ca-cert\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:55:38.177810 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:38.177779 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86d728fd-be08-46cc-8d15-7135f6ae561a-istio-podinfo\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:55:38.181500 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:38.181480 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl"] Jul 08 13:55:38.187118 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:38.187093 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-8f6d76-openshift-default-75d4dff4df-nh9tl"] Jul 08 13:55:39.795403 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:39.795372 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86d728fd-be08-46cc-8d15-7135f6ae561a" path="/var/lib/kubelet/pods/86d728fd-be08-46cc-8d15-7135f6ae561a/volumes" Jul 08 13:55:42.287812 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:42.287783 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq"] Jul 08 13:55:42.288214 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:42.288105 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb4bd630-5b36-435f-8d68-87876a2f5a99" containerName="istio-proxy" Jul 08 13:55:42.288214 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:42.288117 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb4bd630-5b36-435f-8d68-87876a2f5a99" containerName="istio-proxy" Jul 08 13:55:42.288214 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:42.288137 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86d728fd-be08-46cc-8d15-7135f6ae561a" containerName="istio-proxy" Jul 08 13:55:42.288214 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:42.288142 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="86d728fd-be08-46cc-8d15-7135f6ae561a" containerName="istio-proxy" Jul 08 13:55:42.288214 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:42.288151 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86d728fd-be08-46cc-8d15-7135f6ae561a" containerName="istio-proxy" Jul 08 13:55:42.288214 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:42.288157 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="86d728fd-be08-46cc-8d15-7135f6ae561a" containerName="istio-proxy" Jul 08 13:55:42.288214 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:42.288207 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="86d728fd-be08-46cc-8d15-7135f6ae561a" containerName="istio-proxy" Jul 08 13:55:42.288214 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:42.288214 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="86d728fd-be08-46cc-8d15-7135f6ae561a" containerName="istio-proxy" Jul 08 13:55:42.292723 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:42.292704 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq" Jul 08 13:55:42.295247 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:42.295225 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-d1efd5-openshift-default-dockercfg-lb27h\"" Jul 08 13:55:42.302878 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:42.302851 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq"] Jul 08 13:55:42.413852 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:42.413809 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c57186e8-fbbb-4bea-b681-ec3a6712972d-istio-envoy\") pod \"e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq\" (UID: \"c57186e8-fbbb-4bea-b681-ec3a6712972d\") " pod="openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq" Jul 08 13:55:42.414035 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:42.413899 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c57186e8-fbbb-4bea-b681-ec3a6712972d-istiod-ca-cert\") pod \"e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq\" (UID: \"c57186e8-fbbb-4bea-b681-ec3a6712972d\") " pod="openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq" Jul 08 13:55:42.414035 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:42.413934 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c57186e8-fbbb-4bea-b681-ec3a6712972d-istio-podinfo\") pod \"e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq\" (UID: \"c57186e8-fbbb-4bea-b681-ec3a6712972d\") " pod="openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq" Jul 08 13:55:42.414035 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:42.413956 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c57186e8-fbbb-4bea-b681-ec3a6712972d-istio-token\") pod \"e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq\" (UID: \"c57186e8-fbbb-4bea-b681-ec3a6712972d\") " pod="openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq" Jul 08 13:55:42.414035 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:42.413986 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84w24\" (UniqueName: \"kubernetes.io/projected/c57186e8-fbbb-4bea-b681-ec3a6712972d-kube-api-access-84w24\") pod \"e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq\" (UID: \"c57186e8-fbbb-4bea-b681-ec3a6712972d\") " pod="openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq" Jul 08 13:55:42.414035 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:42.414007 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c57186e8-fbbb-4bea-b681-ec3a6712972d-istio-data\") pod \"e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq\" (UID: \"c57186e8-fbbb-4bea-b681-ec3a6712972d\") " pod="openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq" Jul 08 13:55:42.414239 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:42.414102 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c57186e8-fbbb-4bea-b681-ec3a6712972d-workload-certs\") pod \"e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq\" (UID: \"c57186e8-fbbb-4bea-b681-ec3a6712972d\") " pod="openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq" Jul 08 13:55:42.414239 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:42.414145 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c57186e8-fbbb-4bea-b681-ec3a6712972d-workload-socket\") pod \"e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq\" (UID: \"c57186e8-fbbb-4bea-b681-ec3a6712972d\") " pod="openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq" Jul 08 13:55:42.414239 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:42.414186 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c57186e8-fbbb-4bea-b681-ec3a6712972d-credential-socket\") pod \"e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq\" (UID: \"c57186e8-fbbb-4bea-b681-ec3a6712972d\") " pod="openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq" Jul 08 13:55:42.515472 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:42.515431 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c57186e8-fbbb-4bea-b681-ec3a6712972d-workload-certs\") pod \"e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq\" (UID: \"c57186e8-fbbb-4bea-b681-ec3a6712972d\") " pod="openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq" Jul 08 13:55:42.515675 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:42.515491 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c57186e8-fbbb-4bea-b681-ec3a6712972d-workload-socket\") pod \"e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq\" (UID: \"c57186e8-fbbb-4bea-b681-ec3a6712972d\") " pod="openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq" Jul 08 13:55:42.515675 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:42.515533 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c57186e8-fbbb-4bea-b681-ec3a6712972d-credential-socket\") pod \"e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq\" (UID: \"c57186e8-fbbb-4bea-b681-ec3a6712972d\") " pod="openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq" Jul 08 13:55:42.515675 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:42.515565 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c57186e8-fbbb-4bea-b681-ec3a6712972d-istio-envoy\") pod \"e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq\" (UID: \"c57186e8-fbbb-4bea-b681-ec3a6712972d\") " pod="openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq" Jul 08 13:55:42.515675 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:42.515603 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c57186e8-fbbb-4bea-b681-ec3a6712972d-istiod-ca-cert\") pod \"e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq\" (UID: \"c57186e8-fbbb-4bea-b681-ec3a6712972d\") " pod="openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq" Jul 08 13:55:42.515675 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:42.515630 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c57186e8-fbbb-4bea-b681-ec3a6712972d-istio-podinfo\") pod \"e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq\" (UID: \"c57186e8-fbbb-4bea-b681-ec3a6712972d\") " pod="openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq" Jul 08 13:55:42.515675 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:42.515654 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c57186e8-fbbb-4bea-b681-ec3a6712972d-istio-token\") pod \"e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq\" (UID: \"c57186e8-fbbb-4bea-b681-ec3a6712972d\") " pod="openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq" Jul 08 13:55:42.515995 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:42.515687 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84w24\" (UniqueName: \"kubernetes.io/projected/c57186e8-fbbb-4bea-b681-ec3a6712972d-kube-api-access-84w24\") pod \"e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq\" (UID: \"c57186e8-fbbb-4bea-b681-ec3a6712972d\") " pod="openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq" Jul 08 13:55:42.515995 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:42.515713 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c57186e8-fbbb-4bea-b681-ec3a6712972d-istio-data\") pod \"e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq\" (UID: \"c57186e8-fbbb-4bea-b681-ec3a6712972d\") " pod="openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq" Jul 08 13:55:42.515995 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:42.515872 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c57186e8-fbbb-4bea-b681-ec3a6712972d-workload-certs\") pod \"e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq\" (UID: \"c57186e8-fbbb-4bea-b681-ec3a6712972d\") " pod="openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq" Jul 08 13:55:42.515995 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:42.515916 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c57186e8-fbbb-4bea-b681-ec3a6712972d-workload-socket\") pod \"e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq\" (UID: \"c57186e8-fbbb-4bea-b681-ec3a6712972d\") " pod="openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq" Jul 08 13:55:42.516245 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:42.516135 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c57186e8-fbbb-4bea-b681-ec3a6712972d-istio-data\") pod \"e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq\" (UID: \"c57186e8-fbbb-4bea-b681-ec3a6712972d\") " pod="openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq" Jul 08 13:55:42.516245 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:42.516225 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c57186e8-fbbb-4bea-b681-ec3a6712972d-credential-socket\") pod \"e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq\" (UID: \"c57186e8-fbbb-4bea-b681-ec3a6712972d\") " pod="openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq" Jul 08 13:55:42.516503 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:42.516482 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c57186e8-fbbb-4bea-b681-ec3a6712972d-istiod-ca-cert\") pod \"e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq\" (UID: \"c57186e8-fbbb-4bea-b681-ec3a6712972d\") " pod="openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq" Jul 08 13:55:42.518527 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:42.518502 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c57186e8-fbbb-4bea-b681-ec3a6712972d-istio-envoy\") pod \"e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq\" (UID: \"c57186e8-fbbb-4bea-b681-ec3a6712972d\") " pod="openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq" Jul 08 13:55:42.518668 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:42.518649 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c57186e8-fbbb-4bea-b681-ec3a6712972d-istio-podinfo\") pod \"e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq\" (UID: \"c57186e8-fbbb-4bea-b681-ec3a6712972d\") " pod="openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq" Jul 08 13:55:42.525022 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:42.524988 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c57186e8-fbbb-4bea-b681-ec3a6712972d-istio-token\") pod \"e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq\" (UID: \"c57186e8-fbbb-4bea-b681-ec3a6712972d\") " pod="openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq" Jul 08 13:55:42.531265 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:42.531231 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-84w24\" (UniqueName: \"kubernetes.io/projected/c57186e8-fbbb-4bea-b681-ec3a6712972d-kube-api-access-84w24\") pod \"e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq\" (UID: \"c57186e8-fbbb-4bea-b681-ec3a6712972d\") " pod="openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq" Jul 08 13:55:42.603898 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:42.603797 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq" Jul 08 13:55:42.743186 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:42.743159 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq"] Jul 08 13:55:42.744627 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:55:42.744596 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc57186e8_fbbb_4bea_b681_ec3a6712972d.slice/crio-c37e0435318398a7eff1f2ffe8b2baeb8e7c319f76ee783aea0ff289ac5774cd WatchSource:0}: Error finding container c37e0435318398a7eff1f2ffe8b2baeb8e7c319f76ee783aea0ff289ac5774cd: Status 404 returned error can't find the container with id c37e0435318398a7eff1f2ffe8b2baeb8e7c319f76ee783aea0ff289ac5774cd Jul 08 13:55:42.746645 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:42.746612 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:55:42.746739 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:42.746675 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:55:42.746739 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:42.746703 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:55:43.182550 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:43.182507 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq" event={"ID":"c57186e8-fbbb-4bea-b681-ec3a6712972d","Type":"ContainerStarted","Data":"8d2451bc5580a79079dcee9e0ac7c38525cb709bccc1b347c267fe763c68a51a"} Jul 08 13:55:43.182734 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:43.182575 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq" event={"ID":"c57186e8-fbbb-4bea-b681-ec3a6712972d","Type":"ContainerStarted","Data":"c37e0435318398a7eff1f2ffe8b2baeb8e7c319f76ee783aea0ff289ac5774cd"} Jul 08 13:55:43.203416 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:43.203362 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq" podStartSLOduration=1.203344312 podStartE2EDuration="1.203344312s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:55:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:55:43.202001852 +0000 UTC m=+1951.998942511" watchObservedRunningTime="2026-07-08 13:55:43.203344312 +0000 UTC m=+1952.000285069" Jul 08 13:55:43.403380 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:43.403350 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:55:43.603997 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:43.603962 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq" Jul 08 13:55:43.609238 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:43.609210 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq" Jul 08 13:55:44.187889 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:44.187861 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq" Jul 08 13:55:44.188897 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:44.188876 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq" Jul 08 13:55:51.652505 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:51.652462 2567 patch_prober.go:28] interesting pod/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": read tcp 10.133.0.2:60308->10.133.0.39:15021: read: connection reset by peer" start-of-body= Jul 08 13:55:51.653024 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:51.652529 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt" podUID="70a84f6f-0e62-4cb5-883c-b5b0b7adf253" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": read tcp 10.133.0.2:60308->10.133.0.39:15021: read: connection reset by peer" Jul 08 13:55:52.236503 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:52.236466 2567 generic.go:356] "Generic (PLEG): container finished" podID="70a84f6f-0e62-4cb5-883c-b5b0b7adf253" containerID="6e973a41d58cf5d572dbc6de7021d10c94a4af5399b42cf86dd356f5ab0fbc3b" exitCode=0 Jul 08 13:55:52.236713 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:52.236555 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt" event={"ID":"70a84f6f-0e62-4cb5-883c-b5b0b7adf253","Type":"ContainerDied","Data":"6e973a41d58cf5d572dbc6de7021d10c94a4af5399b42cf86dd356f5ab0fbc3b"} Jul 08 13:55:52.237009 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:52.236990 2567 scope.go:117] "RemoveContainer" containerID="6e973a41d58cf5d572dbc6de7021d10c94a4af5399b42cf86dd356f5ab0fbc3b" Jul 08 13:55:52.238185 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:52.238150 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:55:52.238264 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:52.238235 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:55:52.238302 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:52.238279 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:55:53.243783 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:53.243751 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt" event={"ID":"70a84f6f-0e62-4cb5-883c-b5b0b7adf253","Type":"ContainerStarted","Data":"d4af842d6c2d50ccfbc3d7beb6ccc9e35f0605a02d899709530de5b969f0aee5"} Jul 08 13:55:53.922895 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:53.922854 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt" Jul 08 13:55:53.923380 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:53.923254 2567 patch_prober.go:28] interesting pod/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Jul 08 13:55:53.923592 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:53.923565 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt" podUID="70a84f6f-0e62-4cb5-883c-b5b0b7adf253" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Jul 08 13:55:54.922741 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:54.922701 2567 patch_prober.go:28] interesting pod/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Jul 08 13:55:54.923231 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:54.922805 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt" podUID="70a84f6f-0e62-4cb5-883c-b5b0b7adf253" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Jul 08 13:55:55.922771 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:55.922728 2567 patch_prober.go:28] interesting pod/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Jul 08 13:55:55.923212 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:55.922803 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt" podUID="70a84f6f-0e62-4cb5-883c-b5b0b7adf253" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Jul 08 13:55:56.922241 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:56.922193 2567 patch_prober.go:28] interesting pod/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Jul 08 13:55:56.922507 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:56.922267 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt" podUID="70a84f6f-0e62-4cb5-883c-b5b0b7adf253" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Jul 08 13:55:57.096238 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:57.096181 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:55:57.927750 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:57.926758 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt" Jul 08 13:55:57.927750 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:57.927641 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt" Jul 08 13:55:57.928743 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:57.928714 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt" Jul 08 13:55:58.271506 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:58.271478 2567 generic.go:356] "Generic (PLEG): container finished" podID="c57186e8-fbbb-4bea-b681-ec3a6712972d" containerID="8d2451bc5580a79079dcee9e0ac7c38525cb709bccc1b347c267fe763c68a51a" exitCode=0 Jul 08 13:55:58.271884 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:58.271554 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq" event={"ID":"c57186e8-fbbb-4bea-b681-ec3a6712972d","Type":"ContainerDied","Data":"8d2451bc5580a79079dcee9e0ac7c38525cb709bccc1b347c267fe763c68a51a"} Jul 08 13:55:58.272018 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:58.272004 2567 scope.go:117] "RemoveContainer" containerID="8d2451bc5580a79079dcee9e0ac7c38525cb709bccc1b347c267fe763c68a51a" Jul 08 13:55:58.272939 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:58.272916 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:55:58.273006 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:58.272994 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:55:58.273046 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:58.273027 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:55:59.187960 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:59.187927 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq" Jul 08 13:55:59.277941 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:59.277907 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq" event={"ID":"c57186e8-fbbb-4bea-b681-ec3a6712972d","Type":"ContainerStarted","Data":"3d27f273010ca2fd01159a6abec0ca1d7514a8a6a355f71b78a2b05a46a99e69"} Jul 08 13:55:59.604822 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:59.604780 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq" Jul 08 13:55:59.606052 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:59.606016 2567 patch_prober.go:28] interesting pod/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Jul 08 13:55:59.606196 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:55:59.606098 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq" podUID="c57186e8-fbbb-4bea-b681-ec3a6712972d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Jul 08 13:56:00.604830 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:00.604778 2567 patch_prober.go:28] interesting pod/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Jul 08 13:56:00.605381 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:00.604850 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq" podUID="c57186e8-fbbb-4bea-b681-ec3a6712972d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Jul 08 13:56:01.604518 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:01.604479 2567 patch_prober.go:28] interesting pod/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Jul 08 13:56:01.604811 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:01.604781 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq" podUID="c57186e8-fbbb-4bea-b681-ec3a6712972d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Jul 08 13:56:02.604567 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:02.604522 2567 patch_prober.go:28] interesting pod/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Jul 08 13:56:02.605229 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:02.605181 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq" podUID="c57186e8-fbbb-4bea-b681-ec3a6712972d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Jul 08 13:56:03.604807 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:03.604700 2567 patch_prober.go:28] interesting pod/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Jul 08 13:56:03.604807 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:03.604770 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq" podUID="c57186e8-fbbb-4bea-b681-ec3a6712972d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Jul 08 13:56:04.236956 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:04.236921 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt"] Jul 08 13:56:04.237184 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:04.237159 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt" podUID="70a84f6f-0e62-4cb5-883c-b5b0b7adf253" containerName="istio-proxy" containerID="cri-o://d4af842d6c2d50ccfbc3d7beb6ccc9e35f0605a02d899709530de5b969f0aee5" gracePeriod=30 Jul 08 13:56:04.608432 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:04.608358 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq" Jul 08 13:56:04.608802 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:04.608728 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq" Jul 08 13:56:04.609503 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:04.609489 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq" Jul 08 13:56:08.110177 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:08.110140 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:56:09.512756 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:09.512731 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt" Jul 08 13:56:09.581210 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:09.581179 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70a84f6f-0e62-4cb5-883c-b5b0b7adf253-workload-socket\") pod \"70a84f6f-0e62-4cb5-883c-b5b0b7adf253\" (UID: \"70a84f6f-0e62-4cb5-883c-b5b0b7adf253\") " Jul 08 13:56:09.581210 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:09.581213 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70a84f6f-0e62-4cb5-883c-b5b0b7adf253-istio-envoy\") pod \"70a84f6f-0e62-4cb5-883c-b5b0b7adf253\" (UID: \"70a84f6f-0e62-4cb5-883c-b5b0b7adf253\") " Jul 08 13:56:09.581424 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:09.581238 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70a84f6f-0e62-4cb5-883c-b5b0b7adf253-istio-token\") pod \"70a84f6f-0e62-4cb5-883c-b5b0b7adf253\" (UID: \"70a84f6f-0e62-4cb5-883c-b5b0b7adf253\") " Jul 08 13:56:09.581424 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:09.581263 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hf2bd\" (UniqueName: \"kubernetes.io/projected/70a84f6f-0e62-4cb5-883c-b5b0b7adf253-kube-api-access-hf2bd\") pod \"70a84f6f-0e62-4cb5-883c-b5b0b7adf253\" (UID: \"70a84f6f-0e62-4cb5-883c-b5b0b7adf253\") " Jul 08 13:56:09.581424 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:09.581291 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70a84f6f-0e62-4cb5-883c-b5b0b7adf253-istio-podinfo\") pod \"70a84f6f-0e62-4cb5-883c-b5b0b7adf253\" (UID: \"70a84f6f-0e62-4cb5-883c-b5b0b7adf253\") " Jul 08 13:56:09.581424 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:09.581309 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70a84f6f-0e62-4cb5-883c-b5b0b7adf253-istio-data\") pod \"70a84f6f-0e62-4cb5-883c-b5b0b7adf253\" (UID: \"70a84f6f-0e62-4cb5-883c-b5b0b7adf253\") " Jul 08 13:56:09.581424 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:09.581375 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70a84f6f-0e62-4cb5-883c-b5b0b7adf253-istiod-ca-cert\") pod \"70a84f6f-0e62-4cb5-883c-b5b0b7adf253\" (UID: \"70a84f6f-0e62-4cb5-883c-b5b0b7adf253\") " Jul 08 13:56:09.581424 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:09.581414 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70a84f6f-0e62-4cb5-883c-b5b0b7adf253-workload-certs\") pod \"70a84f6f-0e62-4cb5-883c-b5b0b7adf253\" (UID: \"70a84f6f-0e62-4cb5-883c-b5b0b7adf253\") " Jul 08 13:56:09.581753 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:09.581462 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70a84f6f-0e62-4cb5-883c-b5b0b7adf253-credential-socket\") pod \"70a84f6f-0e62-4cb5-883c-b5b0b7adf253\" (UID: \"70a84f6f-0e62-4cb5-883c-b5b0b7adf253\") " Jul 08 13:56:09.581753 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:09.581481 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70a84f6f-0e62-4cb5-883c-b5b0b7adf253-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "70a84f6f-0e62-4cb5-883c-b5b0b7adf253" (UID: "70a84f6f-0e62-4cb5-883c-b5b0b7adf253"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:56:09.581753 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:09.581704 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70a84f6f-0e62-4cb5-883c-b5b0b7adf253-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "70a84f6f-0e62-4cb5-883c-b5b0b7adf253" (UID: "70a84f6f-0e62-4cb5-883c-b5b0b7adf253"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:56:09.581753 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:09.581718 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70a84f6f-0e62-4cb5-883c-b5b0b7adf253-workload-socket\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:56:09.581972 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:09.581904 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70a84f6f-0e62-4cb5-883c-b5b0b7adf253-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "70a84f6f-0e62-4cb5-883c-b5b0b7adf253" (UID: "70a84f6f-0e62-4cb5-883c-b5b0b7adf253"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:56:09.582285 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:09.582255 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70a84f6f-0e62-4cb5-883c-b5b0b7adf253-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "70a84f6f-0e62-4cb5-883c-b5b0b7adf253" (UID: "70a84f6f-0e62-4cb5-883c-b5b0b7adf253"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 13:56:09.582426 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:09.582395 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70a84f6f-0e62-4cb5-883c-b5b0b7adf253-istio-data" (OuterVolumeSpecName: "istio-data") pod "70a84f6f-0e62-4cb5-883c-b5b0b7adf253" (UID: "70a84f6f-0e62-4cb5-883c-b5b0b7adf253"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:56:09.583636 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:09.583601 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70a84f6f-0e62-4cb5-883c-b5b0b7adf253-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "70a84f6f-0e62-4cb5-883c-b5b0b7adf253" (UID: "70a84f6f-0e62-4cb5-883c-b5b0b7adf253"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:56:09.583636 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:09.583605 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70a84f6f-0e62-4cb5-883c-b5b0b7adf253-istio-token" (OuterVolumeSpecName: "istio-token") pod "70a84f6f-0e62-4cb5-883c-b5b0b7adf253" (UID: "70a84f6f-0e62-4cb5-883c-b5b0b7adf253"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 13:56:09.583780 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:09.583611 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/70a84f6f-0e62-4cb5-883c-b5b0b7adf253-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "70a84f6f-0e62-4cb5-883c-b5b0b7adf253" (UID: "70a84f6f-0e62-4cb5-883c-b5b0b7adf253"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 13:56:09.583780 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:09.583632 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70a84f6f-0e62-4cb5-883c-b5b0b7adf253-kube-api-access-hf2bd" (OuterVolumeSpecName: "kube-api-access-hf2bd") pod "70a84f6f-0e62-4cb5-883c-b5b0b7adf253" (UID: "70a84f6f-0e62-4cb5-883c-b5b0b7adf253"). InnerVolumeSpecName "kube-api-access-hf2bd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 13:56:09.682488 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:09.682431 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hf2bd\" (UniqueName: \"kubernetes.io/projected/70a84f6f-0e62-4cb5-883c-b5b0b7adf253-kube-api-access-hf2bd\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:56:09.682488 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:09.682483 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70a84f6f-0e62-4cb5-883c-b5b0b7adf253-istio-podinfo\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:56:09.682488 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:09.682496 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70a84f6f-0e62-4cb5-883c-b5b0b7adf253-istio-data\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:56:09.682738 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:09.682507 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70a84f6f-0e62-4cb5-883c-b5b0b7adf253-istiod-ca-cert\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:56:09.682738 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:09.682518 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70a84f6f-0e62-4cb5-883c-b5b0b7adf253-workload-certs\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:56:09.682738 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:09.682529 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70a84f6f-0e62-4cb5-883c-b5b0b7adf253-credential-socket\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:56:09.682738 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:09.682540 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70a84f6f-0e62-4cb5-883c-b5b0b7adf253-istio-envoy\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:56:09.682738 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:09.682550 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70a84f6f-0e62-4cb5-883c-b5b0b7adf253-istio-token\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:56:10.342633 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:10.342591 2567 generic.go:356] "Generic (PLEG): container finished" podID="70a84f6f-0e62-4cb5-883c-b5b0b7adf253" containerID="d4af842d6c2d50ccfbc3d7beb6ccc9e35f0605a02d899709530de5b969f0aee5" exitCode=0 Jul 08 13:56:10.342824 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:10.342642 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt" Jul 08 13:56:10.342824 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:10.342673 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt" event={"ID":"70a84f6f-0e62-4cb5-883c-b5b0b7adf253","Type":"ContainerDied","Data":"d4af842d6c2d50ccfbc3d7beb6ccc9e35f0605a02d899709530de5b969f0aee5"} Jul 08 13:56:10.342824 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:10.342739 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt" event={"ID":"70a84f6f-0e62-4cb5-883c-b5b0b7adf253","Type":"ContainerDied","Data":"e8092b14fcb29b6df47c2ea7e7a75a25b160d79f09f0d34d85e0e8758afd9228"} Jul 08 13:56:10.342824 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:10.342771 2567 scope.go:117] "RemoveContainer" containerID="d4af842d6c2d50ccfbc3d7beb6ccc9e35f0605a02d899709530de5b969f0aee5" Jul 08 13:56:10.351550 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:10.351530 2567 scope.go:117] "RemoveContainer" containerID="6e973a41d58cf5d572dbc6de7021d10c94a4af5399b42cf86dd356f5ab0fbc3b" Jul 08 13:56:10.360767 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:10.360525 2567 scope.go:117] "RemoveContainer" containerID="d4af842d6c2d50ccfbc3d7beb6ccc9e35f0605a02d899709530de5b969f0aee5" Jul 08 13:56:10.360849 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:56:10.360791 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4af842d6c2d50ccfbc3d7beb6ccc9e35f0605a02d899709530de5b969f0aee5\": container with ID starting with d4af842d6c2d50ccfbc3d7beb6ccc9e35f0605a02d899709530de5b969f0aee5 not found: ID does not exist" containerID="d4af842d6c2d50ccfbc3d7beb6ccc9e35f0605a02d899709530de5b969f0aee5" Jul 08 13:56:10.360849 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:10.360817 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4af842d6c2d50ccfbc3d7beb6ccc9e35f0605a02d899709530de5b969f0aee5"} err="failed to get container status \"d4af842d6c2d50ccfbc3d7beb6ccc9e35f0605a02d899709530de5b969f0aee5\": rpc error: code = NotFound desc = could not find container \"d4af842d6c2d50ccfbc3d7beb6ccc9e35f0605a02d899709530de5b969f0aee5\": container with ID starting with d4af842d6c2d50ccfbc3d7beb6ccc9e35f0605a02d899709530de5b969f0aee5 not found: ID does not exist" Jul 08 13:56:10.360849 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:10.360834 2567 scope.go:117] "RemoveContainer" containerID="6e973a41d58cf5d572dbc6de7021d10c94a4af5399b42cf86dd356f5ab0fbc3b" Jul 08 13:56:10.361108 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:56:10.361070 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e973a41d58cf5d572dbc6de7021d10c94a4af5399b42cf86dd356f5ab0fbc3b\": container with ID starting with 6e973a41d58cf5d572dbc6de7021d10c94a4af5399b42cf86dd356f5ab0fbc3b not found: ID does not exist" containerID="6e973a41d58cf5d572dbc6de7021d10c94a4af5399b42cf86dd356f5ab0fbc3b" Jul 08 13:56:10.361189 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:10.361112 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e973a41d58cf5d572dbc6de7021d10c94a4af5399b42cf86dd356f5ab0fbc3b"} err="failed to get container status \"6e973a41d58cf5d572dbc6de7021d10c94a4af5399b42cf86dd356f5ab0fbc3b\": rpc error: code = NotFound desc = could not find container \"6e973a41d58cf5d572dbc6de7021d10c94a4af5399b42cf86dd356f5ab0fbc3b\": container with ID starting with 6e973a41d58cf5d572dbc6de7021d10c94a4af5399b42cf86dd356f5ab0fbc3b not found: ID does not exist" Jul 08 13:56:10.368133 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:10.368103 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt"] Jul 08 13:56:10.372001 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:10.371979 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-8dd341-openshift-default-6fd745dbf5-gj6dt"] Jul 08 13:56:11.585012 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:11.584976 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq"] Jul 08 13:56:11.585442 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:11.585214 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq" podUID="c57186e8-fbbb-4bea-b681-ec3a6712972d" containerName="istio-proxy" containerID="cri-o://3d27f273010ca2fd01159a6abec0ca1d7514a8a6a355f71b78a2b05a46a99e69" gracePeriod=30 Jul 08 13:56:11.795772 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:11.795738 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70a84f6f-0e62-4cb5-883c-b5b0b7adf253" path="/var/lib/kubelet/pods/70a84f6f-0e62-4cb5-883c-b5b0b7adf253/volumes" Jul 08 13:56:12.442599 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:12.442563 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6"] Jul 08 13:56:12.442958 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:12.442941 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70a84f6f-0e62-4cb5-883c-b5b0b7adf253" containerName="istio-proxy" Jul 08 13:56:12.443055 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:12.442961 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="70a84f6f-0e62-4cb5-883c-b5b0b7adf253" containerName="istio-proxy" Jul 08 13:56:12.443055 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:12.442994 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70a84f6f-0e62-4cb5-883c-b5b0b7adf253" containerName="istio-proxy" Jul 08 13:56:12.443055 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:12.443003 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="70a84f6f-0e62-4cb5-883c-b5b0b7adf253" containerName="istio-proxy" Jul 08 13:56:12.443240 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:12.443091 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="70a84f6f-0e62-4cb5-883c-b5b0b7adf253" containerName="istio-proxy" Jul 08 13:56:12.443240 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:12.443105 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="70a84f6f-0e62-4cb5-883c-b5b0b7adf253" containerName="istio-proxy" Jul 08 13:56:12.446333 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:12.446308 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6" Jul 08 13:56:12.448954 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:12.448929 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-4f14d0-openshift-default-dockercfg-jpl67\"" Jul 08 13:56:12.460100 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:12.460042 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6"] Jul 08 13:56:12.508062 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:12.508027 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc7e1b28-587f-42bc-b09e-da10952440eb-istio-podinfo\") pod \"e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6\" (UID: \"cc7e1b28-587f-42bc-b09e-da10952440eb\") " pod="openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6" Jul 08 13:56:12.508247 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:12.508070 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc7e1b28-587f-42bc-b09e-da10952440eb-istio-token\") pod \"e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6\" (UID: \"cc7e1b28-587f-42bc-b09e-da10952440eb\") " pod="openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6" Jul 08 13:56:12.508247 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:12.508123 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc7e1b28-587f-42bc-b09e-da10952440eb-istiod-ca-cert\") pod \"e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6\" (UID: \"cc7e1b28-587f-42bc-b09e-da10952440eb\") " pod="openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6" Jul 08 13:56:12.508247 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:12.508149 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc7e1b28-587f-42bc-b09e-da10952440eb-workload-certs\") pod \"e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6\" (UID: \"cc7e1b28-587f-42bc-b09e-da10952440eb\") " pod="openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6" Jul 08 13:56:12.508247 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:12.508176 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc7e1b28-587f-42bc-b09e-da10952440eb-istio-data\") pod \"e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6\" (UID: \"cc7e1b28-587f-42bc-b09e-da10952440eb\") " pod="openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6" Jul 08 13:56:12.508247 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:12.508219 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc7e1b28-587f-42bc-b09e-da10952440eb-workload-socket\") pod \"e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6\" (UID: \"cc7e1b28-587f-42bc-b09e-da10952440eb\") " pod="openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6" Jul 08 13:56:12.508247 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:12.508236 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42cvm\" (UniqueName: \"kubernetes.io/projected/cc7e1b28-587f-42bc-b09e-da10952440eb-kube-api-access-42cvm\") pod \"e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6\" (UID: \"cc7e1b28-587f-42bc-b09e-da10952440eb\") " pod="openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6" Jul 08 13:56:12.508455 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:12.508281 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc7e1b28-587f-42bc-b09e-da10952440eb-istio-envoy\") pod \"e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6\" (UID: \"cc7e1b28-587f-42bc-b09e-da10952440eb\") " pod="openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6" Jul 08 13:56:12.508455 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:12.508307 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc7e1b28-587f-42bc-b09e-da10952440eb-credential-socket\") pod \"e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6\" (UID: \"cc7e1b28-587f-42bc-b09e-da10952440eb\") " pod="openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6" Jul 08 13:56:12.609754 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:12.609703 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc7e1b28-587f-42bc-b09e-da10952440eb-workload-socket\") pod \"e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6\" (UID: \"cc7e1b28-587f-42bc-b09e-da10952440eb\") " pod="openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6" Jul 08 13:56:12.609754 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:12.609752 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42cvm\" (UniqueName: \"kubernetes.io/projected/cc7e1b28-587f-42bc-b09e-da10952440eb-kube-api-access-42cvm\") pod \"e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6\" (UID: \"cc7e1b28-587f-42bc-b09e-da10952440eb\") " pod="openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6" Jul 08 13:56:12.610287 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:12.609875 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc7e1b28-587f-42bc-b09e-da10952440eb-istio-envoy\") pod \"e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6\" (UID: \"cc7e1b28-587f-42bc-b09e-da10952440eb\") " pod="openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6" Jul 08 13:56:12.610287 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:12.609923 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc7e1b28-587f-42bc-b09e-da10952440eb-credential-socket\") pod \"e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6\" (UID: \"cc7e1b28-587f-42bc-b09e-da10952440eb\") " pod="openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6" Jul 08 13:56:12.610287 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:12.609957 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc7e1b28-587f-42bc-b09e-da10952440eb-istio-podinfo\") pod \"e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6\" (UID: \"cc7e1b28-587f-42bc-b09e-da10952440eb\") " pod="openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6" Jul 08 13:56:12.610287 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:12.609984 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc7e1b28-587f-42bc-b09e-da10952440eb-istio-token\") pod \"e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6\" (UID: \"cc7e1b28-587f-42bc-b09e-da10952440eb\") " pod="openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6" Jul 08 13:56:12.610287 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:12.610008 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc7e1b28-587f-42bc-b09e-da10952440eb-istiod-ca-cert\") pod \"e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6\" (UID: \"cc7e1b28-587f-42bc-b09e-da10952440eb\") " pod="openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6" Jul 08 13:56:12.610287 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:12.610048 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc7e1b28-587f-42bc-b09e-da10952440eb-workload-certs\") pod \"e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6\" (UID: \"cc7e1b28-587f-42bc-b09e-da10952440eb\") " pod="openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6" Jul 08 13:56:12.610287 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:12.610123 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc7e1b28-587f-42bc-b09e-da10952440eb-istio-data\") pod \"e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6\" (UID: \"cc7e1b28-587f-42bc-b09e-da10952440eb\") " pod="openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6" Jul 08 13:56:12.610287 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:12.610174 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc7e1b28-587f-42bc-b09e-da10952440eb-workload-socket\") pod \"e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6\" (UID: \"cc7e1b28-587f-42bc-b09e-da10952440eb\") " pod="openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6" Jul 08 13:56:12.610640 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:12.610407 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc7e1b28-587f-42bc-b09e-da10952440eb-credential-socket\") pod \"e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6\" (UID: \"cc7e1b28-587f-42bc-b09e-da10952440eb\") " pod="openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6" Jul 08 13:56:12.610640 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:12.610447 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc7e1b28-587f-42bc-b09e-da10952440eb-istio-data\") pod \"e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6\" (UID: \"cc7e1b28-587f-42bc-b09e-da10952440eb\") " pod="openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6" Jul 08 13:56:12.610640 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:12.610496 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc7e1b28-587f-42bc-b09e-da10952440eb-workload-certs\") pod \"e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6\" (UID: \"cc7e1b28-587f-42bc-b09e-da10952440eb\") " pod="openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6" Jul 08 13:56:12.610748 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:12.610688 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc7e1b28-587f-42bc-b09e-da10952440eb-istiod-ca-cert\") pod \"e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6\" (UID: \"cc7e1b28-587f-42bc-b09e-da10952440eb\") " pod="openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6" Jul 08 13:56:12.612285 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:12.612248 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc7e1b28-587f-42bc-b09e-da10952440eb-istio-envoy\") pod \"e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6\" (UID: \"cc7e1b28-587f-42bc-b09e-da10952440eb\") " pod="openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6" Jul 08 13:56:12.612455 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:12.612428 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc7e1b28-587f-42bc-b09e-da10952440eb-istio-podinfo\") pod \"e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6\" (UID: \"cc7e1b28-587f-42bc-b09e-da10952440eb\") " pod="openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6" Jul 08 13:56:12.623524 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:12.623491 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-42cvm\" (UniqueName: \"kubernetes.io/projected/cc7e1b28-587f-42bc-b09e-da10952440eb-kube-api-access-42cvm\") pod \"e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6\" (UID: \"cc7e1b28-587f-42bc-b09e-da10952440eb\") " pod="openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6" Jul 08 13:56:12.623712 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:12.623663 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc7e1b28-587f-42bc-b09e-da10952440eb-istio-token\") pod \"e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6\" (UID: \"cc7e1b28-587f-42bc-b09e-da10952440eb\") " pod="openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6" Jul 08 13:56:12.756939 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:12.756901 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6" Jul 08 13:56:12.897096 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:12.897050 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6"] Jul 08 13:56:12.898703 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:56:12.898675 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc7e1b28_587f_42bc_b09e_da10952440eb.slice/crio-a8405314928fcc2ec40c0e56b0f8ea55f53a1a7cdf53432f2f40b30f494d1e26 WatchSource:0}: Error finding container a8405314928fcc2ec40c0e56b0f8ea55f53a1a7cdf53432f2f40b30f494d1e26: Status 404 returned error can't find the container with id a8405314928fcc2ec40c0e56b0f8ea55f53a1a7cdf53432f2f40b30f494d1e26 Jul 08 13:56:12.900711 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:12.900680 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:56:12.900778 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:12.900748 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:56:12.900811 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:12.900780 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:56:13.359738 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:13.359701 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6" event={"ID":"cc7e1b28-587f-42bc-b09e-da10952440eb","Type":"ContainerStarted","Data":"69f66d3b0b7e593d53c1f534c471091369a0fb58e35668825d1e1ad69006ad80"} Jul 08 13:56:13.359914 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:13.359757 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6" event={"ID":"cc7e1b28-587f-42bc-b09e-da10952440eb","Type":"ContainerStarted","Data":"a8405314928fcc2ec40c0e56b0f8ea55f53a1a7cdf53432f2f40b30f494d1e26"} Jul 08 13:56:13.381378 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:13.381328 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6" podStartSLOduration=1.381313988 podStartE2EDuration="1.381313988s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:56:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:56:13.378845985 +0000 UTC m=+1982.175786652" watchObservedRunningTime="2026-07-08 13:56:13.381313988 +0000 UTC m=+1982.178254641" Jul 08 13:56:13.757554 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:13.757525 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6" Jul 08 13:56:13.763101 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:13.763062 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6" Jul 08 13:56:14.365380 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:14.365353 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6" Jul 08 13:56:14.366224 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:14.366206 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6" Jul 08 13:56:16.829300 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:16.829278 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq" Jul 08 13:56:16.953857 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:16.953773 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c57186e8-fbbb-4bea-b681-ec3a6712972d-istio-data\") pod \"c57186e8-fbbb-4bea-b681-ec3a6712972d\" (UID: \"c57186e8-fbbb-4bea-b681-ec3a6712972d\") " Jul 08 13:56:16.953857 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:16.953813 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c57186e8-fbbb-4bea-b681-ec3a6712972d-workload-certs\") pod \"c57186e8-fbbb-4bea-b681-ec3a6712972d\" (UID: \"c57186e8-fbbb-4bea-b681-ec3a6712972d\") " Jul 08 13:56:16.953857 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:16.953854 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c57186e8-fbbb-4bea-b681-ec3a6712972d-istio-token\") pod \"c57186e8-fbbb-4bea-b681-ec3a6712972d\" (UID: \"c57186e8-fbbb-4bea-b681-ec3a6712972d\") " Jul 08 13:56:16.954148 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:16.953891 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c57186e8-fbbb-4bea-b681-ec3a6712972d-istio-podinfo\") pod \"c57186e8-fbbb-4bea-b681-ec3a6712972d\" (UID: \"c57186e8-fbbb-4bea-b681-ec3a6712972d\") " Jul 08 13:56:16.954148 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:16.953909 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-84w24\" (UniqueName: \"kubernetes.io/projected/c57186e8-fbbb-4bea-b681-ec3a6712972d-kube-api-access-84w24\") pod \"c57186e8-fbbb-4bea-b681-ec3a6712972d\" (UID: \"c57186e8-fbbb-4bea-b681-ec3a6712972d\") " Jul 08 13:56:16.954148 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:16.953939 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c57186e8-fbbb-4bea-b681-ec3a6712972d-istiod-ca-cert\") pod \"c57186e8-fbbb-4bea-b681-ec3a6712972d\" (UID: \"c57186e8-fbbb-4bea-b681-ec3a6712972d\") " Jul 08 13:56:16.954148 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:16.954000 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c57186e8-fbbb-4bea-b681-ec3a6712972d-credential-socket\") pod \"c57186e8-fbbb-4bea-b681-ec3a6712972d\" (UID: \"c57186e8-fbbb-4bea-b681-ec3a6712972d\") " Jul 08 13:56:16.954148 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:16.954137 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c57186e8-fbbb-4bea-b681-ec3a6712972d-workload-socket\") pod \"c57186e8-fbbb-4bea-b681-ec3a6712972d\" (UID: \"c57186e8-fbbb-4bea-b681-ec3a6712972d\") " Jul 08 13:56:16.954389 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:16.954167 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c57186e8-fbbb-4bea-b681-ec3a6712972d-istio-envoy\") pod \"c57186e8-fbbb-4bea-b681-ec3a6712972d\" (UID: \"c57186e8-fbbb-4bea-b681-ec3a6712972d\") " Jul 08 13:56:16.954389 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:16.954316 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c57186e8-fbbb-4bea-b681-ec3a6712972d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c57186e8-fbbb-4bea-b681-ec3a6712972d" (UID: "c57186e8-fbbb-4bea-b681-ec3a6712972d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:56:16.954389 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:16.954333 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c57186e8-fbbb-4bea-b681-ec3a6712972d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c57186e8-fbbb-4bea-b681-ec3a6712972d" (UID: "c57186e8-fbbb-4bea-b681-ec3a6712972d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:56:16.954552 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:16.954531 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c57186e8-fbbb-4bea-b681-ec3a6712972d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c57186e8-fbbb-4bea-b681-ec3a6712972d" (UID: "c57186e8-fbbb-4bea-b681-ec3a6712972d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:56:16.954602 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:16.954537 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c57186e8-fbbb-4bea-b681-ec3a6712972d-credential-socket\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:56:16.954602 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:16.954578 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c57186e8-fbbb-4bea-b681-ec3a6712972d-workload-certs\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:56:16.954783 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:16.954689 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c57186e8-fbbb-4bea-b681-ec3a6712972d-istio-data" (OuterVolumeSpecName: "istio-data") pod "c57186e8-fbbb-4bea-b681-ec3a6712972d" (UID: "c57186e8-fbbb-4bea-b681-ec3a6712972d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:56:16.954871 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:16.954838 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c57186e8-fbbb-4bea-b681-ec3a6712972d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c57186e8-fbbb-4bea-b681-ec3a6712972d" (UID: "c57186e8-fbbb-4bea-b681-ec3a6712972d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 13:56:16.956336 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:16.956302 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c57186e8-fbbb-4bea-b681-ec3a6712972d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c57186e8-fbbb-4bea-b681-ec3a6712972d" (UID: "c57186e8-fbbb-4bea-b681-ec3a6712972d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 13:56:16.956436 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:16.956327 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c57186e8-fbbb-4bea-b681-ec3a6712972d-kube-api-access-84w24" (OuterVolumeSpecName: "kube-api-access-84w24") pod "c57186e8-fbbb-4bea-b681-ec3a6712972d" (UID: "c57186e8-fbbb-4bea-b681-ec3a6712972d"). InnerVolumeSpecName "kube-api-access-84w24". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 13:56:16.956558 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:16.956542 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c57186e8-fbbb-4bea-b681-ec3a6712972d-istio-token" (OuterVolumeSpecName: "istio-token") pod "c57186e8-fbbb-4bea-b681-ec3a6712972d" (UID: "c57186e8-fbbb-4bea-b681-ec3a6712972d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 13:56:16.956617 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:16.956604 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c57186e8-fbbb-4bea-b681-ec3a6712972d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c57186e8-fbbb-4bea-b681-ec3a6712972d" (UID: "c57186e8-fbbb-4bea-b681-ec3a6712972d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:56:17.055791 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:17.055757 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c57186e8-fbbb-4bea-b681-ec3a6712972d-workload-socket\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:56:17.055791 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:17.055784 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c57186e8-fbbb-4bea-b681-ec3a6712972d-istio-envoy\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:56:17.055791 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:17.055793 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c57186e8-fbbb-4bea-b681-ec3a6712972d-istio-data\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:56:17.055791 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:17.055801 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c57186e8-fbbb-4bea-b681-ec3a6712972d-istio-token\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:56:17.056046 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:17.055809 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c57186e8-fbbb-4bea-b681-ec3a6712972d-istio-podinfo\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:56:17.056046 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:17.055818 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-84w24\" (UniqueName: \"kubernetes.io/projected/c57186e8-fbbb-4bea-b681-ec3a6712972d-kube-api-access-84w24\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:56:17.056046 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:17.055827 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c57186e8-fbbb-4bea-b681-ec3a6712972d-istiod-ca-cert\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:56:17.380595 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:17.380559 2567 generic.go:356] "Generic (PLEG): container finished" podID="c57186e8-fbbb-4bea-b681-ec3a6712972d" containerID="3d27f273010ca2fd01159a6abec0ca1d7514a8a6a355f71b78a2b05a46a99e69" exitCode=0 Jul 08 13:56:17.380816 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:17.380601 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq" event={"ID":"c57186e8-fbbb-4bea-b681-ec3a6712972d","Type":"ContainerDied","Data":"3d27f273010ca2fd01159a6abec0ca1d7514a8a6a355f71b78a2b05a46a99e69"} Jul 08 13:56:17.380816 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:17.380607 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq" Jul 08 13:56:17.380816 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:17.380674 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq" event={"ID":"c57186e8-fbbb-4bea-b681-ec3a6712972d","Type":"ContainerDied","Data":"c37e0435318398a7eff1f2ffe8b2baeb8e7c319f76ee783aea0ff289ac5774cd"} Jul 08 13:56:17.380816 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:17.380696 2567 scope.go:117] "RemoveContainer" containerID="3d27f273010ca2fd01159a6abec0ca1d7514a8a6a355f71b78a2b05a46a99e69" Jul 08 13:56:17.389896 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:17.389880 2567 scope.go:117] "RemoveContainer" containerID="8d2451bc5580a79079dcee9e0ac7c38525cb709bccc1b347c267fe763c68a51a" Jul 08 13:56:17.397493 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:17.397476 2567 scope.go:117] "RemoveContainer" containerID="3d27f273010ca2fd01159a6abec0ca1d7514a8a6a355f71b78a2b05a46a99e69" Jul 08 13:56:17.397737 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:56:17.397719 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d27f273010ca2fd01159a6abec0ca1d7514a8a6a355f71b78a2b05a46a99e69\": container with ID starting with 3d27f273010ca2fd01159a6abec0ca1d7514a8a6a355f71b78a2b05a46a99e69 not found: ID does not exist" containerID="3d27f273010ca2fd01159a6abec0ca1d7514a8a6a355f71b78a2b05a46a99e69" Jul 08 13:56:17.397776 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:17.397747 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d27f273010ca2fd01159a6abec0ca1d7514a8a6a355f71b78a2b05a46a99e69"} err="failed to get container status \"3d27f273010ca2fd01159a6abec0ca1d7514a8a6a355f71b78a2b05a46a99e69\": rpc error: code = NotFound desc = could not find container \"3d27f273010ca2fd01159a6abec0ca1d7514a8a6a355f71b78a2b05a46a99e69\": container with ID starting with 3d27f273010ca2fd01159a6abec0ca1d7514a8a6a355f71b78a2b05a46a99e69 not found: ID does not exist" Jul 08 13:56:17.397776 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:17.397763 2567 scope.go:117] "RemoveContainer" containerID="8d2451bc5580a79079dcee9e0ac7c38525cb709bccc1b347c267fe763c68a51a" Jul 08 13:56:17.397981 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:56:17.397960 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d2451bc5580a79079dcee9e0ac7c38525cb709bccc1b347c267fe763c68a51a\": container with ID starting with 8d2451bc5580a79079dcee9e0ac7c38525cb709bccc1b347c267fe763c68a51a not found: ID does not exist" containerID="8d2451bc5580a79079dcee9e0ac7c38525cb709bccc1b347c267fe763c68a51a" Jul 08 13:56:17.398027 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:17.397986 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d2451bc5580a79079dcee9e0ac7c38525cb709bccc1b347c267fe763c68a51a"} err="failed to get container status \"8d2451bc5580a79079dcee9e0ac7c38525cb709bccc1b347c267fe763c68a51a\": rpc error: code = NotFound desc = could not find container \"8d2451bc5580a79079dcee9e0ac7c38525cb709bccc1b347c267fe763c68a51a\": container with ID starting with 8d2451bc5580a79079dcee9e0ac7c38525cb709bccc1b347c267fe763c68a51a not found: ID does not exist" Jul 08 13:56:17.402868 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:17.402847 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq"] Jul 08 13:56:17.407382 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:17.407355 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-d1efd5-openshift-default-66f556b667-cbsjq"] Jul 08 13:56:17.795896 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:17.795859 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c57186e8-fbbb-4bea-b681-ec3a6712972d" path="/var/lib/kubelet/pods/c57186e8-fbbb-4bea-b681-ec3a6712972d/volumes" Jul 08 13:56:20.397035 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:20.397002 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz"] Jul 08 13:56:20.397428 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:20.397335 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c57186e8-fbbb-4bea-b681-ec3a6712972d" containerName="istio-proxy" Jul 08 13:56:20.397428 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:20.397347 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c57186e8-fbbb-4bea-b681-ec3a6712972d" containerName="istio-proxy" Jul 08 13:56:20.397428 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:20.397358 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c57186e8-fbbb-4bea-b681-ec3a6712972d" containerName="istio-proxy" Jul 08 13:56:20.397428 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:20.397365 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c57186e8-fbbb-4bea-b681-ec3a6712972d" containerName="istio-proxy" Jul 08 13:56:20.397428 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:20.397420 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c57186e8-fbbb-4bea-b681-ec3a6712972d" containerName="istio-proxy" Jul 08 13:56:20.397428 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:20.397429 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c57186e8-fbbb-4bea-b681-ec3a6712972d" containerName="istio-proxy" Jul 08 13:56:20.400948 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:20.400926 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz" Jul 08 13:56:20.403348 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:20.403328 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-e225ef-openshift-default-dockercfg-lvdv8\"" Jul 08 13:56:20.410990 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:20.410964 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz"] Jul 08 13:56:20.486647 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:20.486597 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f52dd567-378a-4522-a7e9-0facd6925ee6-istio-token\") pod \"e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz\" (UID: \"f52dd567-378a-4522-a7e9-0facd6925ee6\") " pod="openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz" Jul 08 13:56:20.486647 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:20.486651 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f52dd567-378a-4522-a7e9-0facd6925ee6-istio-data\") pod \"e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz\" (UID: \"f52dd567-378a-4522-a7e9-0facd6925ee6\") " pod="openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz" Jul 08 13:56:20.486893 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:20.486760 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f52dd567-378a-4522-a7e9-0facd6925ee6-workload-certs\") pod \"e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz\" (UID: \"f52dd567-378a-4522-a7e9-0facd6925ee6\") " pod="openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz" Jul 08 13:56:20.486893 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:20.486788 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f52dd567-378a-4522-a7e9-0facd6925ee6-workload-socket\") pod \"e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz\" (UID: \"f52dd567-378a-4522-a7e9-0facd6925ee6\") " pod="openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz" Jul 08 13:56:20.486893 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:20.486815 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f52dd567-378a-4522-a7e9-0facd6925ee6-istiod-ca-cert\") pod \"e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz\" (UID: \"f52dd567-378a-4522-a7e9-0facd6925ee6\") " pod="openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz" Jul 08 13:56:20.487048 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:20.486925 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f52dd567-378a-4522-a7e9-0facd6925ee6-istio-podinfo\") pod \"e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz\" (UID: \"f52dd567-378a-4522-a7e9-0facd6925ee6\") " pod="openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz" Jul 08 13:56:20.487048 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:20.486963 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f52dd567-378a-4522-a7e9-0facd6925ee6-credential-socket\") pod \"e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz\" (UID: \"f52dd567-378a-4522-a7e9-0facd6925ee6\") " pod="openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz" Jul 08 13:56:20.487048 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:20.486992 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4jpt\" (UniqueName: \"kubernetes.io/projected/f52dd567-378a-4522-a7e9-0facd6925ee6-kube-api-access-z4jpt\") pod \"e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz\" (UID: \"f52dd567-378a-4522-a7e9-0facd6925ee6\") " pod="openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz" Jul 08 13:56:20.487048 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:20.487014 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f52dd567-378a-4522-a7e9-0facd6925ee6-istio-envoy\") pod \"e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz\" (UID: \"f52dd567-378a-4522-a7e9-0facd6925ee6\") " pod="openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz" Jul 08 13:56:20.588008 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:20.587973 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f52dd567-378a-4522-a7e9-0facd6925ee6-istio-token\") pod \"e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz\" (UID: \"f52dd567-378a-4522-a7e9-0facd6925ee6\") " pod="openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz" Jul 08 13:56:20.588008 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:20.588012 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f52dd567-378a-4522-a7e9-0facd6925ee6-istio-data\") pod \"e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz\" (UID: \"f52dd567-378a-4522-a7e9-0facd6925ee6\") " pod="openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz" Jul 08 13:56:20.588239 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:20.588055 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f52dd567-378a-4522-a7e9-0facd6925ee6-workload-certs\") pod \"e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz\" (UID: \"f52dd567-378a-4522-a7e9-0facd6925ee6\") " pod="openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz" Jul 08 13:56:20.588239 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:20.588201 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f52dd567-378a-4522-a7e9-0facd6925ee6-workload-socket\") pod \"e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz\" (UID: \"f52dd567-378a-4522-a7e9-0facd6925ee6\") " pod="openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz" Jul 08 13:56:20.588323 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:20.588238 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f52dd567-378a-4522-a7e9-0facd6925ee6-istiod-ca-cert\") pod \"e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz\" (UID: \"f52dd567-378a-4522-a7e9-0facd6925ee6\") " pod="openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz" Jul 08 13:56:20.588323 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:20.588292 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f52dd567-378a-4522-a7e9-0facd6925ee6-istio-podinfo\") pod \"e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz\" (UID: \"f52dd567-378a-4522-a7e9-0facd6925ee6\") " pod="openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz" Jul 08 13:56:20.588420 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:20.588341 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f52dd567-378a-4522-a7e9-0facd6925ee6-credential-socket\") pod \"e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz\" (UID: \"f52dd567-378a-4522-a7e9-0facd6925ee6\") " pod="openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz" Jul 08 13:56:20.588420 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:20.588377 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4jpt\" (UniqueName: \"kubernetes.io/projected/f52dd567-378a-4522-a7e9-0facd6925ee6-kube-api-access-z4jpt\") pod \"e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz\" (UID: \"f52dd567-378a-4522-a7e9-0facd6925ee6\") " pod="openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz" Jul 08 13:56:20.588420 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:20.588407 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f52dd567-378a-4522-a7e9-0facd6925ee6-istio-envoy\") pod \"e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz\" (UID: \"f52dd567-378a-4522-a7e9-0facd6925ee6\") " pod="openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz" Jul 08 13:56:20.588584 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:20.588426 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f52dd567-378a-4522-a7e9-0facd6925ee6-workload-certs\") pod \"e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz\" (UID: \"f52dd567-378a-4522-a7e9-0facd6925ee6\") " pod="openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz" Jul 08 13:56:20.588584 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:20.588468 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f52dd567-378a-4522-a7e9-0facd6925ee6-istio-data\") pod \"e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz\" (UID: \"f52dd567-378a-4522-a7e9-0facd6925ee6\") " pod="openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz" Jul 08 13:56:20.588584 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:20.588559 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f52dd567-378a-4522-a7e9-0facd6925ee6-workload-socket\") pod \"e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz\" (UID: \"f52dd567-378a-4522-a7e9-0facd6925ee6\") " pod="openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz" Jul 08 13:56:20.588959 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:20.588932 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f52dd567-378a-4522-a7e9-0facd6925ee6-credential-socket\") pod \"e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz\" (UID: \"f52dd567-378a-4522-a7e9-0facd6925ee6\") " pod="openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz" Jul 08 13:56:20.589019 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:20.588957 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f52dd567-378a-4522-a7e9-0facd6925ee6-istiod-ca-cert\") pod \"e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz\" (UID: \"f52dd567-378a-4522-a7e9-0facd6925ee6\") " pod="openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz" Jul 08 13:56:20.590594 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:20.590561 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f52dd567-378a-4522-a7e9-0facd6925ee6-istio-envoy\") pod \"e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz\" (UID: \"f52dd567-378a-4522-a7e9-0facd6925ee6\") " pod="openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz" Jul 08 13:56:20.590785 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:20.590763 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f52dd567-378a-4522-a7e9-0facd6925ee6-istio-podinfo\") pod \"e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz\" (UID: \"f52dd567-378a-4522-a7e9-0facd6925ee6\") " pod="openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz" Jul 08 13:56:20.597755 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:20.597731 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f52dd567-378a-4522-a7e9-0facd6925ee6-istio-token\") pod \"e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz\" (UID: \"f52dd567-378a-4522-a7e9-0facd6925ee6\") " pod="openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz" Jul 08 13:56:20.598508 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:20.598489 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4jpt\" (UniqueName: \"kubernetes.io/projected/f52dd567-378a-4522-a7e9-0facd6925ee6-kube-api-access-z4jpt\") pod \"e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz\" (UID: \"f52dd567-378a-4522-a7e9-0facd6925ee6\") " pod="openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz" Jul 08 13:56:20.713543 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:20.713513 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz" Jul 08 13:56:20.852045 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:20.852021 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz"] Jul 08 13:56:20.853746 ip-10-0-137-164 kubenswrapper[2567]: W0708 13:56:20.853712 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf52dd567_378a_4522_a7e9_0facd6925ee6.slice/crio-22cd0a00ea30cbed49e3acdbe45c4db4b7d4050f173971e1ade16e616e083928 WatchSource:0}: Error finding container 22cd0a00ea30cbed49e3acdbe45c4db4b7d4050f173971e1ade16e616e083928: Status 404 returned error can't find the container with id 22cd0a00ea30cbed49e3acdbe45c4db4b7d4050f173971e1ade16e616e083928 Jul 08 13:56:20.856110 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:20.856064 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:56:20.856212 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:20.856144 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:56:20.856212 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:20.856172 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:56:21.402551 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:21.402513 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz" event={"ID":"f52dd567-378a-4522-a7e9-0facd6925ee6","Type":"ContainerStarted","Data":"cd6f6bacd4db92dbd42af1976aa309bd9a741e8e904d8d1e406c8d9dd8cea0a0"} Jul 08 13:56:21.403008 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:21.402562 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz" event={"ID":"f52dd567-378a-4522-a7e9-0facd6925ee6","Type":"ContainerStarted","Data":"22cd0a00ea30cbed49e3acdbe45c4db4b7d4050f173971e1ade16e616e083928"} Jul 08 13:56:21.423197 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:21.423145 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz" podStartSLOduration=1.42312867 podStartE2EDuration="1.42312867s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:56:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:56:21.422197314 +0000 UTC m=+1990.219137971" watchObservedRunningTime="2026-07-08 13:56:21.42312867 +0000 UTC m=+1990.220069326" Jul 08 13:56:21.714534 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:21.714504 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz" Jul 08 13:56:21.719348 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:21.719325 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz" Jul 08 13:56:22.407887 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:22.407855 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz" Jul 08 13:56:22.409003 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:22.408985 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz" Jul 08 13:56:23.096090 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:23.096041 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:56:30.409253 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:30.409205 2567 patch_prober.go:28] interesting pod/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 08 13:56:30.409792 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:30.409274 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6" podUID="cc7e1b28-587f-42bc-b09e-da10952440eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 08 13:56:32.461931 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:32.461896 2567 generic.go:356] "Generic (PLEG): container finished" podID="cc7e1b28-587f-42bc-b09e-da10952440eb" containerID="69f66d3b0b7e593d53c1f534c471091369a0fb58e35668825d1e1ad69006ad80" exitCode=0 Jul 08 13:56:32.462413 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:32.461956 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6" event={"ID":"cc7e1b28-587f-42bc-b09e-da10952440eb","Type":"ContainerDied","Data":"69f66d3b0b7e593d53c1f534c471091369a0fb58e35668825d1e1ad69006ad80"} Jul 08 13:56:32.462413 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:32.462353 2567 scope.go:117] "RemoveContainer" containerID="69f66d3b0b7e593d53c1f534c471091369a0fb58e35668825d1e1ad69006ad80" Jul 08 13:56:32.463403 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:32.463375 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:56:32.463481 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:32.463443 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:56:32.463481 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:32.463473 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:56:33.468816 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:33.468783 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6" event={"ID":"cc7e1b28-587f-42bc-b09e-da10952440eb","Type":"ContainerStarted","Data":"f6adfcf9fd5344ab99d34d3d9793295154004f7aaca85f93c886d56f9e1ca3a7"} Jul 08 13:56:33.757237 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:33.757144 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6" Jul 08 13:56:33.757412 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:33.757390 2567 patch_prober.go:28] interesting pod/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body= Jul 08 13:56:33.757470 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:33.757450 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6" podUID="cc7e1b28-587f-42bc-b09e-da10952440eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" Jul 08 13:56:34.757749 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:34.757712 2567 patch_prober.go:28] interesting pod/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body= Jul 08 13:56:34.758203 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:34.757797 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6" podUID="cc7e1b28-587f-42bc-b09e-da10952440eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" Jul 08 13:56:34.796912 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:34.796863 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:56:35.757520 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:35.757482 2567 patch_prober.go:28] interesting pod/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body= Jul 08 13:56:35.757752 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:35.757541 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6" podUID="cc7e1b28-587f-42bc-b09e-da10952440eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" Jul 08 13:56:36.757731 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:36.757693 2567 patch_prober.go:28] interesting pod/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body= Jul 08 13:56:36.758284 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:36.757761 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6" podUID="cc7e1b28-587f-42bc-b09e-da10952440eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" Jul 08 13:56:37.762036 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:37.762007 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6" Jul 08 13:56:37.762473 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:37.762399 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6" Jul 08 13:56:37.763237 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:37.763212 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6" Jul 08 13:56:38.216849 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:38.216644 2567 patch_prober.go:28] interesting pod/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": read tcp 10.133.0.2:41880->10.133.0.42:15021: read: connection reset by peer" start-of-body= Jul 08 13:56:38.216849 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:38.216703 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz" podUID="f52dd567-378a-4522-a7e9-0facd6925ee6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": read tcp 10.133.0.2:41880->10.133.0.42:15021: read: connection reset by peer" Jul 08 13:56:38.496203 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:38.496124 2567 generic.go:356] "Generic (PLEG): container finished" podID="f52dd567-378a-4522-a7e9-0facd6925ee6" containerID="cd6f6bacd4db92dbd42af1976aa309bd9a741e8e904d8d1e406c8d9dd8cea0a0" exitCode=0 Jul 08 13:56:38.496352 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:38.496194 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz" event={"ID":"f52dd567-378a-4522-a7e9-0facd6925ee6","Type":"ContainerDied","Data":"cd6f6bacd4db92dbd42af1976aa309bd9a741e8e904d8d1e406c8d9dd8cea0a0"} Jul 08 13:56:38.496678 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:38.496665 2567 scope.go:117] "RemoveContainer" containerID="cd6f6bacd4db92dbd42af1976aa309bd9a741e8e904d8d1e406c8d9dd8cea0a0" Jul 08 13:56:38.497621 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:38.497595 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:56:38.497742 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:38.497676 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:56:38.497742 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:38.497721 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 13:56:39.507463 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:39.507423 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz" event={"ID":"f52dd567-378a-4522-a7e9-0facd6925ee6","Type":"ContainerStarted","Data":"631098ef4f90b0028d90a22a1b6d8366f172e8505dedf9adb9c1b3b204645cfe"} Jul 08 13:56:39.713760 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:39.713728 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz" Jul 08 13:56:39.713929 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:39.713849 2567 patch_prober.go:28] interesting pod/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Jul 08 13:56:39.713929 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:39.713909 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz" podUID="f52dd567-378a-4522-a7e9-0facd6925ee6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Jul 08 13:56:40.714307 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:40.714273 2567 patch_prober.go:28] interesting pod/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Jul 08 13:56:40.714782 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:40.714346 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz" podUID="f52dd567-378a-4522-a7e9-0facd6925ee6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Jul 08 13:56:41.714746 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:41.714712 2567 patch_prober.go:28] interesting pod/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Jul 08 13:56:41.715257 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:41.714764 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz" podUID="f52dd567-378a-4522-a7e9-0facd6925ee6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Jul 08 13:56:42.714409 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:42.714373 2567 patch_prober.go:28] interesting pod/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Jul 08 13:56:42.714594 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:42.714429 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz" podUID="f52dd567-378a-4522-a7e9-0facd6925ee6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Jul 08 13:56:43.717718 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:43.717683 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz" Jul 08 13:56:43.718191 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:43.717950 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz" Jul 08 13:56:43.718684 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:56:43.718668 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz" Jul 08 13:59:36.896545 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:36.896508 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:59:39.290674 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:39.290632 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6"] Jul 08 13:59:39.291108 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:39.290868 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6" podUID="cc7e1b28-587f-42bc-b09e-da10952440eb" containerName="istio-proxy" containerID="cri-o://f6adfcf9fd5344ab99d34d3d9793295154004f7aaca85f93c886d56f9e1ca3a7" gracePeriod=30 Jul 08 13:59:44.430491 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:44.430459 2567 generic.go:356] "Generic (PLEG): container finished" podID="cc7e1b28-587f-42bc-b09e-da10952440eb" containerID="f6adfcf9fd5344ab99d34d3d9793295154004f7aaca85f93c886d56f9e1ca3a7" exitCode=0 Jul 08 13:59:44.430866 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:44.430528 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6" event={"ID":"cc7e1b28-587f-42bc-b09e-da10952440eb","Type":"ContainerDied","Data":"f6adfcf9fd5344ab99d34d3d9793295154004f7aaca85f93c886d56f9e1ca3a7"} Jul 08 13:59:44.430866 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:44.430592 2567 scope.go:117] "RemoveContainer" containerID="69f66d3b0b7e593d53c1f534c471091369a0fb58e35668825d1e1ad69006ad80" Jul 08 13:59:44.537209 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:44.537187 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6" Jul 08 13:59:44.578196 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:44.578123 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc7e1b28-587f-42bc-b09e-da10952440eb-istiod-ca-cert\") pod \"cc7e1b28-587f-42bc-b09e-da10952440eb\" (UID: \"cc7e1b28-587f-42bc-b09e-da10952440eb\") " Jul 08 13:59:44.578196 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:44.578167 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc7e1b28-587f-42bc-b09e-da10952440eb-workload-socket\") pod \"cc7e1b28-587f-42bc-b09e-da10952440eb\" (UID: \"cc7e1b28-587f-42bc-b09e-da10952440eb\") " Jul 08 13:59:44.578196 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:44.578187 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-42cvm\" (UniqueName: \"kubernetes.io/projected/cc7e1b28-587f-42bc-b09e-da10952440eb-kube-api-access-42cvm\") pod \"cc7e1b28-587f-42bc-b09e-da10952440eb\" (UID: \"cc7e1b28-587f-42bc-b09e-da10952440eb\") " Jul 08 13:59:44.578435 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:44.578208 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc7e1b28-587f-42bc-b09e-da10952440eb-credential-socket\") pod \"cc7e1b28-587f-42bc-b09e-da10952440eb\" (UID: \"cc7e1b28-587f-42bc-b09e-da10952440eb\") " Jul 08 13:59:44.578435 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:44.578347 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc7e1b28-587f-42bc-b09e-da10952440eb-istio-token\") pod \"cc7e1b28-587f-42bc-b09e-da10952440eb\" (UID: \"cc7e1b28-587f-42bc-b09e-da10952440eb\") " Jul 08 13:59:44.578435 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:44.578430 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc7e1b28-587f-42bc-b09e-da10952440eb-istio-envoy\") pod \"cc7e1b28-587f-42bc-b09e-da10952440eb\" (UID: \"cc7e1b28-587f-42bc-b09e-da10952440eb\") " Jul 08 13:59:44.578599 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:44.578468 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc7e1b28-587f-42bc-b09e-da10952440eb-istio-data\") pod \"cc7e1b28-587f-42bc-b09e-da10952440eb\" (UID: \"cc7e1b28-587f-42bc-b09e-da10952440eb\") " Jul 08 13:59:44.578599 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:44.578508 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc7e1b28-587f-42bc-b09e-da10952440eb-workload-certs\") pod \"cc7e1b28-587f-42bc-b09e-da10952440eb\" (UID: \"cc7e1b28-587f-42bc-b09e-da10952440eb\") " Jul 08 13:59:44.578599 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:44.578552 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc7e1b28-587f-42bc-b09e-da10952440eb-istio-podinfo\") pod \"cc7e1b28-587f-42bc-b09e-da10952440eb\" (UID: \"cc7e1b28-587f-42bc-b09e-da10952440eb\") " Jul 08 13:59:44.578599 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:44.578461 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc7e1b28-587f-42bc-b09e-da10952440eb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cc7e1b28-587f-42bc-b09e-da10952440eb" (UID: "cc7e1b28-587f-42bc-b09e-da10952440eb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:59:44.578599 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:44.578568 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc7e1b28-587f-42bc-b09e-da10952440eb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cc7e1b28-587f-42bc-b09e-da10952440eb" (UID: "cc7e1b28-587f-42bc-b09e-da10952440eb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:59:44.578925 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:44.578754 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc7e1b28-587f-42bc-b09e-da10952440eb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cc7e1b28-587f-42bc-b09e-da10952440eb" (UID: "cc7e1b28-587f-42bc-b09e-da10952440eb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:59:44.578925 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:44.578851 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc7e1b28-587f-42bc-b09e-da10952440eb-workload-certs\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:59:44.578925 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:44.578871 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc7e1b28-587f-42bc-b09e-da10952440eb-workload-socket\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:59:44.578925 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:44.578883 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc7e1b28-587f-42bc-b09e-da10952440eb-credential-socket\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:59:44.578925 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:44.578847 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc7e1b28-587f-42bc-b09e-da10952440eb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cc7e1b28-587f-42bc-b09e-da10952440eb" (UID: "cc7e1b28-587f-42bc-b09e-da10952440eb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 13:59:44.579344 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:44.579315 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc7e1b28-587f-42bc-b09e-da10952440eb-istio-data" (OuterVolumeSpecName: "istio-data") pod "cc7e1b28-587f-42bc-b09e-da10952440eb" (UID: "cc7e1b28-587f-42bc-b09e-da10952440eb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:59:44.580635 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:44.580600 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc7e1b28-587f-42bc-b09e-da10952440eb-istio-token" (OuterVolumeSpecName: "istio-token") pod "cc7e1b28-587f-42bc-b09e-da10952440eb" (UID: "cc7e1b28-587f-42bc-b09e-da10952440eb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 13:59:44.580877 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:44.580846 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc7e1b28-587f-42bc-b09e-da10952440eb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cc7e1b28-587f-42bc-b09e-da10952440eb" (UID: "cc7e1b28-587f-42bc-b09e-da10952440eb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:59:44.581304 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:44.581273 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cc7e1b28-587f-42bc-b09e-da10952440eb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cc7e1b28-587f-42bc-b09e-da10952440eb" (UID: "cc7e1b28-587f-42bc-b09e-da10952440eb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 13:59:44.581492 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:44.581473 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc7e1b28-587f-42bc-b09e-da10952440eb-kube-api-access-42cvm" (OuterVolumeSpecName: "kube-api-access-42cvm") pod "cc7e1b28-587f-42bc-b09e-da10952440eb" (UID: "cc7e1b28-587f-42bc-b09e-da10952440eb"). InnerVolumeSpecName "kube-api-access-42cvm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 13:59:44.679811 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:44.679778 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc7e1b28-587f-42bc-b09e-da10952440eb-istiod-ca-cert\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:59:44.679811 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:44.679804 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-42cvm\" (UniqueName: \"kubernetes.io/projected/cc7e1b28-587f-42bc-b09e-da10952440eb-kube-api-access-42cvm\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:59:44.679811 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:44.679816 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc7e1b28-587f-42bc-b09e-da10952440eb-istio-token\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:59:44.680038 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:44.679828 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc7e1b28-587f-42bc-b09e-da10952440eb-istio-envoy\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:59:44.680038 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:44.679837 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc7e1b28-587f-42bc-b09e-da10952440eb-istio-data\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:59:44.680038 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:44.679844 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc7e1b28-587f-42bc-b09e-da10952440eb-istio-podinfo\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:59:45.436438 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:45.436404 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6" Jul 08 13:59:45.436438 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:45.436413 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6" event={"ID":"cc7e1b28-587f-42bc-b09e-da10952440eb","Type":"ContainerDied","Data":"a8405314928fcc2ec40c0e56b0f8ea55f53a1a7cdf53432f2f40b30f494d1e26"} Jul 08 13:59:45.437019 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:45.436500 2567 scope.go:117] "RemoveContainer" containerID="f6adfcf9fd5344ab99d34d3d9793295154004f7aaca85f93c886d56f9e1ca3a7" Jul 08 13:59:45.466254 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:45.466228 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6"] Jul 08 13:59:45.470845 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:45.470820 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-4f14d0-openshift-default-d56fc587-rz2l6"] Jul 08 13:59:45.795233 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:45.795201 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc7e1b28-587f-42bc-b09e-da10952440eb" path="/var/lib/kubelet/pods/cc7e1b28-587f-42bc-b09e-da10952440eb/volumes" Jul 08 13:59:46.358568 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:46.358539 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz"] Jul 08 13:59:46.358818 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:46.358792 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz" podUID="f52dd567-378a-4522-a7e9-0facd6925ee6" containerName="istio-proxy" containerID="cri-o://631098ef4f90b0028d90a22a1b6d8366f172e8505dedf9adb9c1b3b204645cfe" gracePeriod=30 Jul 08 13:59:49.487324 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:49.487292 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hpf2w"] Jul 08 13:59:51.599349 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:51.599321 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz" Jul 08 13:59:51.739333 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:51.739297 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f52dd567-378a-4522-a7e9-0facd6925ee6-credential-socket\") pod \"f52dd567-378a-4522-a7e9-0facd6925ee6\" (UID: \"f52dd567-378a-4522-a7e9-0facd6925ee6\") " Jul 08 13:59:51.739541 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:51.739350 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z4jpt\" (UniqueName: \"kubernetes.io/projected/f52dd567-378a-4522-a7e9-0facd6925ee6-kube-api-access-z4jpt\") pod \"f52dd567-378a-4522-a7e9-0facd6925ee6\" (UID: \"f52dd567-378a-4522-a7e9-0facd6925ee6\") " Jul 08 13:59:51.739541 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:51.739377 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f52dd567-378a-4522-a7e9-0facd6925ee6-istio-podinfo\") pod \"f52dd567-378a-4522-a7e9-0facd6925ee6\" (UID: \"f52dd567-378a-4522-a7e9-0facd6925ee6\") " Jul 08 13:59:51.739541 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:51.739401 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f52dd567-378a-4522-a7e9-0facd6925ee6-istio-envoy\") pod \"f52dd567-378a-4522-a7e9-0facd6925ee6\" (UID: \"f52dd567-378a-4522-a7e9-0facd6925ee6\") " Jul 08 13:59:51.739541 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:51.739452 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f52dd567-378a-4522-a7e9-0facd6925ee6-istio-token\") pod \"f52dd567-378a-4522-a7e9-0facd6925ee6\" (UID: \"f52dd567-378a-4522-a7e9-0facd6925ee6\") " Jul 08 13:59:51.739541 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:51.739495 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f52dd567-378a-4522-a7e9-0facd6925ee6-workload-certs\") pod \"f52dd567-378a-4522-a7e9-0facd6925ee6\" (UID: \"f52dd567-378a-4522-a7e9-0facd6925ee6\") " Jul 08 13:59:51.739541 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:51.739523 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f52dd567-378a-4522-a7e9-0facd6925ee6-istiod-ca-cert\") pod \"f52dd567-378a-4522-a7e9-0facd6925ee6\" (UID: \"f52dd567-378a-4522-a7e9-0facd6925ee6\") " Jul 08 13:59:51.739836 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:51.739572 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f52dd567-378a-4522-a7e9-0facd6925ee6-workload-socket\") pod \"f52dd567-378a-4522-a7e9-0facd6925ee6\" (UID: \"f52dd567-378a-4522-a7e9-0facd6925ee6\") " Jul 08 13:59:51.739836 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:51.739611 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f52dd567-378a-4522-a7e9-0facd6925ee6-istio-data\") pod \"f52dd567-378a-4522-a7e9-0facd6925ee6\" (UID: \"f52dd567-378a-4522-a7e9-0facd6925ee6\") " Jul 08 13:59:51.739836 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:51.739614 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f52dd567-378a-4522-a7e9-0facd6925ee6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f52dd567-378a-4522-a7e9-0facd6925ee6" (UID: "f52dd567-378a-4522-a7e9-0facd6925ee6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:59:51.739986 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:51.739874 2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f52dd567-378a-4522-a7e9-0facd6925ee6-credential-socket\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:59:51.739986 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:51.739864 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f52dd567-378a-4522-a7e9-0facd6925ee6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f52dd567-378a-4522-a7e9-0facd6925ee6" (UID: "f52dd567-378a-4522-a7e9-0facd6925ee6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:59:51.740126 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:51.740028 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f52dd567-378a-4522-a7e9-0facd6925ee6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f52dd567-378a-4522-a7e9-0facd6925ee6" (UID: "f52dd567-378a-4522-a7e9-0facd6925ee6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:59:51.740126 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:51.740048 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f52dd567-378a-4522-a7e9-0facd6925ee6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f52dd567-378a-4522-a7e9-0facd6925ee6" (UID: "f52dd567-378a-4522-a7e9-0facd6925ee6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 13:59:51.740629 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:51.740604 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f52dd567-378a-4522-a7e9-0facd6925ee6-istio-data" (OuterVolumeSpecName: "istio-data") pod "f52dd567-378a-4522-a7e9-0facd6925ee6" (UID: "f52dd567-378a-4522-a7e9-0facd6925ee6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:59:51.741633 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:51.741611 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f52dd567-378a-4522-a7e9-0facd6925ee6-istio-token" (OuterVolumeSpecName: "istio-token") pod "f52dd567-378a-4522-a7e9-0facd6925ee6" (UID: "f52dd567-378a-4522-a7e9-0facd6925ee6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 13:59:51.741844 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:51.741828 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f52dd567-378a-4522-a7e9-0facd6925ee6-kube-api-access-z4jpt" (OuterVolumeSpecName: "kube-api-access-z4jpt") pod "f52dd567-378a-4522-a7e9-0facd6925ee6" (UID: "f52dd567-378a-4522-a7e9-0facd6925ee6"). InnerVolumeSpecName "kube-api-access-z4jpt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 13:59:51.741960 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:51.741946 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f52dd567-378a-4522-a7e9-0facd6925ee6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f52dd567-378a-4522-a7e9-0facd6925ee6" (UID: "f52dd567-378a-4522-a7e9-0facd6925ee6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 13:59:51.742002 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:51.741975 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f52dd567-378a-4522-a7e9-0facd6925ee6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f52dd567-378a-4522-a7e9-0facd6925ee6" (UID: "f52dd567-378a-4522-a7e9-0facd6925ee6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:59:51.841309 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:51.841282 2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f52dd567-378a-4522-a7e9-0facd6925ee6-workload-certs\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:59:51.841309 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:51.841305 2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f52dd567-378a-4522-a7e9-0facd6925ee6-istiod-ca-cert\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:59:51.841502 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:51.841315 2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f52dd567-378a-4522-a7e9-0facd6925ee6-workload-socket\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:59:51.841502 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:51.841326 2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f52dd567-378a-4522-a7e9-0facd6925ee6-istio-data\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:59:51.841502 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:51.841336 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z4jpt\" (UniqueName: \"kubernetes.io/projected/f52dd567-378a-4522-a7e9-0facd6925ee6-kube-api-access-z4jpt\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:59:51.841502 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:51.841345 2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f52dd567-378a-4522-a7e9-0facd6925ee6-istio-podinfo\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:59:51.841502 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:51.841352 2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f52dd567-378a-4522-a7e9-0facd6925ee6-istio-envoy\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:59:51.841502 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:51.841360 2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f52dd567-378a-4522-a7e9-0facd6925ee6-istio-token\") on node \"ip-10-0-137-164.ec2.internal\" DevicePath \"\"" Jul 08 13:59:52.479425 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:52.479388 2567 generic.go:356] "Generic (PLEG): container finished" podID="f52dd567-378a-4522-a7e9-0facd6925ee6" containerID="631098ef4f90b0028d90a22a1b6d8366f172e8505dedf9adb9c1b3b204645cfe" exitCode=0 Jul 08 13:59:52.479606 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:52.479445 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz" Jul 08 13:59:52.479606 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:52.479469 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz" event={"ID":"f52dd567-378a-4522-a7e9-0facd6925ee6","Type":"ContainerDied","Data":"631098ef4f90b0028d90a22a1b6d8366f172e8505dedf9adb9c1b3b204645cfe"} Jul 08 13:59:52.479606 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:52.479523 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz" event={"ID":"f52dd567-378a-4522-a7e9-0facd6925ee6","Type":"ContainerDied","Data":"22cd0a00ea30cbed49e3acdbe45c4db4b7d4050f173971e1ade16e616e083928"} Jul 08 13:59:52.479606 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:52.479544 2567 scope.go:117] "RemoveContainer" containerID="631098ef4f90b0028d90a22a1b6d8366f172e8505dedf9adb9c1b3b204645cfe" Jul 08 13:59:52.488300 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:52.488274 2567 scope.go:117] "RemoveContainer" containerID="cd6f6bacd4db92dbd42af1976aa309bd9a741e8e904d8d1e406c8d9dd8cea0a0" Jul 08 13:59:52.496212 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:52.496187 2567 scope.go:117] "RemoveContainer" containerID="631098ef4f90b0028d90a22a1b6d8366f172e8505dedf9adb9c1b3b204645cfe" Jul 08 13:59:52.496720 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:59:52.496540 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"631098ef4f90b0028d90a22a1b6d8366f172e8505dedf9adb9c1b3b204645cfe\": container with ID starting with 631098ef4f90b0028d90a22a1b6d8366f172e8505dedf9adb9c1b3b204645cfe not found: ID does not exist" containerID="631098ef4f90b0028d90a22a1b6d8366f172e8505dedf9adb9c1b3b204645cfe" Jul 08 13:59:52.496720 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:52.496599 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"631098ef4f90b0028d90a22a1b6d8366f172e8505dedf9adb9c1b3b204645cfe"} err="failed to get container status \"631098ef4f90b0028d90a22a1b6d8366f172e8505dedf9adb9c1b3b204645cfe\": rpc error: code = NotFound desc = could not find container \"631098ef4f90b0028d90a22a1b6d8366f172e8505dedf9adb9c1b3b204645cfe\": container with ID starting with 631098ef4f90b0028d90a22a1b6d8366f172e8505dedf9adb9c1b3b204645cfe not found: ID does not exist" Jul 08 13:59:52.496720 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:52.496625 2567 scope.go:117] "RemoveContainer" containerID="cd6f6bacd4db92dbd42af1976aa309bd9a741e8e904d8d1e406c8d9dd8cea0a0" Jul 08 13:59:52.497518 ip-10-0-137-164 kubenswrapper[2567]: E0708 13:59:52.497489 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd6f6bacd4db92dbd42af1976aa309bd9a741e8e904d8d1e406c8d9dd8cea0a0\": container with ID starting with cd6f6bacd4db92dbd42af1976aa309bd9a741e8e904d8d1e406c8d9dd8cea0a0 not found: ID does not exist" containerID="cd6f6bacd4db92dbd42af1976aa309bd9a741e8e904d8d1e406c8d9dd8cea0a0" Jul 08 13:59:52.497596 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:52.497525 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd6f6bacd4db92dbd42af1976aa309bd9a741e8e904d8d1e406c8d9dd8cea0a0"} err="failed to get container status \"cd6f6bacd4db92dbd42af1976aa309bd9a741e8e904d8d1e406c8d9dd8cea0a0\": rpc error: code = NotFound desc = could not find container \"cd6f6bacd4db92dbd42af1976aa309bd9a741e8e904d8d1e406c8d9dd8cea0a0\": container with ID starting with cd6f6bacd4db92dbd42af1976aa309bd9a741e8e904d8d1e406c8d9dd8cea0a0 not found: ID does not exist" Jul 08 13:59:52.499145 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:52.499126 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz"] Jul 08 13:59:52.503392 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:52.503368 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-e225ef-openshift-default-5dbbf599bc-srwsz"] Jul 08 13:59:52.578112 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:52.578061 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-774b77d499-hgmh9_e5d7ed9d-7e90-4e91-9588-ac7b72e29049/copy-manifests/0.log" Jul 08 13:59:52.585713 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:52.585687 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-774b77d499-hgmh9_e5d7ed9d-7e90-4e91-9588-ac7b72e29049/manager/0.log" Jul 08 13:59:53.368281 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:53.368251 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qkqv6_ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb/pull/0.log" Jul 08 13:59:53.374723 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:53.374698 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qkqv6_ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb/extract/0.log" Jul 08 13:59:53.380815 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:53.380790 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qkqv6_ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb/util/0.log" Jul 08 13:59:53.496983 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:53.496956 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzvzk_16c5fe64-76fb-4a59-b11f-93e409225a1f/util/0.log" Jul 08 13:59:53.503335 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:53.503311 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzvzk_16c5fe64-76fb-4a59-b11f-93e409225a1f/pull/0.log" Jul 08 13:59:53.511187 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:53.511170 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzvzk_16c5fe64-76fb-4a59-b11f-93e409225a1f/extract/0.log" Jul 08 13:59:53.622028 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:53.621946 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wpdwr_f2b8d968-c4b7-4000-a1ee-b71b786d76a8/util/0.log" Jul 08 13:59:53.628811 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:53.628787 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wpdwr_f2b8d968-c4b7-4000-a1ee-b71b786d76a8/pull/0.log" Jul 08 13:59:53.635645 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:53.635627 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wpdwr_f2b8d968-c4b7-4000-a1ee-b71b786d76a8/extract/0.log" Jul 08 13:59:53.764567 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:53.764533 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wwwml_67eb1d4c-685e-45a3-84e2-9d02835f795b/util/0.log" Jul 08 13:59:53.777110 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:53.777070 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wwwml_67eb1d4c-685e-45a3-84e2-9d02835f795b/pull/0.log" Jul 08 13:59:53.785408 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:53.785350 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wwwml_67eb1d4c-685e-45a3-84e2-9d02835f795b/extract/0.log" Jul 08 13:59:53.795346 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:53.795321 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f52dd567-378a-4522-a7e9-0facd6925ee6" path="/var/lib/kubelet/pods/f52dd567-378a-4522-a7e9-0facd6925ee6/volumes" Jul 08 13:59:54.632030 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:54.632001 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-hpf2w_ae3f52b5-288d-4736-bfb5-f4999246d155/limitador/0.log" Jul 08 13:59:55.103765 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:55.103727 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678lqnzl_3db34d8c-4001-4369-8dc6-2e6e00fcb4e1/istio-proxy/0.log" Jul 08 13:59:55.226814 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:55.226784 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-sfrfw_15f6cf7d-af37-41ff-a35e-9757e3d29f6b/discovery/0.log" Jul 08 13:59:56.730388 ip-10-0-137-164 kubenswrapper[2567]: I0708 13:59:56.730356 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-cn54g_f738043a-0ae8-4e6f-ab4c-c8acbafa822a/main/0.log" Jul 08 14:00:00.143169 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:00.143131 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725320-2qnt9"] Jul 08 14:00:00.143644 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:00.143441 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f52dd567-378a-4522-a7e9-0facd6925ee6" containerName="istio-proxy" Jul 08 14:00:00.143644 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:00.143453 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="f52dd567-378a-4522-a7e9-0facd6925ee6" containerName="istio-proxy" Jul 08 14:00:00.143644 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:00.143467 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f52dd567-378a-4522-a7e9-0facd6925ee6" containerName="istio-proxy" Jul 08 14:00:00.143644 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:00.143472 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="f52dd567-378a-4522-a7e9-0facd6925ee6" containerName="istio-proxy" Jul 08 14:00:00.143644 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:00.143480 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc7e1b28-587f-42bc-b09e-da10952440eb" containerName="istio-proxy" Jul 08 14:00:00.143644 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:00.143485 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc7e1b28-587f-42bc-b09e-da10952440eb" containerName="istio-proxy" Jul 08 14:00:00.143644 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:00.143541 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="f52dd567-378a-4522-a7e9-0facd6925ee6" containerName="istio-proxy" Jul 08 14:00:00.143644 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:00.143547 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc7e1b28-587f-42bc-b09e-da10952440eb" containerName="istio-proxy" Jul 08 14:00:00.143644 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:00.143553 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc7e1b28-587f-42bc-b09e-da10952440eb" containerName="istio-proxy" Jul 08 14:00:00.146049 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:00.146033 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725320-2qnt9" Jul 08 14:00:00.148930 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:00.148895 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 08 14:00:00.148930 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:00.148911 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 08 14:00:00.149914 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:00.149894 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-hz4c8\"" Jul 08 14:00:00.154383 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:00.154360 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725320-2qnt9"] Jul 08 14:00:00.214708 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:00.214674 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2k7j\" (UniqueName: \"kubernetes.io/projected/37820737-06d5-4ba8-b321-c8a9a9032cf1-kube-api-access-r2k7j\") pod \"maas-api-key-cleanup-29725320-2qnt9\" (UID: \"37820737-06d5-4ba8-b321-c8a9a9032cf1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725320-2qnt9" Jul 08 14:00:00.315988 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:00.315951 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2k7j\" (UniqueName: \"kubernetes.io/projected/37820737-06d5-4ba8-b321-c8a9a9032cf1-kube-api-access-r2k7j\") pod \"maas-api-key-cleanup-29725320-2qnt9\" (UID: \"37820737-06d5-4ba8-b321-c8a9a9032cf1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725320-2qnt9" Jul 08 14:00:00.324846 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:00.324820 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2k7j\" (UniqueName: \"kubernetes.io/projected/37820737-06d5-4ba8-b321-c8a9a9032cf1-kube-api-access-r2k7j\") pod \"maas-api-key-cleanup-29725320-2qnt9\" (UID: \"37820737-06d5-4ba8-b321-c8a9a9032cf1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725320-2qnt9" Jul 08 14:00:00.457236 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:00.457145 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725320-2qnt9" Jul 08 14:00:00.585120 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:00.585095 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725320-2qnt9"] Jul 08 14:00:00.587188 ip-10-0-137-164 kubenswrapper[2567]: W0708 14:00:00.587161 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37820737_06d5_4ba8_b321_c8a9a9032cf1.slice/crio-99843e118a8b1ead2e1d7ecfe1a56325f44e862a705fa0ba1f1fc830a546eab7 WatchSource:0}: Error finding container 99843e118a8b1ead2e1d7ecfe1a56325f44e862a705fa0ba1f1fc830a546eab7: Status 404 returned error can't find the container with id 99843e118a8b1ead2e1d7ecfe1a56325f44e862a705fa0ba1f1fc830a546eab7 Jul 08 14:00:00.589059 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:00.589044 2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 14:00:01.524005 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:01.523974 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725320-2qnt9" event={"ID":"37820737-06d5-4ba8-b321-c8a9a9032cf1","Type":"ContainerStarted","Data":"99843e118a8b1ead2e1d7ecfe1a56325f44e862a705fa0ba1f1fc830a546eab7"} Jul 08 14:00:04.773137 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:04.773107 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-tp2lh_f93cae56-ca8c-4c37-866c-51c79f237448/global-pull-secret-syncer/0.log" Jul 08 14:00:04.902724 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:04.902694 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-g4lnf_7bcaa09d-1aac-4ac4-af53-f79bfc73b690/konnectivity-agent/0.log" Jul 08 14:00:04.982818 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:04.982780 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-164.ec2.internal_abd4fe04a7d63176e943bf70844ade69/haproxy/0.log" Jul 08 14:00:09.309122 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:09.309059 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qkqv6_ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb/extract/0.log" Jul 08 14:00:09.333059 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:09.333029 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qkqv6_ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb/util/0.log" Jul 08 14:00:09.361099 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:09.361054 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qkqv6_ab99e9a3-0ef3-40c2-92dd-ff2daaba12bb/pull/0.log" Jul 08 14:00:09.410750 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:09.410708 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzvzk_16c5fe64-76fb-4a59-b11f-93e409225a1f/extract/0.log" Jul 08 14:00:09.432693 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:09.432666 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzvzk_16c5fe64-76fb-4a59-b11f-93e409225a1f/util/0.log" Jul 08 14:00:09.455444 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:09.455409 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzvzk_16c5fe64-76fb-4a59-b11f-93e409225a1f/pull/0.log" Jul 08 14:00:09.480920 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:09.480894 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wpdwr_f2b8d968-c4b7-4000-a1ee-b71b786d76a8/extract/0.log" Jul 08 14:00:09.525980 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:09.525950 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wpdwr_f2b8d968-c4b7-4000-a1ee-b71b786d76a8/util/0.log" Jul 08 14:00:09.548353 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:09.548328 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73wpdwr_f2b8d968-c4b7-4000-a1ee-b71b786d76a8/pull/0.log" Jul 08 14:00:09.586294 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:09.586193 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wwwml_67eb1d4c-685e-45a3-84e2-9d02835f795b/extract/0.log" Jul 08 14:00:09.606543 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:09.606518 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wwwml_67eb1d4c-685e-45a3-84e2-9d02835f795b/util/0.log" Jul 08 14:00:09.630490 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:09.630445 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wwwml_67eb1d4c-685e-45a3-84e2-9d02835f795b/pull/0.log" Jul 08 14:00:10.407819 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:10.407793 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-hpf2w_ae3f52b5-288d-4736-bfb5-f4999246d155/limitador/0.log" Jul 08 14:00:11.572618 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:11.572582 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725320-2qnt9" event={"ID":"37820737-06d5-4ba8-b321-c8a9a9032cf1","Type":"ContainerStarted","Data":"1a747923e1abffdb3deae4d3326925f716ee73d9ad80135e0abdfe27cda3b507"} Jul 08 14:00:11.589012 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:11.588938 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725320-2qnt9" podStartSLOduration=1.7109436420000002 podStartE2EDuration="11.588921658s" totalImagesPullingTime="9.877978016s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 14:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 14:00:11.588227659 +0000 UTC m=+2220.385168395" watchObservedRunningTime="2026-07-08 14:00:11.588921658 +0000 UTC m=+2220.385862313" Jul 08 14:00:12.195467 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:12.195378 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6dafca81-f493-4b6d-ada8-dd9a6043baed/alertmanager/0.log" Jul 08 14:00:12.221861 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:12.221834 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6dafca81-f493-4b6d-ada8-dd9a6043baed/config-reloader/0.log" Jul 08 14:00:12.243159 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:12.243130 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6dafca81-f493-4b6d-ada8-dd9a6043baed/kube-rbac-proxy-web/0.log" Jul 08 14:00:12.267145 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:12.267120 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6dafca81-f493-4b6d-ada8-dd9a6043baed/kube-rbac-proxy/0.log" Jul 08 14:00:12.288090 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:12.288045 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6dafca81-f493-4b6d-ada8-dd9a6043baed/kube-rbac-proxy-metric/0.log" Jul 08 14:00:12.315756 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:12.315732 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6dafca81-f493-4b6d-ada8-dd9a6043baed/prom-label-proxy/0.log" Jul 08 14:00:12.343854 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:12.343824 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6dafca81-f493-4b6d-ada8-dd9a6043baed/init-config-reloader/0.log" Jul 08 14:00:12.481206 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:12.481182 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7bb96f599b-drpkm_061d2273-5038-4af2-a0e0-8659275d5095/metrics-server/0.log" Jul 08 14:00:12.741096 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:12.740999 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dtkrv_36940947-ce58-4160-b7dd-474b1d58a790/node-exporter/0.log" Jul 08 14:00:12.772717 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:12.772689 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dtkrv_36940947-ce58-4160-b7dd-474b1d58a790/kube-rbac-proxy/0.log" Jul 08 14:00:12.800180 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:12.800157 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dtkrv_36940947-ce58-4160-b7dd-474b1d58a790/init-textfile/0.log" Jul 08 14:00:12.827067 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:12.827040 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-pm5fs_1fa12140-dbf1-49c1-afd1-29f85f64d4f5/kube-rbac-proxy-main/0.log" Jul 08 14:00:12.847677 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:12.847647 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-pm5fs_1fa12140-dbf1-49c1-afd1-29f85f64d4f5/kube-rbac-proxy-self/0.log" Jul 08 14:00:12.873838 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:12.873805 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-pm5fs_1fa12140-dbf1-49c1-afd1-29f85f64d4f5/openshift-state-metrics/0.log" Jul 08 14:00:15.657806 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:15.657770 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-tljhk/perf-node-gather-daemonset-6bt2q"] Jul 08 14:00:15.658200 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:15.658098 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc7e1b28-587f-42bc-b09e-da10952440eb" containerName="istio-proxy" Jul 08 14:00:15.658200 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:15.658109 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc7e1b28-587f-42bc-b09e-da10952440eb" containerName="istio-proxy" Jul 08 14:00:15.658200 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:15.658189 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="f52dd567-378a-4522-a7e9-0facd6925ee6" containerName="istio-proxy" Jul 08 14:00:15.660903 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:15.660883 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tljhk/perf-node-gather-daemonset-6bt2q" Jul 08 14:00:15.663509 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:15.663492 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tljhk\"/\"kube-root-ca.crt\"" Jul 08 14:00:15.663686 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:15.663671 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tljhk\"/\"openshift-service-ca.crt\"" Jul 08 14:00:15.663791 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:15.663774 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-tljhk\"/\"default-dockercfg-kt4cr\"" Jul 08 14:00:15.669296 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:15.669276 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tljhk/perf-node-gather-daemonset-6bt2q"] Jul 08 14:00:15.851196 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:15.851161 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/58cc9028-b3e4-4d25-8195-8c1901f59e80-lib-modules\") pod \"perf-node-gather-daemonset-6bt2q\" (UID: \"58cc9028-b3e4-4d25-8195-8c1901f59e80\") " pod="openshift-must-gather-tljhk/perf-node-gather-daemonset-6bt2q" Jul 08 14:00:15.851388 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:15.851216 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/58cc9028-b3e4-4d25-8195-8c1901f59e80-proc\") pod \"perf-node-gather-daemonset-6bt2q\" (UID: \"58cc9028-b3e4-4d25-8195-8c1901f59e80\") " pod="openshift-must-gather-tljhk/perf-node-gather-daemonset-6bt2q" Jul 08 14:00:15.851388 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:15.851246 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/58cc9028-b3e4-4d25-8195-8c1901f59e80-podres\") pod \"perf-node-gather-daemonset-6bt2q\" (UID: \"58cc9028-b3e4-4d25-8195-8c1901f59e80\") " pod="openshift-must-gather-tljhk/perf-node-gather-daemonset-6bt2q" Jul 08 14:00:15.851388 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:15.851281 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/58cc9028-b3e4-4d25-8195-8c1901f59e80-sys\") pod \"perf-node-gather-daemonset-6bt2q\" (UID: \"58cc9028-b3e4-4d25-8195-8c1901f59e80\") " pod="openshift-must-gather-tljhk/perf-node-gather-daemonset-6bt2q" Jul 08 14:00:15.851388 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:15.851300 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxx7n\" (UniqueName: \"kubernetes.io/projected/58cc9028-b3e4-4d25-8195-8c1901f59e80-kube-api-access-fxx7n\") pod \"perf-node-gather-daemonset-6bt2q\" (UID: \"58cc9028-b3e4-4d25-8195-8c1901f59e80\") " pod="openshift-must-gather-tljhk/perf-node-gather-daemonset-6bt2q" Jul 08 14:00:15.951937 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:15.951843 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/58cc9028-b3e4-4d25-8195-8c1901f59e80-lib-modules\") pod \"perf-node-gather-daemonset-6bt2q\" (UID: \"58cc9028-b3e4-4d25-8195-8c1901f59e80\") " pod="openshift-must-gather-tljhk/perf-node-gather-daemonset-6bt2q" Jul 08 14:00:15.951937 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:15.951897 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/58cc9028-b3e4-4d25-8195-8c1901f59e80-proc\") pod \"perf-node-gather-daemonset-6bt2q\" (UID: \"58cc9028-b3e4-4d25-8195-8c1901f59e80\") " pod="openshift-must-gather-tljhk/perf-node-gather-daemonset-6bt2q" Jul 08 14:00:15.951937 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:15.951922 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/58cc9028-b3e4-4d25-8195-8c1901f59e80-podres\") pod \"perf-node-gather-daemonset-6bt2q\" (UID: \"58cc9028-b3e4-4d25-8195-8c1901f59e80\") " pod="openshift-must-gather-tljhk/perf-node-gather-daemonset-6bt2q" Jul 08 14:00:15.952221 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:15.951945 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/58cc9028-b3e4-4d25-8195-8c1901f59e80-sys\") pod \"perf-node-gather-daemonset-6bt2q\" (UID: \"58cc9028-b3e4-4d25-8195-8c1901f59e80\") " pod="openshift-must-gather-tljhk/perf-node-gather-daemonset-6bt2q" Jul 08 14:00:15.952221 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:15.951965 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxx7n\" (UniqueName: \"kubernetes.io/projected/58cc9028-b3e4-4d25-8195-8c1901f59e80-kube-api-access-fxx7n\") pod \"perf-node-gather-daemonset-6bt2q\" (UID: \"58cc9028-b3e4-4d25-8195-8c1901f59e80\") " pod="openshift-must-gather-tljhk/perf-node-gather-daemonset-6bt2q" Jul 08 14:00:15.952221 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:15.952052 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/58cc9028-b3e4-4d25-8195-8c1901f59e80-sys\") pod \"perf-node-gather-daemonset-6bt2q\" (UID: \"58cc9028-b3e4-4d25-8195-8c1901f59e80\") " pod="openshift-must-gather-tljhk/perf-node-gather-daemonset-6bt2q" Jul 08 14:00:15.952221 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:15.952096 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/58cc9028-b3e4-4d25-8195-8c1901f59e80-proc\") pod \"perf-node-gather-daemonset-6bt2q\" (UID: \"58cc9028-b3e4-4d25-8195-8c1901f59e80\") " pod="openshift-must-gather-tljhk/perf-node-gather-daemonset-6bt2q" Jul 08 14:00:15.952221 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:15.952101 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/58cc9028-b3e4-4d25-8195-8c1901f59e80-podres\") pod \"perf-node-gather-daemonset-6bt2q\" (UID: \"58cc9028-b3e4-4d25-8195-8c1901f59e80\") " pod="openshift-must-gather-tljhk/perf-node-gather-daemonset-6bt2q" Jul 08 14:00:15.952221 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:15.952050 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/58cc9028-b3e4-4d25-8195-8c1901f59e80-lib-modules\") pod \"perf-node-gather-daemonset-6bt2q\" (UID: \"58cc9028-b3e4-4d25-8195-8c1901f59e80\") " pod="openshift-must-gather-tljhk/perf-node-gather-daemonset-6bt2q" Jul 08 14:00:15.960983 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:15.960966 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxx7n\" (UniqueName: \"kubernetes.io/projected/58cc9028-b3e4-4d25-8195-8c1901f59e80-kube-api-access-fxx7n\") pod \"perf-node-gather-daemonset-6bt2q\" (UID: \"58cc9028-b3e4-4d25-8195-8c1901f59e80\") " pod="openshift-must-gather-tljhk/perf-node-gather-daemonset-6bt2q" Jul 08 14:00:15.971432 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:15.971415 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tljhk/perf-node-gather-daemonset-6bt2q" Jul 08 14:00:16.097689 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:16.097664 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tljhk/perf-node-gather-daemonset-6bt2q"] Jul 08 14:00:16.099189 ip-10-0-137-164 kubenswrapper[2567]: W0708 14:00:16.099164 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod58cc9028_b3e4_4d25_8195_8c1901f59e80.slice/crio-a32ea9d04da21079b202976173bb25a3297c2adc2b740b015ad42bb1ade4b79c WatchSource:0}: Error finding container a32ea9d04da21079b202976173bb25a3297c2adc2b740b015ad42bb1ade4b79c: Status 404 returned error can't find the container with id a32ea9d04da21079b202976173bb25a3297c2adc2b740b015ad42bb1ade4b79c Jul 08 14:00:16.598571 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:16.598535 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tljhk/perf-node-gather-daemonset-6bt2q" event={"ID":"58cc9028-b3e4-4d25-8195-8c1901f59e80","Type":"ContainerStarted","Data":"9653d54be0f0b6a14bf2f986ff36e8f97f9de757e5d990b024ae369a745d93eb"} Jul 08 14:00:16.598789 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:16.598592 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tljhk/perf-node-gather-daemonset-6bt2q" event={"ID":"58cc9028-b3e4-4d25-8195-8c1901f59e80","Type":"ContainerStarted","Data":"a32ea9d04da21079b202976173bb25a3297c2adc2b740b015ad42bb1ade4b79c"} Jul 08 14:00:16.598789 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:16.598718 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-tljhk/perf-node-gather-daemonset-6bt2q" Jul 08 14:00:16.615848 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:16.615802 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-tljhk/perf-node-gather-daemonset-6bt2q" podStartSLOduration=1.615788418 podStartE2EDuration="1.615788418s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 14:00:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 14:00:16.614624441 +0000 UTC m=+2225.411565100" watchObservedRunningTime="2026-07-08 14:00:16.615788418 +0000 UTC m=+2225.412729126" Jul 08 14:00:17.529721 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:17.529688 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gv29f_faf0a62e-56a2-449c-9aed-2859b1bfe9bc/dns/0.log" Jul 08 14:00:17.547860 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:17.547834 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gv29f_faf0a62e-56a2-449c-9aed-2859b1bfe9bc/kube-rbac-proxy/0.log" Jul 08 14:00:17.614736 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:17.614699 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zs2fw_d492fe73-0e0b-4b21-8ea1-2ab7cfcefe74/dns-node-resolver/0.log" Jul 08 14:00:18.161811 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:18.161767 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-67596d5cf6-6tq7v_a5eefa2d-4b2e-4d78-9eff-086d7ec4ca2f/registry/0.log" Jul 08 14:00:18.204694 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:18.204662 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-pn22q_67e86263-6fde-4f3a-a775-9e29eb8cacef/node-ca/0.log" Jul 08 14:00:19.116462 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:19.116425 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678lqnzl_3db34d8c-4001-4369-8dc6-2e6e00fcb4e1/istio-proxy/0.log" Jul 08 14:00:19.287695 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:19.287662 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-sfrfw_15f6cf7d-af37-41ff-a35e-9757e3d29f6b/discovery/0.log" Jul 08 14:00:20.307163 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:20.307129 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wgz5x_1111874c-8d49-43f3-a84a-fa214f432735/serve-healthcheck-canary/0.log" Jul 08 14:00:20.952906 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:20.952878 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-v8x9r_f54b4d08-6fd7-40fc-b9d9-5776291ff5ad/kube-rbac-proxy/0.log" Jul 08 14:00:20.972054 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:20.972032 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-v8x9r_f54b4d08-6fd7-40fc-b9d9-5776291ff5ad/exporter/0.log" Jul 08 14:00:20.993812 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:20.993783 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-v8x9r_f54b4d08-6fd7-40fc-b9d9-5776291ff5ad/extractor/0.log" Jul 08 14:00:22.613325 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:22.613301 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-tljhk/perf-node-gather-daemonset-6bt2q" Jul 08 14:00:23.812149 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:23.812120 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-774b77d499-hgmh9_e5d7ed9d-7e90-4e91-9588-ac7b72e29049/manager/0.log" Jul 08 14:00:23.833395 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:23.833367 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-774b77d499-hgmh9_e5d7ed9d-7e90-4e91-9588-ac7b72e29049/copy-manifests/0.log" Jul 08 14:00:25.212866 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:25.212832 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7fcdcfc8c4-mrdm8_a2776123-af46-4009-a214-9aeffc5677b6/manager/0.log" Jul 08 14:00:25.494249 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:25.494217 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678lqnzl_3db34d8c-4001-4369-8dc6-2e6e00fcb4e1/istio-proxy/0.log" Jul 08 14:00:25.657512 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:25.657484 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-sfrfw_15f6cf7d-af37-41ff-a35e-9757e3d29f6b/discovery/0.log" Jul 08 14:00:27.005324 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:27.005296 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678lqnzl_3db34d8c-4001-4369-8dc6-2e6e00fcb4e1/istio-proxy/0.log" Jul 08 14:00:27.181881 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:27.181848 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-sfrfw_15f6cf7d-af37-41ff-a35e-9757e3d29f6b/discovery/0.log" Jul 08 14:00:31.888509 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:31.888388 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-85xxk_db3380e5-69cf-4c03-b995-6de00f6b269f/kube-multus/0.log" Jul 08 14:00:32.078482 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:32.078454 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qmwvz_ea6986a3-b0b4-4704-a47a-c991094c04f4/kube-multus-additional-cni-plugins/0.log" Jul 08 14:00:32.097426 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:32.097394 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qmwvz_ea6986a3-b0b4-4704-a47a-c991094c04f4/egress-router-binary-copy/0.log" Jul 08 14:00:32.115339 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:32.115317 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qmwvz_ea6986a3-b0b4-4704-a47a-c991094c04f4/cni-plugins/0.log" Jul 08 14:00:32.135309 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:32.135282 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qmwvz_ea6986a3-b0b4-4704-a47a-c991094c04f4/bond-cni-plugin/0.log" Jul 08 14:00:32.154612 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:32.154533 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qmwvz_ea6986a3-b0b4-4704-a47a-c991094c04f4/routeoverride-cni/0.log" Jul 08 14:00:32.175681 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:32.175650 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qmwvz_ea6986a3-b0b4-4704-a47a-c991094c04f4/whereabouts-cni-bincopy/0.log" Jul 08 14:00:32.194325 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:32.194295 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qmwvz_ea6986a3-b0b4-4704-a47a-c991094c04f4/whereabouts-cni/0.log" Jul 08 14:00:32.367597 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:32.367550 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-cnv6f_a3381953-6131-477f-be6d-92ef7dc333a7/network-metrics-daemon/0.log" Jul 08 14:00:32.386191 ip-10-0-137-164 kubenswrapper[2567]: I0708 14:00:32.386152 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-cnv6f_a3381953-6131-477f-be6d-92ef7dc333a7/kube-rbac-proxy/0.log"