Jul 08 19:03:53.560644 ip-10-0-137-244 systemd[1]: Starting Kubernetes Kubelet... Jul 08 19:03:53.895919 ip-10-0-137-244 kubenswrapper[2583]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 19:03:53.895919 ip-10-0-137-244 kubenswrapper[2583]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 08 19:03:53.895919 ip-10-0-137-244 kubenswrapper[2583]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 19:03:53.895919 ip-10-0-137-244 kubenswrapper[2583]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 08 19:03:53.895919 ip-10-0-137-244 kubenswrapper[2583]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 19:03:53.896841 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.896755 2583 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 08 19:03:53.901248 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901224 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 19:03:53.901248 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901244 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 19:03:53.901248 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901248 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 19:03:53.901248 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901252 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 19:03:53.901446 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901257 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 19:03:53.901446 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901261 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 19:03:53.901446 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901265 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 19:03:53.901446 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901268 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 19:03:53.901446 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901271 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 19:03:53.901446 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901274 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 19:03:53.901446 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901276 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 19:03:53.901446 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901279 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 08 19:03:53.901446 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901281 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 19:03:53.901446 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901283 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 19:03:53.901446 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901286 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 19:03:53.901446 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901288 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 19:03:53.901446 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901291 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 19:03:53.901446 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901293 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 19:03:53.901446 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901296 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 19:03:53.901446 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901298 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 19:03:53.901446 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901318 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 19:03:53.901446 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901323 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 19:03:53.901446 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901327 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 19:03:53.901446 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901329 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 19:03:53.901957 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901331 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 19:03:53.901957 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901334 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 19:03:53.901957 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901336 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 19:03:53.901957 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901339 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 19:03:53.901957 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901341 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 19:03:53.901957 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901344 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 19:03:53.901957 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901346 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 19:03:53.901957 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901349 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 19:03:53.901957 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901352 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 19:03:53.901957 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901354 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 19:03:53.901957 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901357 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 19:03:53.901957 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901359 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 19:03:53.901957 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901362 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 19:03:53.901957 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901365 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 19:03:53.901957 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901368 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 19:03:53.901957 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901371 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 19:03:53.901957 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901374 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 19:03:53.901957 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901376 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 19:03:53.901957 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901379 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 19:03:53.902431 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901381 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 19:03:53.902431 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901384 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 19:03:53.902431 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901386 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 19:03:53.902431 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901388 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 19:03:53.902431 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901391 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 19:03:53.902431 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901394 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 19:03:53.902431 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901396 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 19:03:53.902431 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901399 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 19:03:53.902431 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901402 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 19:03:53.902431 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901404 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 19:03:53.902431 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901407 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 19:03:53.902431 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901409 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 19:03:53.902431 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901411 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 19:03:53.902431 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901413 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 19:03:53.902431 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901416 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 19:03:53.902431 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901418 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 19:03:53.902431 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901421 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 19:03:53.902431 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901425 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 19:03:53.902431 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901429 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 19:03:53.902898 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901432 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 19:03:53.902898 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901435 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 19:03:53.902898 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901438 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 19:03:53.902898 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901441 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 19:03:53.902898 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901443 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 19:03:53.902898 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901446 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 19:03:53.902898 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901449 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 19:03:53.902898 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901453 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 19:03:53.902898 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901456 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 19:03:53.902898 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901459 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 19:03:53.902898 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901463 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 19:03:53.902898 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901466 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 19:03:53.902898 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901469 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 19:03:53.902898 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901472 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 19:03:53.902898 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901476 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 19:03:53.902898 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901480 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 19:03:53.902898 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901483 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 19:03:53.902898 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901485 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 19:03:53.902898 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901488 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 19:03:53.903366 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901491 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 19:03:53.903366 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901493 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 19:03:53.903366 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901496 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 19:03:53.903366 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901498 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 19:03:53.903366 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901501 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 19:03:53.903366 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901503 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 19:03:53.903366 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901505 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 19:03:53.903366 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901508 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 19:03:53.903366 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901510 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 19:03:53.903366 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901513 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 19:03:53.903366 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901515 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 19:03:53.903366 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901517 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 19:03:53.903366 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901520 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 19:03:53.903366 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901522 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 19:03:53.903366 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901525 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 19:03:53.903366 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901527 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 19:03:53.903366 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901530 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 19:03:53.903366 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.901532 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 19:03:53.903366 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902226 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 19:03:53.903366 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902233 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 19:03:53.903834 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902236 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 19:03:53.903834 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902239 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 19:03:53.903834 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902242 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 19:03:53.903834 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902244 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 19:03:53.903834 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902247 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 19:03:53.903834 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902250 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 19:03:53.903834 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902253 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 19:03:53.903834 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902256 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 19:03:53.903834 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902258 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 19:03:53.903834 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902261 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 19:03:53.903834 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902263 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 19:03:53.903834 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902266 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 19:03:53.903834 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902268 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 19:03:53.903834 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902271 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 19:03:53.903834 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902274 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 19:03:53.903834 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902277 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 19:03:53.903834 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902279 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 19:03:53.903834 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902282 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 19:03:53.903834 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902284 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 19:03:53.903834 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902287 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 19:03:53.904361 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902289 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 19:03:53.904361 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902292 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 19:03:53.904361 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902294 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 19:03:53.904361 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902296 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 19:03:53.904361 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902299 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 19:03:53.904361 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902322 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 08 19:03:53.904361 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902325 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 19:03:53.904361 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902328 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 19:03:53.904361 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902330 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 19:03:53.904361 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902333 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 19:03:53.904361 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902335 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 19:03:53.904361 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902339 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 19:03:53.904361 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902342 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 19:03:53.904361 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902344 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 19:03:53.904361 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902347 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 19:03:53.904361 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902349 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 19:03:53.904361 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902352 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 19:03:53.904361 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902355 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 19:03:53.904361 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902357 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 19:03:53.904361 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902360 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 19:03:53.904845 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902362 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 19:03:53.904845 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902365 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 19:03:53.904845 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902367 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 19:03:53.904845 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902369 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 19:03:53.904845 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902372 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 19:03:53.904845 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902374 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 19:03:53.904845 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902377 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 19:03:53.904845 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902379 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 19:03:53.904845 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902382 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 19:03:53.904845 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902385 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 19:03:53.904845 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902387 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 19:03:53.904845 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902391 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 19:03:53.904845 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902393 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 19:03:53.904845 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902395 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 19:03:53.904845 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902398 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 19:03:53.904845 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902400 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 19:03:53.904845 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902403 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 19:03:53.904845 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902405 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 19:03:53.904845 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902408 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 19:03:53.904845 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902410 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 19:03:53.905336 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902413 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 19:03:53.905336 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902415 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 19:03:53.905336 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902418 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 19:03:53.905336 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902420 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 19:03:53.905336 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902423 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 19:03:53.905336 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902427 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 19:03:53.905336 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902431 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 19:03:53.905336 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902434 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 19:03:53.905336 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902436 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 19:03:53.905336 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902439 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 19:03:53.905336 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902442 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 19:03:53.905336 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902444 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 19:03:53.905336 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902447 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 19:03:53.905336 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902450 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 19:03:53.905336 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902454 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 19:03:53.905336 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902456 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 19:03:53.905336 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902459 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 19:03:53.905336 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902462 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 19:03:53.905336 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902465 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 19:03:53.905802 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902467 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 19:03:53.905802 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902470 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 19:03:53.905802 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902473 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 19:03:53.905802 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902475 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 19:03:53.905802 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902478 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 19:03:53.905802 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902480 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 19:03:53.905802 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902482 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 19:03:53.905802 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902485 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 19:03:53.905802 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902487 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 19:03:53.905802 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902490 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 19:03:53.905802 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902494 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 19:03:53.905802 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902497 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 19:03:53.905802 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902500 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 19:03:53.905802 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902503 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 19:03:53.905802 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902506 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 19:03:53.905802 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902509 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 19:03:53.905802 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902512 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 19:03:53.905802 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902515 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 19:03:53.905802 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902898 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 19:03:53.906265 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902904 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 19:03:53.906265 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902907 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 19:03:53.906265 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902910 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 19:03:53.906265 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902912 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 19:03:53.906265 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902915 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 19:03:53.906265 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902917 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 19:03:53.906265 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902920 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 19:03:53.906265 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902923 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 19:03:53.906265 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902925 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 19:03:53.906265 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902928 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 19:03:53.906265 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902930 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 19:03:53.906265 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902932 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 19:03:53.906265 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902935 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 19:03:53.906265 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902937 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 19:03:53.906265 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902940 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 19:03:53.906265 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902944 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 19:03:53.906265 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902948 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 19:03:53.906265 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902951 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 19:03:53.906265 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902954 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 19:03:53.906265 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902956 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 19:03:53.906767 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902959 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 19:03:53.906767 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902961 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 19:03:53.906767 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902964 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 19:03:53.906767 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902969 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 19:03:53.906767 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902971 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 19:03:53.906767 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902974 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 19:03:53.906767 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902978 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 19:03:53.906767 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902981 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 19:03:53.906767 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902983 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 19:03:53.906767 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902986 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 19:03:53.906767 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902989 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 19:03:53.906767 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902992 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 19:03:53.906767 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902995 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 19:03:53.906767 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.902997 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 19:03:53.906767 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903000 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 19:03:53.906767 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903002 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 19:03:53.906767 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903005 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 19:03:53.906767 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903007 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 19:03:53.906767 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903010 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 19:03:53.907332 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903013 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 19:03:53.907332 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903015 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 19:03:53.907332 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903018 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 19:03:53.907332 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903020 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 19:03:53.907332 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903023 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 19:03:53.907332 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903025 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 19:03:53.907332 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903027 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 19:03:53.907332 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903030 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 19:03:53.907332 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903032 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 19:03:53.907332 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903035 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 19:03:53.907332 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903039 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 19:03:53.907332 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903043 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 19:03:53.907332 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903046 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 19:03:53.907332 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903048 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 19:03:53.907332 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903051 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 19:03:53.907332 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903054 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 19:03:53.907332 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903059 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 19:03:53.907332 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903061 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 19:03:53.907332 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903064 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 19:03:53.907795 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903067 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 19:03:53.907795 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903069 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 19:03:53.907795 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903071 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 19:03:53.907795 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903074 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 19:03:53.907795 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903077 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 19:03:53.907795 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903079 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 19:03:53.907795 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903082 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 19:03:53.907795 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903085 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 19:03:53.907795 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903087 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 19:03:53.907795 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903090 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 19:03:53.907795 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903092 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 19:03:53.907795 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903095 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 19:03:53.907795 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903097 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 19:03:53.907795 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903100 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 19:03:53.907795 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903102 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 19:03:53.907795 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903106 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 19:03:53.907795 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903109 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 19:03:53.907795 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903111 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 19:03:53.907795 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903113 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 19:03:53.907795 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903116 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 19:03:53.908268 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903118 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 19:03:53.908268 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903121 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 19:03:53.908268 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903123 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 19:03:53.908268 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903126 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 19:03:53.908268 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903128 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 19:03:53.908268 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903131 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 19:03:53.908268 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903133 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 08 19:03:53.908268 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903135 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 19:03:53.908268 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903138 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 19:03:53.908268 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903143 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 19:03:53.908268 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903145 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 19:03:53.908268 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903148 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 19:03:53.908268 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903150 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 19:03:53.908268 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903152 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 19:03:53.908268 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903155 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 19:03:53.908268 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903158 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 19:03:53.908268 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903160 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 19:03:53.908268 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903162 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 19:03:53.908268 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903165 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 19:03:53.908268 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903168 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 19:03:53.908757 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903240 2583 flags.go:64] FLAG: --address="0.0.0.0" Jul 08 19:03:53.908757 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903247 2583 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 08 19:03:53.908757 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903253 2583 flags.go:64] FLAG: --anonymous-auth="true" Jul 08 19:03:53.908757 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903258 2583 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 08 19:03:53.908757 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903262 2583 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 08 19:03:53.908757 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903265 2583 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 08 19:03:53.908757 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903269 2583 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 08 19:03:53.908757 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903275 2583 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 08 19:03:53.908757 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903278 2583 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 08 19:03:53.908757 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903281 2583 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 08 19:03:53.908757 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903284 2583 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 08 19:03:53.908757 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903288 2583 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 08 19:03:53.908757 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903290 2583 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 08 19:03:53.908757 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903293 2583 flags.go:64] FLAG: --cgroup-root="" Jul 08 19:03:53.908757 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903296 2583 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 08 19:03:53.908757 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903298 2583 flags.go:64] FLAG: --client-ca-file="" Jul 08 19:03:53.908757 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903319 2583 flags.go:64] FLAG: --cloud-provider="external" Jul 08 19:03:53.908757 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903322 2583 flags.go:64] FLAG: --cluster-dns="[]" Jul 08 19:03:53.908757 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903361 2583 flags.go:64] FLAG: --cluster-domain="" Jul 08 19:03:53.908757 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903364 2583 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 08 19:03:53.908757 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903368 2583 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 08 19:03:53.908757 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903372 2583 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 08 19:03:53.908757 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903376 2583 flags.go:64] FLAG: --container-log-max-files="5" Jul 08 19:03:53.908757 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903382 2583 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 08 19:03:53.909331 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903385 2583 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 08 19:03:53.909331 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903388 2583 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 08 19:03:53.909331 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903392 2583 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 08 19:03:53.909331 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903395 2583 flags.go:64] FLAG: --contention-profiling="false" Jul 08 19:03:53.909331 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903398 2583 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 08 19:03:53.909331 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903401 2583 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 08 19:03:53.909331 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903404 2583 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 08 19:03:53.909331 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903407 2583 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 08 19:03:53.909331 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903411 2583 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 08 19:03:53.909331 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903414 2583 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 08 19:03:53.909331 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903416 2583 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 08 19:03:53.909331 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903419 2583 flags.go:64] FLAG: --enable-load-reader="false" Jul 08 19:03:53.909331 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903423 2583 flags.go:64] FLAG: --enable-server="true" Jul 08 19:03:53.909331 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903426 2583 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 08 19:03:53.909331 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903431 2583 flags.go:64] FLAG: --event-burst="100" Jul 08 19:03:53.909331 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903436 2583 flags.go:64] FLAG: --event-qps="50" Jul 08 19:03:53.909331 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903439 2583 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 08 19:03:53.909331 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903442 2583 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 08 19:03:53.909331 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903445 2583 flags.go:64] FLAG: --eviction-hard="" Jul 08 19:03:53.909331 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903448 2583 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 08 19:03:53.909331 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903451 2583 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 08 19:03:53.909331 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903454 2583 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 08 19:03:53.909331 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903456 2583 flags.go:64] FLAG: --eviction-soft="" Jul 08 19:03:53.909331 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903459 2583 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 08 19:03:53.909331 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903462 2583 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 08 19:03:53.909971 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903465 2583 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 08 19:03:53.909971 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903468 2583 flags.go:64] FLAG: --experimental-mounter-path="" Jul 08 19:03:53.909971 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903471 2583 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 08 19:03:53.909971 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903474 2583 flags.go:64] FLAG: --fail-swap-on="true" Jul 08 19:03:53.909971 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903477 2583 flags.go:64] FLAG: --feature-gates="" Jul 08 19:03:53.909971 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903481 2583 flags.go:64] FLAG: --file-check-frequency="20s" Jul 08 19:03:53.909971 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903484 2583 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 08 19:03:53.909971 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903487 2583 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 08 19:03:53.909971 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903490 2583 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 08 19:03:53.909971 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903492 2583 flags.go:64] FLAG: --healthz-port="10248" Jul 08 19:03:53.909971 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903495 2583 flags.go:64] FLAG: --help="false" Jul 08 19:03:53.909971 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903498 2583 flags.go:64] FLAG: --hostname-override="ip-10-0-137-244.ec2.internal" Jul 08 19:03:53.909971 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903501 2583 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 08 19:03:53.909971 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903504 2583 flags.go:64] FLAG: --http-check-frequency="20s" Jul 08 19:03:53.909971 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903507 2583 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 08 19:03:53.909971 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903510 2583 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 08 19:03:53.909971 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903513 2583 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 08 19:03:53.909971 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903516 2583 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 08 19:03:53.909971 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903518 2583 flags.go:64] FLAG: --image-service-endpoint="" Jul 08 19:03:53.909971 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903521 2583 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 08 19:03:53.909971 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903524 2583 flags.go:64] FLAG: --kube-api-burst="100" Jul 08 19:03:53.909971 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903527 2583 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 08 19:03:53.909971 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903531 2583 flags.go:64] FLAG: --kube-api-qps="50" Jul 08 19:03:53.909971 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903533 2583 flags.go:64] FLAG: --kube-reserved="" Jul 08 19:03:53.910569 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903536 2583 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 08 19:03:53.910569 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903539 2583 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 08 19:03:53.910569 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903542 2583 flags.go:64] FLAG: --kubelet-cgroups="" Jul 08 19:03:53.910569 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903544 2583 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 08 19:03:53.910569 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903547 2583 flags.go:64] FLAG: --lock-file="" Jul 08 19:03:53.910569 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903550 2583 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 08 19:03:53.910569 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903552 2583 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 08 19:03:53.910569 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903555 2583 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 08 19:03:53.910569 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903560 2583 flags.go:64] FLAG: --log-json-split-stream="false" Jul 08 19:03:53.910569 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903569 2583 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 08 19:03:53.910569 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903572 2583 flags.go:64] FLAG: --log-text-split-stream="false" Jul 08 19:03:53.910569 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903575 2583 flags.go:64] FLAG: --logging-format="text" Jul 08 19:03:53.910569 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903579 2583 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 08 19:03:53.910569 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903582 2583 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 08 19:03:53.910569 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903585 2583 flags.go:64] FLAG: --manifest-url="" Jul 08 19:03:53.910569 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903587 2583 flags.go:64] FLAG: --manifest-url-header="" Jul 08 19:03:53.910569 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903597 2583 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 08 19:03:53.910569 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903600 2583 flags.go:64] FLAG: --max-open-files="1000000" Jul 08 19:03:53.910569 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903604 2583 flags.go:64] FLAG: --max-pods="110" Jul 08 19:03:53.910569 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903607 2583 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 08 19:03:53.910569 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903610 2583 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 08 19:03:53.910569 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903612 2583 flags.go:64] FLAG: --memory-manager-policy="None" Jul 08 19:03:53.910569 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903615 2583 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 08 19:03:53.910569 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903618 2583 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 08 19:03:53.910569 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903620 2583 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 08 19:03:53.911173 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903623 2583 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 08 19:03:53.911173 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903630 2583 flags.go:64] FLAG: --node-status-max-images="50" Jul 08 19:03:53.911173 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903634 2583 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 08 19:03:53.911173 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903637 2583 flags.go:64] FLAG: --oom-score-adj="-999" Jul 08 19:03:53.911173 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903640 2583 flags.go:64] FLAG: --pod-cidr="" Jul 08 19:03:53.911173 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903644 2583 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 08 19:03:53.911173 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903648 2583 flags.go:64] FLAG: --pod-manifest-path="" Jul 08 19:03:53.911173 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903651 2583 flags.go:64] FLAG: --pod-max-pids="-1" Jul 08 19:03:53.911173 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903654 2583 flags.go:64] FLAG: --pods-per-core="0" Jul 08 19:03:53.911173 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903657 2583 flags.go:64] FLAG: --port="10250" Jul 08 19:03:53.911173 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903660 2583 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 08 19:03:53.911173 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903663 2583 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-09580796c38bcbf6f" Jul 08 19:03:53.911173 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903666 2583 flags.go:64] FLAG: --qos-reserved="" Jul 08 19:03:53.911173 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903668 2583 flags.go:64] FLAG: --read-only-port="10255" Jul 08 19:03:53.911173 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903671 2583 flags.go:64] FLAG: --register-node="true" Jul 08 19:03:53.911173 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903674 2583 flags.go:64] FLAG: --register-with-taints="" Jul 08 19:03:53.911173 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903682 2583 flags.go:64] FLAG: --registry-burst="10" Jul 08 19:03:53.911173 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903685 2583 flags.go:64] FLAG: --registry-qps="5" Jul 08 19:03:53.911173 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903688 2583 flags.go:64] FLAG: --reserved-cpus="" Jul 08 19:03:53.911173 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903690 2583 flags.go:64] FLAG: --reserved-memory="" Jul 08 19:03:53.911173 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903695 2583 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 08 19:03:53.911173 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903698 2583 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 08 19:03:53.911173 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903701 2583 flags.go:64] FLAG: --rotate-certificates="false" Jul 08 19:03:53.911173 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903704 2583 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 08 19:03:53.911173 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903706 2583 flags.go:64] FLAG: --runonce="false" Jul 08 19:03:53.911794 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903709 2583 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 08 19:03:53.911794 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903712 2583 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 08 19:03:53.911794 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903715 2583 flags.go:64] FLAG: --seccomp-default="false" Jul 08 19:03:53.911794 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903718 2583 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 08 19:03:53.911794 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903720 2583 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 08 19:03:53.911794 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903723 2583 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 08 19:03:53.911794 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903726 2583 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 08 19:03:53.911794 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903728 2583 flags.go:64] FLAG: --storage-driver-password="root" Jul 08 19:03:53.911794 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903732 2583 flags.go:64] FLAG: --storage-driver-secure="false" Jul 08 19:03:53.911794 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903735 2583 flags.go:64] FLAG: --storage-driver-table="stats" Jul 08 19:03:53.911794 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903738 2583 flags.go:64] FLAG: --storage-driver-user="root" Jul 08 19:03:53.911794 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903740 2583 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 08 19:03:53.911794 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903744 2583 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 08 19:03:53.911794 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903747 2583 flags.go:64] FLAG: --system-cgroups="" Jul 08 19:03:53.911794 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903750 2583 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 08 19:03:53.911794 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903755 2583 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 08 19:03:53.911794 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903757 2583 flags.go:64] FLAG: --tls-cert-file="" Jul 08 19:03:53.911794 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903764 2583 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 08 19:03:53.911794 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903768 2583 flags.go:64] FLAG: --tls-min-version="" Jul 08 19:03:53.911794 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903771 2583 flags.go:64] FLAG: --tls-private-key-file="" Jul 08 19:03:53.911794 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903773 2583 flags.go:64] FLAG: --topology-manager-policy="none" Jul 08 19:03:53.911794 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903776 2583 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 08 19:03:53.911794 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903779 2583 flags.go:64] FLAG: --topology-manager-scope="container" Jul 08 19:03:53.911794 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903782 2583 flags.go:64] FLAG: --v="2" Jul 08 19:03:53.911794 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903786 2583 flags.go:64] FLAG: --version="false" Jul 08 19:03:53.912409 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903791 2583 flags.go:64] FLAG: --vmodule="" Jul 08 19:03:53.912409 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903794 2583 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 08 19:03:53.912409 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.903798 2583 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 08 19:03:53.912409 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903910 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 19:03:53.912409 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903914 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 19:03:53.912409 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903917 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 19:03:53.912409 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903920 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 19:03:53.912409 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903923 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 19:03:53.912409 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903926 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 19:03:53.912409 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903928 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 19:03:53.912409 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903931 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 19:03:53.912409 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903933 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 19:03:53.912409 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903936 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 19:03:53.912409 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903938 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 19:03:53.912409 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903941 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 19:03:53.912409 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903943 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 19:03:53.912409 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903946 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 19:03:53.912409 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903948 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 19:03:53.912409 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903950 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 19:03:53.912409 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903954 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 19:03:53.912890 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903956 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 19:03:53.912890 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903959 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 19:03:53.912890 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903961 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 19:03:53.912890 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903964 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 19:03:53.912890 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903966 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 19:03:53.912890 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903968 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 19:03:53.912890 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903971 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 19:03:53.912890 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903973 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 19:03:53.912890 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903976 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 08 19:03:53.912890 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903978 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 19:03:53.912890 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903981 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 19:03:53.912890 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903983 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 19:03:53.912890 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903986 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 19:03:53.912890 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903989 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 19:03:53.912890 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903992 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 19:03:53.912890 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903995 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 19:03:53.912890 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.903997 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 19:03:53.912890 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904000 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 19:03:53.912890 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904002 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 19:03:53.912890 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904005 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 19:03:53.913520 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904007 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 19:03:53.913520 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904011 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 19:03:53.913520 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904015 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 19:03:53.913520 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904018 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 19:03:53.913520 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904020 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 19:03:53.913520 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904024 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 19:03:53.913520 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904026 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 19:03:53.913520 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904029 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 19:03:53.913520 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904032 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 19:03:53.913520 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904034 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 19:03:53.913520 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904037 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 19:03:53.913520 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904040 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 19:03:53.913520 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904043 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 19:03:53.913520 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904045 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 19:03:53.913520 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904048 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 19:03:53.913520 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904050 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 19:03:53.913520 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904052 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 19:03:53.913520 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904055 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 19:03:53.913520 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904057 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 19:03:53.913972 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904060 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 19:03:53.913972 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904062 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 19:03:53.913972 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904064 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 19:03:53.913972 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904067 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 19:03:53.913972 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904070 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 19:03:53.913972 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904074 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 19:03:53.913972 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904076 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 19:03:53.913972 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904081 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 19:03:53.913972 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904083 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 19:03:53.913972 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904086 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 19:03:53.913972 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904089 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 19:03:53.913972 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904091 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 19:03:53.913972 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904093 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 19:03:53.913972 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904096 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 19:03:53.913972 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904098 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 19:03:53.913972 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904101 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 19:03:53.913972 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904103 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 19:03:53.913972 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904106 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 19:03:53.913972 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904108 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 19:03:53.913972 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904111 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 19:03:53.914474 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904113 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 19:03:53.914474 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904116 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 19:03:53.914474 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904118 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 19:03:53.914474 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904121 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 19:03:53.914474 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904125 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 19:03:53.914474 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904127 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 19:03:53.914474 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904130 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 19:03:53.914474 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904132 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 19:03:53.914474 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904135 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 19:03:53.914474 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904137 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 19:03:53.914474 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904140 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 19:03:53.914474 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904142 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 19:03:53.914474 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904145 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 19:03:53.914474 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904147 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 19:03:53.914474 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904149 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 19:03:53.914474 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904152 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 19:03:53.914474 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904155 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 19:03:53.914474 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904158 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 19:03:53.914474 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904161 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 19:03:53.914948 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904165 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 19:03:53.914948 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904168 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 19:03:53.914948 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904171 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 19:03:53.914948 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.904173 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 19:03:53.914948 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.904179 2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 19:03:53.914948 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.911113 2583 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 08 19:03:53.914948 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.911130 2583 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 08 19:03:53.914948 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.911767 2583 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 08 19:03:53.914948 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.911775 2583 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 08 19:03:53.914948 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911832 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 19:03:53.914948 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911837 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 19:03:53.914948 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911840 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 19:03:53.914948 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911843 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 19:03:53.914948 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911846 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 19:03:53.914948 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911849 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 19:03:53.915316 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911851 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 19:03:53.915316 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911855 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 19:03:53.915316 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911858 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 19:03:53.915316 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911860 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 19:03:53.915316 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911863 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 19:03:53.915316 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911865 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 19:03:53.915316 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911867 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 19:03:53.915316 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911870 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 19:03:53.915316 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911872 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 19:03:53.915316 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911875 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 19:03:53.915316 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911877 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 19:03:53.915316 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911880 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 19:03:53.915316 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911882 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 19:03:53.915316 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911885 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 19:03:53.915316 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911887 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 19:03:53.915316 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911890 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 19:03:53.915316 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911893 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 19:03:53.915316 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911895 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 19:03:53.915316 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911898 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 19:03:53.915316 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911900 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 19:03:53.915791 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911903 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 19:03:53.915791 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911905 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 19:03:53.915791 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911908 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 19:03:53.915791 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911911 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 19:03:53.915791 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911914 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 19:03:53.915791 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911916 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 19:03:53.915791 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911919 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 19:03:53.915791 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911921 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 19:03:53.915791 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911924 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 19:03:53.915791 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911926 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 19:03:53.915791 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911928 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 19:03:53.915791 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911931 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 19:03:53.915791 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911933 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 19:03:53.915791 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911936 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 19:03:53.915791 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911938 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 19:03:53.915791 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911940 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 19:03:53.915791 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911943 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 19:03:53.915791 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911945 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 19:03:53.915791 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911947 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 19:03:53.915791 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911950 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 19:03:53.916282 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911953 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 19:03:53.916282 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911955 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 19:03:53.916282 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911958 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 19:03:53.916282 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911960 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 19:03:53.916282 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911963 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 19:03:53.916282 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911966 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 19:03:53.916282 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911970 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 19:03:53.916282 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911974 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 19:03:53.916282 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911977 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 19:03:53.916282 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911979 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 19:03:53.916282 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911982 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 19:03:53.916282 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911984 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 19:03:53.916282 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911986 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 19:03:53.916282 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911989 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 19:03:53.916282 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911992 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 19:03:53.916282 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911994 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 19:03:53.916282 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.911998 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 19:03:53.916282 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912000 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 19:03:53.916282 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912004 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 19:03:53.916753 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912007 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 19:03:53.916753 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912010 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 19:03:53.916753 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912013 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 19:03:53.916753 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912016 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 19:03:53.916753 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912018 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 19:03:53.916753 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912020 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 08 19:03:53.916753 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912023 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 19:03:53.916753 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912025 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 19:03:53.916753 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912028 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 19:03:53.916753 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912030 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 19:03:53.916753 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912033 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 19:03:53.916753 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912035 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 19:03:53.916753 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912039 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 19:03:53.916753 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912042 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 19:03:53.916753 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912045 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 19:03:53.916753 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912048 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 19:03:53.916753 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912050 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 19:03:53.916753 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912053 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 19:03:53.916753 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912055 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 19:03:53.916753 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912058 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 19:03:53.917254 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912061 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 19:03:53.917254 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912064 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 19:03:53.917254 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912066 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 19:03:53.917254 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912069 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 19:03:53.917254 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912071 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 19:03:53.917254 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912074 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 19:03:53.917254 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912076 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 19:03:53.917254 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912079 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 19:03:53.917254 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912081 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 19:03:53.917254 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912084 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 19:03:53.917254 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912087 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 19:03:53.917254 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912089 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 19:03:53.917254 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912091 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 19:03:53.917254 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912093 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 19:03:53.917254 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.912098 2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 19:03:53.917691 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912212 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 19:03:53.917691 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912216 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 19:03:53.917691 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912219 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 19:03:53.917691 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912221 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 19:03:53.917691 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912224 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 19:03:53.917691 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912226 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 19:03:53.917691 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912228 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 19:03:53.917691 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912231 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 19:03:53.917691 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912233 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 19:03:53.917691 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912236 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 19:03:53.917691 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912238 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 19:03:53.917691 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912240 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 19:03:53.917691 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912243 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 19:03:53.917691 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912245 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 19:03:53.917691 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912248 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 19:03:53.917691 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912251 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 19:03:53.917691 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912253 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 19:03:53.917691 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912256 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 19:03:53.917691 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912258 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 19:03:53.917691 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912260 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 19:03:53.918160 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912263 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 19:03:53.918160 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912265 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 19:03:53.918160 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912268 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 19:03:53.918160 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912270 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 19:03:53.918160 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912272 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 19:03:53.918160 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912275 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 19:03:53.918160 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912277 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 19:03:53.918160 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912279 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 19:03:53.918160 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912283 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 19:03:53.918160 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912285 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 19:03:53.918160 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912288 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 19:03:53.918160 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912290 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 19:03:53.918160 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912292 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 19:03:53.918160 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912295 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 19:03:53.918160 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912297 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 19:03:53.918160 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912315 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 19:03:53.918160 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912319 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 19:03:53.918160 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912323 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 19:03:53.918160 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912326 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 19:03:53.918638 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912328 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 19:03:53.918638 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912331 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 19:03:53.918638 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912333 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 19:03:53.918638 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912335 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 19:03:53.918638 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912338 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 19:03:53.918638 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912340 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 19:03:53.918638 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912342 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 19:03:53.918638 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912345 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 19:03:53.918638 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912348 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 19:03:53.918638 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912351 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 19:03:53.918638 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912353 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 19:03:53.918638 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912356 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 19:03:53.918638 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912358 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 19:03:53.918638 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912361 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 19:03:53.918638 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912363 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 19:03:53.918638 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912367 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 19:03:53.918638 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912370 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 19:03:53.918638 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912374 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 19:03:53.918638 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912377 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 19:03:53.919112 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912380 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 19:03:53.919112 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912382 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 19:03:53.919112 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912385 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 19:03:53.919112 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912389 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 08 19:03:53.919112 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912392 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 19:03:53.919112 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912394 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 19:03:53.919112 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912397 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 19:03:53.919112 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912399 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 19:03:53.919112 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912401 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 19:03:53.919112 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912404 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 19:03:53.919112 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912406 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 19:03:53.919112 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912409 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 19:03:53.919112 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912411 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 19:03:53.919112 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912414 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 19:03:53.919112 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912416 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 19:03:53.919112 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912420 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 19:03:53.919112 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912423 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 19:03:53.919112 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912426 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 19:03:53.919112 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912429 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 19:03:53.919112 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912432 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 19:03:53.919607 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912435 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 19:03:53.919607 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912437 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 19:03:53.919607 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912440 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 19:03:53.919607 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912443 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 19:03:53.919607 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912445 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 19:03:53.919607 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912448 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 19:03:53.919607 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912450 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 19:03:53.919607 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912453 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 19:03:53.919607 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912455 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 19:03:53.919607 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912458 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 19:03:53.919607 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912460 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 19:03:53.919607 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912462 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 19:03:53.919607 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912465 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 19:03:53.919607 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912467 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 19:03:53.919607 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912469 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 19:03:53.919607 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912472 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 19:03:53.919607 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912475 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 19:03:53.919607 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912477 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 19:03:53.919607 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912480 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 19:03:53.919607 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912482 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 19:03:53.920098 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:53.912485 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 19:03:53.920098 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.912489 2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 19:03:53.920098 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.912619 2583 server.go:962] "Client rotation is on, will bootstrap in background" Jul 08 19:03:53.920098 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.914638 2583 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 08 19:03:53.920098 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.915841 2583 server.go:1019] "Starting client certificate rotation" Jul 08 19:03:53.920098 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.915936 2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 19:03:53.920098 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.915978 2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 19:03:53.920098 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.918640 2583 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 19:03:53.924360 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.924335 2583 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 19:03:53.932081 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.932062 2583 log.go:25] "Validated CRI v1 runtime API" Jul 08 19:03:53.934163 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.934144 2583 log.go:25] "Validated CRI v1 image API" Jul 08 19:03:53.935020 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.934992 2583 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 08 19:03:53.937163 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.937140 2583 fs.go:135] Filesystem UUIDs: map[08a73f80-da1d-48cd-b5c9-c3958ee25224:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 da12ad51-1f25-40e9-b337-5918ede0b859:/dev/nvme0n1p3] Jul 08 19:03:53.937242 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.937161 2583 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 08 19:03:53.942242 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.942115 2583 manager.go:216] Machine: {Timestamp:2026-07-08 19:03:53.940849387 +0000 UTC m=+0.277064691 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102366 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec20e91c73e545ea7a6a4fa3df201c87 SystemUUID:ec20e91c-73e5-45ea-7a6a-4fa3df201c87 BootID:1395b698-b428-4122-b768-6af876df13e9 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:c3:6a:1e:4c:cb Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:c3:6a:1e:4c:cb Speed:0 Mtu:9001} {Name:ovs-system MacAddress:2a:cd:03:9e:31:4d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 08 19:03:53.942242 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.942228 2583 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 08 19:03:53.942430 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.942358 2583 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 08 19:03:53.945066 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.945036 2583 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 08 19:03:53.945226 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.945070 2583 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-244.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 08 19:03:53.946249 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.946238 2583 topology_manager.go:138] "Creating topology manager with none policy" Jul 08 19:03:53.946283 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.946252 2583 container_manager_linux.go:309] "Creating device plugin manager" Jul 08 19:03:53.946283 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.946259 2583 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 19:03:53.946427 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.946410 2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 19:03:53.946895 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.946883 2583 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 19:03:53.948145 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.948134 2583 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 08 19:03:53.952059 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.952046 2583 state_mem.go:36] "Initialized new in-memory state store" Jul 08 19:03:53.952190 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.952181 2583 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 08 19:03:53.952228 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.952219 2583 kubelet.go:486] "Attempting to sync node with API server" Jul 08 19:03:53.952258 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.952236 2583 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 08 19:03:53.952258 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.952255 2583 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 08 19:03:53.952344 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.952334 2583 kubelet.go:398] "Adding apiserver pod source" Jul 08 19:03:53.952377 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.952351 2583 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 08 19:03:53.952958 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.952934 2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 19:03:53.953013 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.952994 2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 19:03:53.955209 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.955178 2583 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 08 19:03:53.957049 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.957035 2583 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 08 19:03:53.957097 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.957067 2583 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 08 19:03:53.958155 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.958141 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 08 19:03:53.958202 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.958165 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 08 19:03:53.958202 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.958175 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 08 19:03:53.958202 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.958184 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 08 19:03:53.958202 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.958197 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 08 19:03:53.958330 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.958205 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 08 19:03:53.958330 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.958217 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 08 19:03:53.958330 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.958225 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 08 19:03:53.958330 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.958235 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 08 19:03:53.958330 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.958245 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 08 19:03:53.958330 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.958258 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 08 19:03:53.958330 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.958269 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 08 19:03:53.958894 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.958881 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 08 19:03:53.958925 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.958896 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 08 19:03:53.962335 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.962319 2583 server.go:1268] "Started kubelet" Jul 08 19:03:53.962458 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.962430 2583 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 08 19:03:53.962944 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.962893 2583 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 08 19:03:53.963391 ip-10-0-137-244 systemd[1]: Started Kubernetes Kubelet. Jul 08 19:03:53.964940 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.964891 2583 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 08 19:03:53.965627 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.965601 2583 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-244.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 08 19:03:53.965824 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:03:53.965797 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-244.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 08 19:03:53.966156 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.966136 2583 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 08 19:03:53.966245 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:03:53.966195 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 08 19:03:53.970085 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:03:53.968123 2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-244.ec2.internal.18c065d8baeec8cd default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-244.ec2.internal,UID:ip-10-0-137-244.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-244.ec2.internal,},FirstTimestamp:2026-07-08 19:03:53.962268877 +0000 UTC m=+0.298484180,LastTimestamp:2026-07-08 19:03:53.962268877 +0000 UTC m=+0.298484180,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-244.ec2.internal,}" Jul 08 19:03:53.970262 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.970244 2583 server.go:310] "Adding debug handlers to kubelet server" Jul 08 19:03:53.970360 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:03:53.970323 2583 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 08 19:03:53.974198 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.974181 2583 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 08 19:03:53.974272 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.974200 2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 08 19:03:53.974344 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:03:53.974291 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-244.ec2.internal\" not found" Jul 08 19:03:53.974879 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.974834 2583 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 08 19:03:53.974879 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.974846 2583 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 08 19:03:53.974879 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.974873 2583 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 08 19:03:53.975062 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.975047 2583 reconstruct.go:97] "Volume reconstruction finished" Jul 08 19:03:53.975062 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.975060 2583 reconciler.go:29] "Reconciler: start to sync state" Jul 08 19:03:53.975533 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.975516 2583 factory.go:153] Registering CRI-O factory Jul 08 19:03:53.975597 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.975538 2583 factory.go:223] Registration of the crio container factory successfully Jul 08 19:03:53.975597 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.975594 2583 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 08 19:03:53.975701 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.975603 2583 factory.go:55] Registering systemd factory Jul 08 19:03:53.975701 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.975612 2583 factory.go:223] Registration of the systemd container factory successfully Jul 08 19:03:53.975701 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.975635 2583 factory.go:103] Registering Raw factory Jul 08 19:03:53.975701 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.975649 2583 manager.go:1196] Started watching for new ooms in manager Jul 08 19:03:53.976029 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.976015 2583 manager.go:349] Starting recovery of all containers Jul 08 19:03:53.977295 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:03:53.977267 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 08 19:03:53.977411 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:03:53.977372 2583 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-137-244.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 08 19:03:53.986056 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.985901 2583 manager.go:354] Recovery completed Jul 08 19:03:53.986560 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.986538 2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rsf6h" Jul 08 19:03:53.990190 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.990177 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 19:03:53.991959 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.991943 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-244.ec2.internal" event="NodeHasSufficientMemory" Jul 08 19:03:53.992038 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.991987 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-244.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 19:03:53.992038 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.991999 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-244.ec2.internal" event="NodeHasSufficientPID" Jul 08 19:03:53.993015 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.993002 2583 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 08 19:03:53.993015 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.993008 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 19:03:53.993015 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.993014 2583 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 08 19:03:53.993159 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.993031 2583 state_mem.go:36] "Initialized new in-memory state store" Jul 08 19:03:53.993934 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.993920 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-244.ec2.internal" event="NodeHasSufficientMemory" Jul 08 19:03:53.993994 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.993950 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-244.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 19:03:53.993994 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.993963 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-244.ec2.internal" event="NodeHasSufficientPID" Jul 08 19:03:53.995194 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.995181 2583 policy_none.go:49] "None policy: Start" Jul 08 19:03:53.995247 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.995198 2583 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 08 19:03:53.995247 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.995208 2583 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 08 19:03:53.996047 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.996033 2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rsf6h" Jul 08 19:03:53.997338 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:53.997327 2583 policy_none.go:47] "Start" Jul 08 19:03:54.030917 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:54.030891 2583 manager.go:337] "Starting Device Plugin manager" Jul 08 19:03:54.031041 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:03:54.030930 2583 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 08 19:03:54.031041 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:54.030942 2583 server.go:87] "Starting device plugin registration server" Jul 08 19:03:54.031237 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:54.031223 2583 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 08 19:03:54.031298 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:54.031239 2583 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 08 19:03:54.031362 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:54.031330 2583 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 08 19:03:54.031427 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:54.031415 2583 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 08 19:03:54.031470 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:54.031427 2583 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 08 19:03:54.031966 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:03:54.031945 2583 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 08 19:03:54.032054 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:03:54.031989 2583 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-244.ec2.internal\" not found" Jul 08 19:03:54.109453 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:54.109419 2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 08 19:03:54.111637 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:54.110780 2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 08 19:03:54.111637 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:54.110834 2583 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 08 19:03:54.111637 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:54.110857 2583 kubelet.go:2448] "Starting kubelet main sync loop" Jul 08 19:03:54.111637 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:03:54.110889 2583 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 08 19:03:54.113334 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:54.113293 2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 19:03:54.131388 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:54.131364 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 19:03:54.132227 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:54.132209 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-244.ec2.internal" event="NodeHasSufficientMemory" Jul 08 19:03:54.132329 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:54.132240 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-244.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 19:03:54.132329 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:54.132251 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-244.ec2.internal" event="NodeHasSufficientPID" Jul 08 19:03:54.132329 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:54.132279 2583 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-244.ec2.internal" Jul 08 19:03:54.141352 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:54.141335 2583 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-244.ec2.internal" Jul 08 19:03:54.141453 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:03:54.141363 2583 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-244.ec2.internal\": node \"ip-10-0-137-244.ec2.internal\" not found" Jul 08 19:03:54.153497 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:03:54.153444 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-244.ec2.internal\" not found" Jul 08 19:03:54.211989 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:54.211944 2583 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-244.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-244.ec2.internal"] Jul 08 19:03:54.212140 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:54.212032 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 19:03:54.213592 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:54.213578 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-244.ec2.internal" event="NodeHasSufficientMemory" Jul 08 19:03:54.213674 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:54.213605 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-244.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 19:03:54.213674 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:54.213615 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-244.ec2.internal" event="NodeHasSufficientPID" Jul 08 19:03:54.215998 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:54.215984 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 19:03:54.216161 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:54.216148 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-244.ec2.internal" Jul 08 19:03:54.216206 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:54.216177 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 19:03:54.216763 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:54.216747 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-244.ec2.internal" event="NodeHasSufficientMemory" Jul 08 19:03:54.216850 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:54.216770 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-244.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 19:03:54.216850 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:54.216782 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-244.ec2.internal" event="NodeHasSufficientPID" Jul 08 19:03:54.216850 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:54.216816 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-244.ec2.internal" event="NodeHasSufficientMemory" Jul 08 19:03:54.216850 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:54.216843 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-244.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 19:03:54.217026 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:54.216857 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-244.ec2.internal" event="NodeHasSufficientPID" Jul 08 19:03:54.219124 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:54.219108 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-244.ec2.internal" Jul 08 19:03:54.219202 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:54.219140 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 19:03:54.219868 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:54.219851 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-244.ec2.internal" event="NodeHasSufficientMemory" Jul 08 19:03:54.219952 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:54.219879 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-244.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 19:03:54.219952 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:54.219888 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-244.ec2.internal" event="NodeHasSufficientPID" Jul 08 19:03:54.241006 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:03:54.240978 2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-244.ec2.internal\" not found" node="ip-10-0-137-244.ec2.internal" Jul 08 19:03:54.245611 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:03:54.245592 2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-244.ec2.internal\" not found" node="ip-10-0-137-244.ec2.internal" Jul 08 19:03:54.253737 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:03:54.253719 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-244.ec2.internal\" not found" Jul 08 19:03:54.276629 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:54.276605 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2bde17ed4dbbb226aeb25d822491cdfc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-244.ec2.internal\" (UID: \"2bde17ed4dbbb226aeb25d822491cdfc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-244.ec2.internal" Jul 08 19:03:54.276735 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:54.276632 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/09c7821b3b1a8a9c6dd9709ff325da43-config\") pod \"kube-apiserver-proxy-ip-10-0-137-244.ec2.internal\" (UID: \"09c7821b3b1a8a9c6dd9709ff325da43\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-244.ec2.internal" Jul 08 19:03:54.276735 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:54.276652 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2bde17ed4dbbb226aeb25d822491cdfc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-244.ec2.internal\" (UID: \"2bde17ed4dbbb226aeb25d822491cdfc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-244.ec2.internal" Jul 08 19:03:54.354459 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:03:54.354422 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-244.ec2.internal\" not found" Jul 08 19:03:54.377004 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:54.376974 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2bde17ed4dbbb226aeb25d822491cdfc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-244.ec2.internal\" (UID: \"2bde17ed4dbbb226aeb25d822491cdfc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-244.ec2.internal" Jul 08 19:03:54.377122 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:54.377023 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2bde17ed4dbbb226aeb25d822491cdfc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-244.ec2.internal\" (UID: \"2bde17ed4dbbb226aeb25d822491cdfc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-244.ec2.internal" Jul 08 19:03:54.377122 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:54.377069 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/09c7821b3b1a8a9c6dd9709ff325da43-config\") pod \"kube-apiserver-proxy-ip-10-0-137-244.ec2.internal\" (UID: \"09c7821b3b1a8a9c6dd9709ff325da43\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-244.ec2.internal" Jul 08 19:03:54.377122 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:54.377082 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2bde17ed4dbbb226aeb25d822491cdfc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-244.ec2.internal\" (UID: \"2bde17ed4dbbb226aeb25d822491cdfc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-244.ec2.internal" Jul 08 19:03:54.377122 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:54.377111 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/09c7821b3b1a8a9c6dd9709ff325da43-config\") pod \"kube-apiserver-proxy-ip-10-0-137-244.ec2.internal\" (UID: \"09c7821b3b1a8a9c6dd9709ff325da43\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-244.ec2.internal" Jul 08 19:03:54.377265 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:54.377136 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2bde17ed4dbbb226aeb25d822491cdfc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-244.ec2.internal\" (UID: \"2bde17ed4dbbb226aeb25d822491cdfc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-244.ec2.internal" Jul 08 19:03:54.455399 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:03:54.455337 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-244.ec2.internal\" not found" Jul 08 19:03:54.543910 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:54.543864 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-244.ec2.internal" Jul 08 19:03:54.548832 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:54.548719 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-244.ec2.internal" Jul 08 19:03:54.556276 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:03:54.556252 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-244.ec2.internal\" not found" Jul 08 19:03:54.656813 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:03:54.656777 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-244.ec2.internal\" not found" Jul 08 19:03:54.757372 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:03:54.757257 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-244.ec2.internal\" not found" Jul 08 19:03:55.914900 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:03:55.914866 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-244.ec2.internal\" not found" Jul 08 19:03:55.972334 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:55.972301 2583 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 08 19:03:55.972967 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:55.972493 2583 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 19:03:56.015829 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:03:56.015794 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-244.ec2.internal\" not found" Jul 08 19:03:56.058678 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:56.058636 2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-07 18:58:55 +0000 UTC" deadline="2028-01-20 00:01:57.330073109 +0000 UTC" Jul 08 19:03:56.058678 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:56.058670 2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13444h58m1.271406619s" Jul 08 19:03:56.116009 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:03:56.115967 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-244.ec2.internal\" not found" Jul 08 19:03:56.118777 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:56.118759 2583 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 19:03:56.208659 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:56.208620 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09c7821b3b1a8a9c6dd9709ff325da43.slice/crio-c73598be0c850e1c3ded56b3b69d9296de3c489b5dd19c268d930db748e040d7 WatchSource:0}: Error finding container c73598be0c850e1c3ded56b3b69d9296de3c489b5dd19c268d930db748e040d7: Status 404 returned error can't find the container with id c73598be0c850e1c3ded56b3b69d9296de3c489b5dd19c268d930db748e040d7 Jul 08 19:03:56.209058 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:56.209035 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bde17ed4dbbb226aeb25d822491cdfc.slice/crio-4afbbafcd6bbf362f965c90278fed12f6db26122f5842ff96958d070405c62c8 WatchSource:0}: Error finding container 4afbbafcd6bbf362f965c90278fed12f6db26122f5842ff96958d070405c62c8: Status 404 returned error can't find the container with id 4afbbafcd6bbf362f965c90278fed12f6db26122f5842ff96958d070405c62c8 Jul 08 19:03:56.214036 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:56.214019 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 19:03:56.216788 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:03:56.216770 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-244.ec2.internal\" not found" Jul 08 19:03:56.267568 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:56.267538 2583 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 19:03:56.317903 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:03:56.317869 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-244.ec2.internal\" not found" Jul 08 19:03:56.353972 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:56.353917 2583 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 19:03:56.432216 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:56.432188 2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-244.ec2.internal" Jul 08 19:03:56.449385 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:56.449364 2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 19:03:56.450388 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:56.450374 2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-244.ec2.internal" Jul 08 19:03:56.463154 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:56.463130 2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 19:03:56.831482 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:56.831388 2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 19:03:57.011715 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.011684 2583 apiserver.go:52] "Watching apiserver" Jul 08 19:03:57.025864 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.025829 2583 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 08 19:03:57.026301 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.026261 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-qwcgr","openshift-dns/node-resolver-zjgc5","openshift-multus/network-metrics-daemon-hs752","openshift-ovn-kubernetes/ovnkube-node-2t8gf","kube-system/konnectivity-agent-n8prt","kube-system/kube-apiserver-proxy-ip-10-0-137-244.ec2.internal","openshift-image-registry/node-ca-jw4sj","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-244.ec2.internal","openshift-multus/multus-6kmdh","openshift-multus/multus-additional-cni-plugins-mpz48","openshift-network-diagnostics/network-check-target-z4hnv","openshift-network-operator/iptables-alerter-mq9pr","kube-system/global-pull-secret-syncer-qx98w","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xh2z8"] Jul 08 19:03:57.028983 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.028964 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jw4sj" Jul 08 19:03:57.030952 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.030924 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qx98w" Jul 08 19:03:57.031061 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:03:57.030990 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qx98w" podUID="9a902ad1-6161-490d-afb2-006966897633" Jul 08 19:03:57.031928 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.031896 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-pvkzb\"" Jul 08 19:03:57.032275 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.032226 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 08 19:03:57.032372 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.032273 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 08 19:03:57.032519 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.032503 2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 08 19:03:57.032519 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.032510 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 08 19:03:57.033609 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.033590 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hs752" Jul 08 19:03:57.033700 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:03:57.033656 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hs752" podUID="6701701d-6ce5-4028-a9ac-9fabc3925522" Jul 08 19:03:57.036368 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.036350 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.039153 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.039135 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 08 19:03:57.039300 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.039286 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 08 19:03:57.039369 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.039321 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-75qjl\"" Jul 08 19:03:57.039424 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.039406 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 08 19:03:57.039643 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.039626 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 08 19:03:57.039738 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.039720 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-n8prt" Jul 08 19:03:57.040275 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.040232 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 08 19:03:57.040275 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.040270 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 08 19:03:57.041754 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.041739 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xh2z8" Jul 08 19:03:57.042323 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.042301 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/bb8c1f0b-7584-4d24-aa13-c73cad10576c-serviceca\") pod \"node-ca-jw4sj\" (UID: \"bb8c1f0b-7584-4d24-aa13-c73cad10576c\") " pod="openshift-image-registry/node-ca-jw4sj" Jul 08 19:03:57.042407 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.042337 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9khr\" (UniqueName: \"kubernetes.io/projected/bb8c1f0b-7584-4d24-aa13-c73cad10576c-kube-api-access-t9khr\") pod \"node-ca-jw4sj\" (UID: \"bb8c1f0b-7584-4d24-aa13-c73cad10576c\") " pod="openshift-image-registry/node-ca-jw4sj" Jul 08 19:03:57.042407 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.042369 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9a902ad1-6161-490d-afb2-006966897633-kubelet-config\") pod \"global-pull-secret-syncer-qx98w\" (UID: \"9a902ad1-6161-490d-afb2-006966897633\") " pod="kube-system/global-pull-secret-syncer-qx98w" Jul 08 19:03:57.042407 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.042392 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9a902ad1-6161-490d-afb2-006966897633-original-pull-secret\") pod \"global-pull-secret-syncer-qx98w\" (UID: \"9a902ad1-6161-490d-afb2-006966897633\") " pod="kube-system/global-pull-secret-syncer-qx98w" Jul 08 19:03:57.042584 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.042449 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6701701d-6ce5-4028-a9ac-9fabc3925522-metrics-certs\") pod \"network-metrics-daemon-hs752\" (UID: \"6701701d-6ce5-4028-a9ac-9fabc3925522\") " pod="openshift-multus/network-metrics-daemon-hs752" Jul 08 19:03:57.042584 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.042513 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnwcq\" (UniqueName: \"kubernetes.io/projected/6701701d-6ce5-4028-a9ac-9fabc3925522-kube-api-access-vnwcq\") pod \"network-metrics-daemon-hs752\" (UID: \"6701701d-6ce5-4028-a9ac-9fabc3925522\") " pod="openshift-multus/network-metrics-daemon-hs752" Jul 08 19:03:57.042584 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.042567 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bb8c1f0b-7584-4d24-aa13-c73cad10576c-host\") pod \"node-ca-jw4sj\" (UID: \"bb8c1f0b-7584-4d24-aa13-c73cad10576c\") " pod="openshift-image-registry/node-ca-jw4sj" Jul 08 19:03:57.042733 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.042592 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9a902ad1-6161-490d-afb2-006966897633-dbus\") pod \"global-pull-secret-syncer-qx98w\" (UID: \"9a902ad1-6161-490d-afb2-006966897633\") " pod="kube-system/global-pull-secret-syncer-qx98w" Jul 08 19:03:57.042733 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.042715 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-crn2m\"" Jul 08 19:03:57.043027 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.043006 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 08 19:03:57.043213 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.043198 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 08 19:03:57.044395 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.044375 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-mvht8\"" Jul 08 19:03:57.044698 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.044680 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6kmdh" Jul 08 19:03:57.044856 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.044833 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 08 19:03:57.045189 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.045169 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 08 19:03:57.045363 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.045347 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 08 19:03:57.047157 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.047137 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qwcgr" Jul 08 19:03:57.047645 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.047632 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 08 19:03:57.047770 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.047750 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-2nhd7\"" Jul 08 19:03:57.048177 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.048161 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 08 19:03:57.048246 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.048180 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 08 19:03:57.048477 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.048462 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 08 19:03:57.048774 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.048755 2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 19:03:57.049973 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.049954 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 08 19:03:57.050223 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.050200 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 08 19:03:57.050288 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.050274 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-z6zlx\"" Jul 08 19:03:57.050367 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.050206 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zjgc5" Jul 08 19:03:57.051816 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.051780 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mpz48" Jul 08 19:03:57.052708 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.052687 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 08 19:03:57.053319 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.053114 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 08 19:03:57.053850 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.053821 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-kxdnd\"" Jul 08 19:03:57.054195 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.054179 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 08 19:03:57.056865 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.055954 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 08 19:03:57.056865 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.056286 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-jpmpp\"" Jul 08 19:03:57.056865 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.056506 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4hnv" Jul 08 19:03:57.056865 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:03:57.056629 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z4hnv" podUID="7f193dea-aa4a-47c5-8cb2-8f34029b1355" Jul 08 19:03:57.060263 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.058381 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mq9pr" Jul 08 19:03:57.061200 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.061183 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 08 19:03:57.061569 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.061551 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 08 19:03:57.061650 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.061583 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-pdxs6\"" Jul 08 19:03:57.061844 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.061821 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 08 19:03:57.079560 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.079048 2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-dtgkj" Jul 08 19:03:57.089175 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.089142 2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-dtgkj" Jul 08 19:03:57.134027 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.134004 2583 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 08 19:03:57.143733 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.143690 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3b8b33cc-c6e1-4b41-9fe3-4ca56a434892-etc-sysctl-d\") pod \"tuned-qwcgr\" (UID: \"3b8b33cc-c6e1-4b41-9fe3-4ca56a434892\") " pod="openshift-cluster-node-tuning-operator/tuned-qwcgr" Jul 08 19:03:57.143906 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.143740 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3b8b33cc-c6e1-4b41-9fe3-4ca56a434892-run\") pod \"tuned-qwcgr\" (UID: \"3b8b33cc-c6e1-4b41-9fe3-4ca56a434892\") " pod="openshift-cluster-node-tuning-operator/tuned-qwcgr" Jul 08 19:03:57.143906 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.143768 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fb6af782-3262-467f-95e4-31d549573cd6-os-release\") pod \"multus-additional-cni-plugins-mpz48\" (UID: \"fb6af782-3262-467f-95e4-31d549573cd6\") " pod="openshift-multus/multus-additional-cni-plugins-mpz48" Jul 08 19:03:57.143906 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.143800 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9a902ad1-6161-490d-afb2-006966897633-original-pull-secret\") pod \"global-pull-secret-syncer-qx98w\" (UID: \"9a902ad1-6161-490d-afb2-006966897633\") " pod="kube-system/global-pull-secret-syncer-qx98w" Jul 08 19:03:57.143906 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.143827 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6701701d-6ce5-4028-a9ac-9fabc3925522-metrics-certs\") pod \"network-metrics-daemon-hs752\" (UID: \"6701701d-6ce5-4028-a9ac-9fabc3925522\") " pod="openshift-multus/network-metrics-daemon-hs752" Jul 08 19:03:57.143906 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.143853 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-node-log\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.143906 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.143884 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/001f0989-80a3-4b9d-8a57-9f41846bef7b-multus-cni-dir\") pod \"multus-6kmdh\" (UID: \"001f0989-80a3-4b9d-8a57-9f41846bef7b\") " pod="openshift-multus/multus-6kmdh" Jul 08 19:03:57.144210 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.143909 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fb6af782-3262-467f-95e4-31d549573cd6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mpz48\" (UID: \"fb6af782-3262-467f-95e4-31d549573cd6\") " pod="openshift-multus/multus-additional-cni-plugins-mpz48" Jul 08 19:03:57.144210 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.143958 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqh5r\" (UniqueName: \"kubernetes.io/projected/7f193dea-aa4a-47c5-8cb2-8f34029b1355-kube-api-access-wqh5r\") pod \"network-check-target-z4hnv\" (UID: \"7f193dea-aa4a-47c5-8cb2-8f34029b1355\") " pod="openshift-network-diagnostics/network-check-target-z4hnv" Jul 08 19:03:57.144210 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.143982 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-etc-openvswitch\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.144210 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.144006 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3b8b33cc-c6e1-4b41-9fe3-4ca56a434892-etc-kubernetes\") pod \"tuned-qwcgr\" (UID: \"3b8b33cc-c6e1-4b41-9fe3-4ca56a434892\") " pod="openshift-cluster-node-tuning-operator/tuned-qwcgr" Jul 08 19:03:57.144210 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.144033 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3b8b33cc-c6e1-4b41-9fe3-4ca56a434892-etc-systemd\") pod \"tuned-qwcgr\" (UID: \"3b8b33cc-c6e1-4b41-9fe3-4ca56a434892\") " pod="openshift-cluster-node-tuning-operator/tuned-qwcgr" Jul 08 19:03:57.144210 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.144054 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/31a4f57f-1bb0-4b8f-bb58-9a78ef1565c6-hosts-file\") pod \"node-resolver-zjgc5\" (UID: \"31a4f57f-1bb0-4b8f-bb58-9a78ef1565c6\") " pod="openshift-dns/node-resolver-zjgc5" Jul 08 19:03:57.144210 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.144081 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fb6af782-3262-467f-95e4-31d549573cd6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mpz48\" (UID: \"fb6af782-3262-467f-95e4-31d549573cd6\") " pod="openshift-multus/multus-additional-cni-plugins-mpz48" Jul 08 19:03:57.144210 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.144126 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9khr\" (UniqueName: \"kubernetes.io/projected/bb8c1f0b-7584-4d24-aa13-c73cad10576c-kube-api-access-t9khr\") pod \"node-ca-jw4sj\" (UID: \"bb8c1f0b-7584-4d24-aa13-c73cad10576c\") " pod="openshift-image-registry/node-ca-jw4sj" Jul 08 19:03:57.144210 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.144151 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-host-run-ovn-kubernetes\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.144210 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.144175 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-host-cni-netd\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.144210 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.144203 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8a5d7413-ff2a-47c9-a084-2b8e1939522d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xh2z8\" (UID: \"8a5d7413-ff2a-47c9-a084-2b8e1939522d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xh2z8" Jul 08 19:03:57.144729 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.144226 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8a5d7413-ff2a-47c9-a084-2b8e1939522d-etc-selinux\") pod \"aws-ebs-csi-driver-node-xh2z8\" (UID: \"8a5d7413-ff2a-47c9-a084-2b8e1939522d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xh2z8" Jul 08 19:03:57.144729 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.144249 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/001f0989-80a3-4b9d-8a57-9f41846bef7b-etc-kubernetes\") pod \"multus-6kmdh\" (UID: \"001f0989-80a3-4b9d-8a57-9f41846bef7b\") " pod="openshift-multus/multus-6kmdh" Jul 08 19:03:57.144729 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.144272 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fb6af782-3262-467f-95e4-31d549573cd6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mpz48\" (UID: \"fb6af782-3262-467f-95e4-31d549573cd6\") " pod="openshift-multus/multus-additional-cni-plugins-mpz48" Jul 08 19:03:57.144729 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.144297 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-host-cni-bin\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.144729 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.144319 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-ovn-node-metrics-cert\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.144729 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.144346 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3b8b33cc-c6e1-4b41-9fe3-4ca56a434892-var-lib-kubelet\") pod \"tuned-qwcgr\" (UID: \"3b8b33cc-c6e1-4b41-9fe3-4ca56a434892\") " pod="openshift-cluster-node-tuning-operator/tuned-qwcgr" Jul 08 19:03:57.144729 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.144367 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3b8b33cc-c6e1-4b41-9fe3-4ca56a434892-tmp\") pod \"tuned-qwcgr\" (UID: \"3b8b33cc-c6e1-4b41-9fe3-4ca56a434892\") " pod="openshift-cluster-node-tuning-operator/tuned-qwcgr" Jul 08 19:03:57.144729 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.144391 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-run-systemd\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.144729 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.144413 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/001f0989-80a3-4b9d-8a57-9f41846bef7b-host-run-netns\") pod \"multus-6kmdh\" (UID: \"001f0989-80a3-4b9d-8a57-9f41846bef7b\") " pod="openshift-multus/multus-6kmdh" Jul 08 19:03:57.144729 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.144464 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/001f0989-80a3-4b9d-8a57-9f41846bef7b-host-run-multus-certs\") pod \"multus-6kmdh\" (UID: \"001f0989-80a3-4b9d-8a57-9f41846bef7b\") " pod="openshift-multus/multus-6kmdh" Jul 08 19:03:57.144729 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.144494 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/001f0989-80a3-4b9d-8a57-9f41846bef7b-hostroot\") pod \"multus-6kmdh\" (UID: \"001f0989-80a3-4b9d-8a57-9f41846bef7b\") " pod="openshift-multus/multus-6kmdh" Jul 08 19:03:57.144729 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.144524 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/dcd11726-94ec-45c8-980e-9e362ab8c537-iptables-alerter-script\") pod \"iptables-alerter-mq9pr\" (UID: \"dcd11726-94ec-45c8-980e-9e362ab8c537\") " pod="openshift-network-operator/iptables-alerter-mq9pr" Jul 08 19:03:57.144729 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.144551 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tq9tm\" (UniqueName: \"kubernetes.io/projected/dcd11726-94ec-45c8-980e-9e362ab8c537-kube-api-access-tq9tm\") pod \"iptables-alerter-mq9pr\" (UID: \"dcd11726-94ec-45c8-980e-9e362ab8c537\") " pod="openshift-network-operator/iptables-alerter-mq9pr" Jul 08 19:03:57.144729 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.144576 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-host-kubelet\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.144729 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.144600 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/001f0989-80a3-4b9d-8a57-9f41846bef7b-system-cni-dir\") pod \"multus-6kmdh\" (UID: \"001f0989-80a3-4b9d-8a57-9f41846bef7b\") " pod="openshift-multus/multus-6kmdh" Jul 08 19:03:57.144729 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.144630 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3b8b33cc-c6e1-4b41-9fe3-4ca56a434892-etc-sysconfig\") pod \"tuned-qwcgr\" (UID: \"3b8b33cc-c6e1-4b41-9fe3-4ca56a434892\") " pod="openshift-cluster-node-tuning-operator/tuned-qwcgr" Jul 08 19:03:57.145571 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.144657 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ms7dw\" (UniqueName: \"kubernetes.io/projected/3b8b33cc-c6e1-4b41-9fe3-4ca56a434892-kube-api-access-ms7dw\") pod \"tuned-qwcgr\" (UID: \"3b8b33cc-c6e1-4b41-9fe3-4ca56a434892\") " pod="openshift-cluster-node-tuning-operator/tuned-qwcgr" Jul 08 19:03:57.145571 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.144712 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fb6af782-3262-467f-95e4-31d549573cd6-system-cni-dir\") pod \"multus-additional-cni-plugins-mpz48\" (UID: \"fb6af782-3262-467f-95e4-31d549573cd6\") " pod="openshift-multus/multus-additional-cni-plugins-mpz48" Jul 08 19:03:57.145571 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.144737 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1033c4ea-5be7-4c7c-9c6c-0bbb663fad60-agent-certs\") pod \"konnectivity-agent-n8prt\" (UID: \"1033c4ea-5be7-4c7c-9c6c-0bbb663fad60\") " pod="kube-system/konnectivity-agent-n8prt" Jul 08 19:03:57.145571 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.144763 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-var-lib-openvswitch\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.145571 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.144786 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/001f0989-80a3-4b9d-8a57-9f41846bef7b-host-run-k8s-cni-cncf-io\") pod \"multus-6kmdh\" (UID: \"001f0989-80a3-4b9d-8a57-9f41846bef7b\") " pod="openshift-multus/multus-6kmdh" Jul 08 19:03:57.145571 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.144809 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/001f0989-80a3-4b9d-8a57-9f41846bef7b-multus-conf-dir\") pod \"multus-6kmdh\" (UID: \"001f0989-80a3-4b9d-8a57-9f41846bef7b\") " pod="openshift-multus/multus-6kmdh" Jul 08 19:03:57.145571 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.144831 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3b8b33cc-c6e1-4b41-9fe3-4ca56a434892-etc-tuned\") pod \"tuned-qwcgr\" (UID: \"3b8b33cc-c6e1-4b41-9fe3-4ca56a434892\") " pod="openshift-cluster-node-tuning-operator/tuned-qwcgr" Jul 08 19:03:57.145571 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.144855 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fb6af782-3262-467f-95e4-31d549573cd6-cnibin\") pod \"multus-additional-cni-plugins-mpz48\" (UID: \"fb6af782-3262-467f-95e4-31d549573cd6\") " pod="openshift-multus/multus-additional-cni-plugins-mpz48" Jul 08 19:03:57.145571 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.144878 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dcd11726-94ec-45c8-980e-9e362ab8c537-host-slash\") pod \"iptables-alerter-mq9pr\" (UID: \"dcd11726-94ec-45c8-980e-9e362ab8c537\") " pod="openshift-network-operator/iptables-alerter-mq9pr" Jul 08 19:03:57.145571 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.144900 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-host-run-netns\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.145571 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.144926 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-run-openvswitch\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.145571 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.144952 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5bfd\" (UniqueName: \"kubernetes.io/projected/8a5d7413-ff2a-47c9-a084-2b8e1939522d-kube-api-access-c5bfd\") pod \"aws-ebs-csi-driver-node-xh2z8\" (UID: \"8a5d7413-ff2a-47c9-a084-2b8e1939522d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xh2z8" Jul 08 19:03:57.145571 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.144977 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/001f0989-80a3-4b9d-8a57-9f41846bef7b-multus-socket-dir-parent\") pod \"multus-6kmdh\" (UID: \"001f0989-80a3-4b9d-8a57-9f41846bef7b\") " pod="openshift-multus/multus-6kmdh" Jul 08 19:03:57.145571 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.145013 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/001f0989-80a3-4b9d-8a57-9f41846bef7b-multus-daemon-config\") pod \"multus-6kmdh\" (UID: \"001f0989-80a3-4b9d-8a57-9f41846bef7b\") " pod="openshift-multus/multus-6kmdh" Jul 08 19:03:57.145571 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.145036 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3b8b33cc-c6e1-4b41-9fe3-4ca56a434892-etc-modprobe-d\") pod \"tuned-qwcgr\" (UID: \"3b8b33cc-c6e1-4b41-9fe3-4ca56a434892\") " pod="openshift-cluster-node-tuning-operator/tuned-qwcgr" Jul 08 19:03:57.145571 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.145058 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3b8b33cc-c6e1-4b41-9fe3-4ca56a434892-sys\") pod \"tuned-qwcgr\" (UID: \"3b8b33cc-c6e1-4b41-9fe3-4ca56a434892\") " pod="openshift-cluster-node-tuning-operator/tuned-qwcgr" Jul 08 19:03:57.146285 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.145080 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3b8b33cc-c6e1-4b41-9fe3-4ca56a434892-lib-modules\") pod \"tuned-qwcgr\" (UID: \"3b8b33cc-c6e1-4b41-9fe3-4ca56a434892\") " pod="openshift-cluster-node-tuning-operator/tuned-qwcgr" Jul 08 19:03:57.146285 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.145107 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9a902ad1-6161-490d-afb2-006966897633-dbus\") pod \"global-pull-secret-syncer-qx98w\" (UID: \"9a902ad1-6161-490d-afb2-006966897633\") " pod="kube-system/global-pull-secret-syncer-qx98w" Jul 08 19:03:57.146285 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.145137 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-host-slash\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.146285 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.145161 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-log-socket\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.146285 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.145187 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.146285 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.145213 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-ovnkube-config\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.146285 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.145239 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/001f0989-80a3-4b9d-8a57-9f41846bef7b-cni-binary-copy\") pod \"multus-6kmdh\" (UID: \"001f0989-80a3-4b9d-8a57-9f41846bef7b\") " pod="openshift-multus/multus-6kmdh" Jul 08 19:03:57.146285 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.145263 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ctl7\" (UniqueName: \"kubernetes.io/projected/fb6af782-3262-467f-95e4-31d549573cd6-kube-api-access-2ctl7\") pod \"multus-additional-cni-plugins-mpz48\" (UID: \"fb6af782-3262-467f-95e4-31d549573cd6\") " pod="openshift-multus/multus-additional-cni-plugins-mpz48" Jul 08 19:03:57.146285 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.145287 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1033c4ea-5be7-4c7c-9c6c-0bbb663fad60-konnectivity-ca\") pod \"konnectivity-agent-n8prt\" (UID: \"1033c4ea-5be7-4c7c-9c6c-0bbb663fad60\") " pod="kube-system/konnectivity-agent-n8prt" Jul 08 19:03:57.146285 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.145315 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9a902ad1-6161-490d-afb2-006966897633-kubelet-config\") pod \"global-pull-secret-syncer-qx98w\" (UID: \"9a902ad1-6161-490d-afb2-006966897633\") " pod="kube-system/global-pull-secret-syncer-qx98w" Jul 08 19:03:57.146285 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.145339 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8a5d7413-ff2a-47c9-a084-2b8e1939522d-sys-fs\") pod \"aws-ebs-csi-driver-node-xh2z8\" (UID: \"8a5d7413-ff2a-47c9-a084-2b8e1939522d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xh2z8" Jul 08 19:03:57.146285 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.145373 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-env-overrides\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.146285 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.145397 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8a5d7413-ff2a-47c9-a084-2b8e1939522d-registration-dir\") pod \"aws-ebs-csi-driver-node-xh2z8\" (UID: \"8a5d7413-ff2a-47c9-a084-2b8e1939522d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xh2z8" Jul 08 19:03:57.146285 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.145422 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/001f0989-80a3-4b9d-8a57-9f41846bef7b-host-var-lib-kubelet\") pod \"multus-6kmdh\" (UID: \"001f0989-80a3-4b9d-8a57-9f41846bef7b\") " pod="openshift-multus/multus-6kmdh" Jul 08 19:03:57.146285 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.145464 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3b8b33cc-c6e1-4b41-9fe3-4ca56a434892-host\") pod \"tuned-qwcgr\" (UID: \"3b8b33cc-c6e1-4b41-9fe3-4ca56a434892\") " pod="openshift-cluster-node-tuning-operator/tuned-qwcgr" Jul 08 19:03:57.146285 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.145492 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/bb8c1f0b-7584-4d24-aa13-c73cad10576c-serviceca\") pod \"node-ca-jw4sj\" (UID: \"bb8c1f0b-7584-4d24-aa13-c73cad10576c\") " pod="openshift-image-registry/node-ca-jw4sj" Jul 08 19:03:57.147093 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.145516 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-run-ovn\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.147093 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.145541 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8a5d7413-ff2a-47c9-a084-2b8e1939522d-socket-dir\") pod \"aws-ebs-csi-driver-node-xh2z8\" (UID: \"8a5d7413-ff2a-47c9-a084-2b8e1939522d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xh2z8" Jul 08 19:03:57.147093 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.145574 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/001f0989-80a3-4b9d-8a57-9f41846bef7b-os-release\") pod \"multus-6kmdh\" (UID: \"001f0989-80a3-4b9d-8a57-9f41846bef7b\") " pod="openshift-multus/multus-6kmdh" Jul 08 19:03:57.147093 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.145599 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-systemd-units\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.147093 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.145624 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8a5d7413-ff2a-47c9-a084-2b8e1939522d-device-dir\") pod \"aws-ebs-csi-driver-node-xh2z8\" (UID: \"8a5d7413-ff2a-47c9-a084-2b8e1939522d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xh2z8" Jul 08 19:03:57.147093 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.145648 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/001f0989-80a3-4b9d-8a57-9f41846bef7b-host-var-lib-cni-multus\") pod \"multus-6kmdh\" (UID: \"001f0989-80a3-4b9d-8a57-9f41846bef7b\") " pod="openshift-multus/multus-6kmdh" Jul 08 19:03:57.147093 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.145685 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31a4f57f-1bb0-4b8f-bb58-9a78ef1565c6-tmp-dir\") pod \"node-resolver-zjgc5\" (UID: \"31a4f57f-1bb0-4b8f-bb58-9a78ef1565c6\") " pod="openshift-dns/node-resolver-zjgc5" Jul 08 19:03:57.147093 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.145710 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fb6af782-3262-467f-95e4-31d549573cd6-cni-binary-copy\") pod \"multus-additional-cni-plugins-mpz48\" (UID: \"fb6af782-3262-467f-95e4-31d549573cd6\") " pod="openshift-multus/multus-additional-cni-plugins-mpz48" Jul 08 19:03:57.147093 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.145742 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnwcq\" (UniqueName: \"kubernetes.io/projected/6701701d-6ce5-4028-a9ac-9fabc3925522-kube-api-access-vnwcq\") pod \"network-metrics-daemon-hs752\" (UID: \"6701701d-6ce5-4028-a9ac-9fabc3925522\") " pod="openshift-multus/network-metrics-daemon-hs752" Jul 08 19:03:57.147093 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.145771 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-ovnkube-script-lib\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.147093 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.145796 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pb5d\" (UniqueName: \"kubernetes.io/projected/001f0989-80a3-4b9d-8a57-9f41846bef7b-kube-api-access-8pb5d\") pod \"multus-6kmdh\" (UID: \"001f0989-80a3-4b9d-8a57-9f41846bef7b\") " pod="openshift-multus/multus-6kmdh" Jul 08 19:03:57.147093 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.145842 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3b8b33cc-c6e1-4b41-9fe3-4ca56a434892-etc-sysctl-conf\") pod \"tuned-qwcgr\" (UID: \"3b8b33cc-c6e1-4b41-9fe3-4ca56a434892\") " pod="openshift-cluster-node-tuning-operator/tuned-qwcgr" Jul 08 19:03:57.147093 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.145869 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvfdn\" (UniqueName: \"kubernetes.io/projected/31a4f57f-1bb0-4b8f-bb58-9a78ef1565c6-kube-api-access-pvfdn\") pod \"node-resolver-zjgc5\" (UID: \"31a4f57f-1bb0-4b8f-bb58-9a78ef1565c6\") " pod="openshift-dns/node-resolver-zjgc5" Jul 08 19:03:57.147093 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.145894 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bb8c1f0b-7584-4d24-aa13-c73cad10576c-host\") pod \"node-ca-jw4sj\" (UID: \"bb8c1f0b-7584-4d24-aa13-c73cad10576c\") " pod="openshift-image-registry/node-ca-jw4sj" Jul 08 19:03:57.147093 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.145923 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76xj5\" (UniqueName: \"kubernetes.io/projected/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-kube-api-access-76xj5\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.147093 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.145951 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/001f0989-80a3-4b9d-8a57-9f41846bef7b-cnibin\") pod \"multus-6kmdh\" (UID: \"001f0989-80a3-4b9d-8a57-9f41846bef7b\") " pod="openshift-multus/multus-6kmdh" Jul 08 19:03:57.147093 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.145983 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/001f0989-80a3-4b9d-8a57-9f41846bef7b-host-var-lib-cni-bin\") pod \"multus-6kmdh\" (UID: \"001f0989-80a3-4b9d-8a57-9f41846bef7b\") " pod="openshift-multus/multus-6kmdh" Jul 08 19:03:57.147931 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:03:57.146159 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 19:03:57.147931 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:03:57.146235 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a902ad1-6161-490d-afb2-006966897633-original-pull-secret podName:9a902ad1-6161-490d-afb2-006966897633 nodeName:}" failed. No retries permitted until 2026-07-08 19:03:57.646202456 +0000 UTC m=+2.925290071 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9a902ad1-6161-490d-afb2-006966897633-original-pull-secret") pod "global-pull-secret-syncer-qx98w" (UID: "9a902ad1-6161-490d-afb2-006966897633") : object "kube-system"/"original-pull-secret" not registered Jul 08 19:03:57.147931 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:03:57.146466 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:03:57.147931 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:03:57.146504 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6701701d-6ce5-4028-a9ac-9fabc3925522-metrics-certs podName:6701701d-6ce5-4028-a9ac-9fabc3925522 nodeName:}" failed. No retries permitted until 2026-07-08 19:03:57.646494177 +0000 UTC m=+2.925581788 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6701701d-6ce5-4028-a9ac-9fabc3925522-metrics-certs") pod "network-metrics-daemon-hs752" (UID: "6701701d-6ce5-4028-a9ac-9fabc3925522") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:03:57.147931 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.147196 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9a902ad1-6161-490d-afb2-006966897633-dbus\") pod \"global-pull-secret-syncer-qx98w\" (UID: \"9a902ad1-6161-490d-afb2-006966897633\") " pod="kube-system/global-pull-secret-syncer-qx98w" Jul 08 19:03:57.147931 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.147335 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9a902ad1-6161-490d-afb2-006966897633-kubelet-config\") pod \"global-pull-secret-syncer-qx98w\" (UID: \"9a902ad1-6161-490d-afb2-006966897633\") " pod="kube-system/global-pull-secret-syncer-qx98w" Jul 08 19:03:57.148231 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.147957 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/bb8c1f0b-7584-4d24-aa13-c73cad10576c-serviceca\") pod \"node-ca-jw4sj\" (UID: \"bb8c1f0b-7584-4d24-aa13-c73cad10576c\") " pod="openshift-image-registry/node-ca-jw4sj" Jul 08 19:03:57.148387 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.148368 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bb8c1f0b-7584-4d24-aa13-c73cad10576c-host\") pod \"node-ca-jw4sj\" (UID: \"bb8c1f0b-7584-4d24-aa13-c73cad10576c\") " pod="openshift-image-registry/node-ca-jw4sj" Jul 08 19:03:57.155271 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.155245 2583 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 08 19:03:57.160643 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.159525 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnwcq\" (UniqueName: \"kubernetes.io/projected/6701701d-6ce5-4028-a9ac-9fabc3925522-kube-api-access-vnwcq\") pod \"network-metrics-daemon-hs752\" (UID: \"6701701d-6ce5-4028-a9ac-9fabc3925522\") " pod="openshift-multus/network-metrics-daemon-hs752" Jul 08 19:03:57.160643 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.159995 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9khr\" (UniqueName: \"kubernetes.io/projected/bb8c1f0b-7584-4d24-aa13-c73cad10576c-kube-api-access-t9khr\") pod \"node-ca-jw4sj\" (UID: \"bb8c1f0b-7584-4d24-aa13-c73cad10576c\") " pod="openshift-image-registry/node-ca-jw4sj" Jul 08 19:03:57.174388 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.174313 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-244.ec2.internal" event={"ID":"09c7821b3b1a8a9c6dd9709ff325da43","Type":"ContainerStarted","Data":"c73598be0c850e1c3ded56b3b69d9296de3c489b5dd19c268d930db748e040d7"} Jul 08 19:03:57.175504 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.175478 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-244.ec2.internal" event={"ID":"2bde17ed4dbbb226aeb25d822491cdfc","Type":"ContainerStarted","Data":"4afbbafcd6bbf362f965c90278fed12f6db26122f5842ff96958d070405c62c8"} Jul 08 19:03:57.246878 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.246840 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fb6af782-3262-467f-95e4-31d549573cd6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mpz48\" (UID: \"fb6af782-3262-467f-95e4-31d549573cd6\") " pod="openshift-multus/multus-additional-cni-plugins-mpz48" Jul 08 19:03:57.247055 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.246888 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-host-cni-bin\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.247055 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.246917 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-ovn-node-metrics-cert\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.247055 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.246941 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3b8b33cc-c6e1-4b41-9fe3-4ca56a434892-var-lib-kubelet\") pod \"tuned-qwcgr\" (UID: \"3b8b33cc-c6e1-4b41-9fe3-4ca56a434892\") " pod="openshift-cluster-node-tuning-operator/tuned-qwcgr" Jul 08 19:03:57.247055 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.246965 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3b8b33cc-c6e1-4b41-9fe3-4ca56a434892-tmp\") pod \"tuned-qwcgr\" (UID: \"3b8b33cc-c6e1-4b41-9fe3-4ca56a434892\") " pod="openshift-cluster-node-tuning-operator/tuned-qwcgr" Jul 08 19:03:57.247055 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.246988 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-run-systemd\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.247055 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.247013 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/001f0989-80a3-4b9d-8a57-9f41846bef7b-host-run-netns\") pod \"multus-6kmdh\" (UID: \"001f0989-80a3-4b9d-8a57-9f41846bef7b\") " pod="openshift-multus/multus-6kmdh" Jul 08 19:03:57.247055 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.247037 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/001f0989-80a3-4b9d-8a57-9f41846bef7b-host-run-multus-certs\") pod \"multus-6kmdh\" (UID: \"001f0989-80a3-4b9d-8a57-9f41846bef7b\") " pod="openshift-multus/multus-6kmdh" Jul 08 19:03:57.247331 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.247061 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/001f0989-80a3-4b9d-8a57-9f41846bef7b-hostroot\") pod \"multus-6kmdh\" (UID: \"001f0989-80a3-4b9d-8a57-9f41846bef7b\") " pod="openshift-multus/multus-6kmdh" Jul 08 19:03:57.247331 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.247084 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/dcd11726-94ec-45c8-980e-9e362ab8c537-iptables-alerter-script\") pod \"iptables-alerter-mq9pr\" (UID: \"dcd11726-94ec-45c8-980e-9e362ab8c537\") " pod="openshift-network-operator/iptables-alerter-mq9pr" Jul 08 19:03:57.247331 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.247114 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tq9tm\" (UniqueName: \"kubernetes.io/projected/dcd11726-94ec-45c8-980e-9e362ab8c537-kube-api-access-tq9tm\") pod \"iptables-alerter-mq9pr\" (UID: \"dcd11726-94ec-45c8-980e-9e362ab8c537\") " pod="openshift-network-operator/iptables-alerter-mq9pr" Jul 08 19:03:57.247331 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.247137 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-host-kubelet\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.247331 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.247159 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/001f0989-80a3-4b9d-8a57-9f41846bef7b-system-cni-dir\") pod \"multus-6kmdh\" (UID: \"001f0989-80a3-4b9d-8a57-9f41846bef7b\") " pod="openshift-multus/multus-6kmdh" Jul 08 19:03:57.247331 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.247181 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3b8b33cc-c6e1-4b41-9fe3-4ca56a434892-etc-sysconfig\") pod \"tuned-qwcgr\" (UID: \"3b8b33cc-c6e1-4b41-9fe3-4ca56a434892\") " pod="openshift-cluster-node-tuning-operator/tuned-qwcgr" Jul 08 19:03:57.247331 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.247203 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ms7dw\" (UniqueName: \"kubernetes.io/projected/3b8b33cc-c6e1-4b41-9fe3-4ca56a434892-kube-api-access-ms7dw\") pod \"tuned-qwcgr\" (UID: \"3b8b33cc-c6e1-4b41-9fe3-4ca56a434892\") " pod="openshift-cluster-node-tuning-operator/tuned-qwcgr" Jul 08 19:03:57.247331 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.247228 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fb6af782-3262-467f-95e4-31d549573cd6-system-cni-dir\") pod \"multus-additional-cni-plugins-mpz48\" (UID: \"fb6af782-3262-467f-95e4-31d549573cd6\") " pod="openshift-multus/multus-additional-cni-plugins-mpz48" Jul 08 19:03:57.247331 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.247250 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1033c4ea-5be7-4c7c-9c6c-0bbb663fad60-agent-certs\") pod \"konnectivity-agent-n8prt\" (UID: \"1033c4ea-5be7-4c7c-9c6c-0bbb663fad60\") " pod="kube-system/konnectivity-agent-n8prt" Jul 08 19:03:57.247331 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.247273 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-var-lib-openvswitch\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.247331 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.247296 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/001f0989-80a3-4b9d-8a57-9f41846bef7b-host-run-k8s-cni-cncf-io\") pod \"multus-6kmdh\" (UID: \"001f0989-80a3-4b9d-8a57-9f41846bef7b\") " pod="openshift-multus/multus-6kmdh" Jul 08 19:03:57.247331 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.247318 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/001f0989-80a3-4b9d-8a57-9f41846bef7b-multus-conf-dir\") pod \"multus-6kmdh\" (UID: \"001f0989-80a3-4b9d-8a57-9f41846bef7b\") " pod="openshift-multus/multus-6kmdh" Jul 08 19:03:57.247915 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.247340 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3b8b33cc-c6e1-4b41-9fe3-4ca56a434892-etc-tuned\") pod \"tuned-qwcgr\" (UID: \"3b8b33cc-c6e1-4b41-9fe3-4ca56a434892\") " pod="openshift-cluster-node-tuning-operator/tuned-qwcgr" Jul 08 19:03:57.247915 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.247364 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fb6af782-3262-467f-95e4-31d549573cd6-cnibin\") pod \"multus-additional-cni-plugins-mpz48\" (UID: \"fb6af782-3262-467f-95e4-31d549573cd6\") " pod="openshift-multus/multus-additional-cni-plugins-mpz48" Jul 08 19:03:57.247915 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.247386 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dcd11726-94ec-45c8-980e-9e362ab8c537-host-slash\") pod \"iptables-alerter-mq9pr\" (UID: \"dcd11726-94ec-45c8-980e-9e362ab8c537\") " pod="openshift-network-operator/iptables-alerter-mq9pr" Jul 08 19:03:57.247915 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.247415 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-host-run-netns\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.247915 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.247454 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-run-openvswitch\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.247915 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.247479 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5bfd\" (UniqueName: \"kubernetes.io/projected/8a5d7413-ff2a-47c9-a084-2b8e1939522d-kube-api-access-c5bfd\") pod \"aws-ebs-csi-driver-node-xh2z8\" (UID: \"8a5d7413-ff2a-47c9-a084-2b8e1939522d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xh2z8" Jul 08 19:03:57.247915 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.247503 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/001f0989-80a3-4b9d-8a57-9f41846bef7b-multus-socket-dir-parent\") pod \"multus-6kmdh\" (UID: \"001f0989-80a3-4b9d-8a57-9f41846bef7b\") " pod="openshift-multus/multus-6kmdh" Jul 08 19:03:57.247915 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.247527 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/001f0989-80a3-4b9d-8a57-9f41846bef7b-multus-daemon-config\") pod \"multus-6kmdh\" (UID: \"001f0989-80a3-4b9d-8a57-9f41846bef7b\") " pod="openshift-multus/multus-6kmdh" Jul 08 19:03:57.247915 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.247550 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3b8b33cc-c6e1-4b41-9fe3-4ca56a434892-etc-modprobe-d\") pod \"tuned-qwcgr\" (UID: \"3b8b33cc-c6e1-4b41-9fe3-4ca56a434892\") " pod="openshift-cluster-node-tuning-operator/tuned-qwcgr" Jul 08 19:03:57.247915 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.247572 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3b8b33cc-c6e1-4b41-9fe3-4ca56a434892-sys\") pod \"tuned-qwcgr\" (UID: \"3b8b33cc-c6e1-4b41-9fe3-4ca56a434892\") " pod="openshift-cluster-node-tuning-operator/tuned-qwcgr" Jul 08 19:03:57.247915 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.247594 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3b8b33cc-c6e1-4b41-9fe3-4ca56a434892-lib-modules\") pod \"tuned-qwcgr\" (UID: \"3b8b33cc-c6e1-4b41-9fe3-4ca56a434892\") " pod="openshift-cluster-node-tuning-operator/tuned-qwcgr" Jul 08 19:03:57.247915 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.247619 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-host-slash\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.247915 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.247641 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-log-socket\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.247915 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.247666 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.247915 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.247689 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-ovnkube-config\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.247915 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.247713 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/001f0989-80a3-4b9d-8a57-9f41846bef7b-cni-binary-copy\") pod \"multus-6kmdh\" (UID: \"001f0989-80a3-4b9d-8a57-9f41846bef7b\") " pod="openshift-multus/multus-6kmdh" Jul 08 19:03:57.247915 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.247739 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ctl7\" (UniqueName: \"kubernetes.io/projected/fb6af782-3262-467f-95e4-31d549573cd6-kube-api-access-2ctl7\") pod \"multus-additional-cni-plugins-mpz48\" (UID: \"fb6af782-3262-467f-95e4-31d549573cd6\") " pod="openshift-multus/multus-additional-cni-plugins-mpz48" Jul 08 19:03:57.248650 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.247764 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1033c4ea-5be7-4c7c-9c6c-0bbb663fad60-konnectivity-ca\") pod \"konnectivity-agent-n8prt\" (UID: \"1033c4ea-5be7-4c7c-9c6c-0bbb663fad60\") " pod="kube-system/konnectivity-agent-n8prt" Jul 08 19:03:57.248650 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.247789 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8a5d7413-ff2a-47c9-a084-2b8e1939522d-sys-fs\") pod \"aws-ebs-csi-driver-node-xh2z8\" (UID: \"8a5d7413-ff2a-47c9-a084-2b8e1939522d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xh2z8" Jul 08 19:03:57.248650 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.247813 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-env-overrides\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.248650 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.247836 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8a5d7413-ff2a-47c9-a084-2b8e1939522d-registration-dir\") pod \"aws-ebs-csi-driver-node-xh2z8\" (UID: \"8a5d7413-ff2a-47c9-a084-2b8e1939522d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xh2z8" Jul 08 19:03:57.248650 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.247868 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/001f0989-80a3-4b9d-8a57-9f41846bef7b-host-var-lib-kubelet\") pod \"multus-6kmdh\" (UID: \"001f0989-80a3-4b9d-8a57-9f41846bef7b\") " pod="openshift-multus/multus-6kmdh" Jul 08 19:03:57.248650 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.247892 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3b8b33cc-c6e1-4b41-9fe3-4ca56a434892-host\") pod \"tuned-qwcgr\" (UID: \"3b8b33cc-c6e1-4b41-9fe3-4ca56a434892\") " pod="openshift-cluster-node-tuning-operator/tuned-qwcgr" Jul 08 19:03:57.248650 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.247918 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-run-ovn\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.248650 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.247942 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8a5d7413-ff2a-47c9-a084-2b8e1939522d-socket-dir\") pod \"aws-ebs-csi-driver-node-xh2z8\" (UID: \"8a5d7413-ff2a-47c9-a084-2b8e1939522d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xh2z8" Jul 08 19:03:57.248650 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.247967 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/001f0989-80a3-4b9d-8a57-9f41846bef7b-os-release\") pod \"multus-6kmdh\" (UID: \"001f0989-80a3-4b9d-8a57-9f41846bef7b\") " pod="openshift-multus/multus-6kmdh" Jul 08 19:03:57.248650 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.247996 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-systemd-units\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.248650 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.248022 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8a5d7413-ff2a-47c9-a084-2b8e1939522d-device-dir\") pod \"aws-ebs-csi-driver-node-xh2z8\" (UID: \"8a5d7413-ff2a-47c9-a084-2b8e1939522d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xh2z8" Jul 08 19:03:57.248650 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.248072 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/001f0989-80a3-4b9d-8a57-9f41846bef7b-host-var-lib-cni-multus\") pod \"multus-6kmdh\" (UID: \"001f0989-80a3-4b9d-8a57-9f41846bef7b\") " pod="openshift-multus/multus-6kmdh" Jul 08 19:03:57.248650 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.248090 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31a4f57f-1bb0-4b8f-bb58-9a78ef1565c6-tmp-dir\") pod \"node-resolver-zjgc5\" (UID: \"31a4f57f-1bb0-4b8f-bb58-9a78ef1565c6\") " pod="openshift-dns/node-resolver-zjgc5" Jul 08 19:03:57.248650 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.248106 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fb6af782-3262-467f-95e4-31d549573cd6-cni-binary-copy\") pod \"multus-additional-cni-plugins-mpz48\" (UID: \"fb6af782-3262-467f-95e4-31d549573cd6\") " pod="openshift-multus/multus-additional-cni-plugins-mpz48" Jul 08 19:03:57.248650 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.248124 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-ovnkube-script-lib\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.248650 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.248147 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pb5d\" (UniqueName: \"kubernetes.io/projected/001f0989-80a3-4b9d-8a57-9f41846bef7b-kube-api-access-8pb5d\") pod \"multus-6kmdh\" (UID: \"001f0989-80a3-4b9d-8a57-9f41846bef7b\") " pod="openshift-multus/multus-6kmdh" Jul 08 19:03:57.248650 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.248170 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3b8b33cc-c6e1-4b41-9fe3-4ca56a434892-etc-sysctl-conf\") pod \"tuned-qwcgr\" (UID: \"3b8b33cc-c6e1-4b41-9fe3-4ca56a434892\") " pod="openshift-cluster-node-tuning-operator/tuned-qwcgr" Jul 08 19:03:57.249485 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.248191 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvfdn\" (UniqueName: \"kubernetes.io/projected/31a4f57f-1bb0-4b8f-bb58-9a78ef1565c6-kube-api-access-pvfdn\") pod \"node-resolver-zjgc5\" (UID: \"31a4f57f-1bb0-4b8f-bb58-9a78ef1565c6\") " pod="openshift-dns/node-resolver-zjgc5" Jul 08 19:03:57.249485 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.248219 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76xj5\" (UniqueName: \"kubernetes.io/projected/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-kube-api-access-76xj5\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.249485 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.248236 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/001f0989-80a3-4b9d-8a57-9f41846bef7b-cnibin\") pod \"multus-6kmdh\" (UID: \"001f0989-80a3-4b9d-8a57-9f41846bef7b\") " pod="openshift-multus/multus-6kmdh" Jul 08 19:03:57.249485 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.248262 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/001f0989-80a3-4b9d-8a57-9f41846bef7b-host-var-lib-cni-bin\") pod \"multus-6kmdh\" (UID: \"001f0989-80a3-4b9d-8a57-9f41846bef7b\") " pod="openshift-multus/multus-6kmdh" Jul 08 19:03:57.249485 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.248286 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3b8b33cc-c6e1-4b41-9fe3-4ca56a434892-etc-sysctl-d\") pod \"tuned-qwcgr\" (UID: \"3b8b33cc-c6e1-4b41-9fe3-4ca56a434892\") " pod="openshift-cluster-node-tuning-operator/tuned-qwcgr" Jul 08 19:03:57.249485 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.248309 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3b8b33cc-c6e1-4b41-9fe3-4ca56a434892-run\") pod \"tuned-qwcgr\" (UID: \"3b8b33cc-c6e1-4b41-9fe3-4ca56a434892\") " pod="openshift-cluster-node-tuning-operator/tuned-qwcgr" Jul 08 19:03:57.249485 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.248335 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fb6af782-3262-467f-95e4-31d549573cd6-os-release\") pod \"multus-additional-cni-plugins-mpz48\" (UID: \"fb6af782-3262-467f-95e4-31d549573cd6\") " pod="openshift-multus/multus-additional-cni-plugins-mpz48" Jul 08 19:03:57.249485 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.248393 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-node-log\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.249485 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.248422 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/001f0989-80a3-4b9d-8a57-9f41846bef7b-multus-cni-dir\") pod \"multus-6kmdh\" (UID: \"001f0989-80a3-4b9d-8a57-9f41846bef7b\") " pod="openshift-multus/multus-6kmdh" Jul 08 19:03:57.249485 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.248466 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fb6af782-3262-467f-95e4-31d549573cd6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mpz48\" (UID: \"fb6af782-3262-467f-95e4-31d549573cd6\") " pod="openshift-multus/multus-additional-cni-plugins-mpz48" Jul 08 19:03:57.249485 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.248492 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqh5r\" (UniqueName: \"kubernetes.io/projected/7f193dea-aa4a-47c5-8cb2-8f34029b1355-kube-api-access-wqh5r\") pod \"network-check-target-z4hnv\" (UID: \"7f193dea-aa4a-47c5-8cb2-8f34029b1355\") " pod="openshift-network-diagnostics/network-check-target-z4hnv" Jul 08 19:03:57.249485 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.248514 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-etc-openvswitch\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.249485 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.248529 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3b8b33cc-c6e1-4b41-9fe3-4ca56a434892-etc-kubernetes\") pod \"tuned-qwcgr\" (UID: \"3b8b33cc-c6e1-4b41-9fe3-4ca56a434892\") " pod="openshift-cluster-node-tuning-operator/tuned-qwcgr" Jul 08 19:03:57.249485 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.248546 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3b8b33cc-c6e1-4b41-9fe3-4ca56a434892-etc-systemd\") pod \"tuned-qwcgr\" (UID: \"3b8b33cc-c6e1-4b41-9fe3-4ca56a434892\") " pod="openshift-cluster-node-tuning-operator/tuned-qwcgr" Jul 08 19:03:57.249485 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.248568 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/31a4f57f-1bb0-4b8f-bb58-9a78ef1565c6-hosts-file\") pod \"node-resolver-zjgc5\" (UID: \"31a4f57f-1bb0-4b8f-bb58-9a78ef1565c6\") " pod="openshift-dns/node-resolver-zjgc5" Jul 08 19:03:57.249485 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.248586 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fb6af782-3262-467f-95e4-31d549573cd6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mpz48\" (UID: \"fb6af782-3262-467f-95e4-31d549573cd6\") " pod="openshift-multus/multus-additional-cni-plugins-mpz48" Jul 08 19:03:57.249485 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.248617 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-host-run-ovn-kubernetes\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.250275 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.248647 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-host-cni-netd\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.250275 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.248672 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8a5d7413-ff2a-47c9-a084-2b8e1939522d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xh2z8\" (UID: \"8a5d7413-ff2a-47c9-a084-2b8e1939522d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xh2z8" Jul 08 19:03:57.250275 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.248697 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8a5d7413-ff2a-47c9-a084-2b8e1939522d-etc-selinux\") pod \"aws-ebs-csi-driver-node-xh2z8\" (UID: \"8a5d7413-ff2a-47c9-a084-2b8e1939522d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xh2z8" Jul 08 19:03:57.250275 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.248720 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/001f0989-80a3-4b9d-8a57-9f41846bef7b-etc-kubernetes\") pod \"multus-6kmdh\" (UID: \"001f0989-80a3-4b9d-8a57-9f41846bef7b\") " pod="openshift-multus/multus-6kmdh" Jul 08 19:03:57.250275 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.248821 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/001f0989-80a3-4b9d-8a57-9f41846bef7b-etc-kubernetes\") pod \"multus-6kmdh\" (UID: \"001f0989-80a3-4b9d-8a57-9f41846bef7b\") " pod="openshift-multus/multus-6kmdh" Jul 08 19:03:57.250275 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.249250 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fb6af782-3262-467f-95e4-31d549573cd6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mpz48\" (UID: \"fb6af782-3262-467f-95e4-31d549573cd6\") " pod="openshift-multus/multus-additional-cni-plugins-mpz48" Jul 08 19:03:57.250275 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.249312 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-host-cni-bin\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.250275 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.249677 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8a5d7413-ff2a-47c9-a084-2b8e1939522d-sys-fs\") pod \"aws-ebs-csi-driver-node-xh2z8\" (UID: \"8a5d7413-ff2a-47c9-a084-2b8e1939522d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xh2z8" Jul 08 19:03:57.250275 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.249750 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3b8b33cc-c6e1-4b41-9fe3-4ca56a434892-var-lib-kubelet\") pod \"tuned-qwcgr\" (UID: \"3b8b33cc-c6e1-4b41-9fe3-4ca56a434892\") " pod="openshift-cluster-node-tuning-operator/tuned-qwcgr" Jul 08 19:03:57.250275 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.249752 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3b8b33cc-c6e1-4b41-9fe3-4ca56a434892-sys\") pod \"tuned-qwcgr\" (UID: \"3b8b33cc-c6e1-4b41-9fe3-4ca56a434892\") " pod="openshift-cluster-node-tuning-operator/tuned-qwcgr" Jul 08 19:03:57.250275 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.249833 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3b8b33cc-c6e1-4b41-9fe3-4ca56a434892-lib-modules\") pod \"tuned-qwcgr\" (UID: \"3b8b33cc-c6e1-4b41-9fe3-4ca56a434892\") " pod="openshift-cluster-node-tuning-operator/tuned-qwcgr" Jul 08 19:03:57.250275 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.249892 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-log-socket\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.250275 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.249899 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/001f0989-80a3-4b9d-8a57-9f41846bef7b-host-run-netns\") pod \"multus-6kmdh\" (UID: \"001f0989-80a3-4b9d-8a57-9f41846bef7b\") " pod="openshift-multus/multus-6kmdh" Jul 08 19:03:57.250275 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.249930 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-run-systemd\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.250275 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.249953 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/001f0989-80a3-4b9d-8a57-9f41846bef7b-host-run-multus-certs\") pod \"multus-6kmdh\" (UID: \"001f0989-80a3-4b9d-8a57-9f41846bef7b\") " pod="openshift-multus/multus-6kmdh" Jul 08 19:03:57.250275 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.249966 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-host-slash\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.250275 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.250004 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/001f0989-80a3-4b9d-8a57-9f41846bef7b-hostroot\") pod \"multus-6kmdh\" (UID: \"001f0989-80a3-4b9d-8a57-9f41846bef7b\") " pod="openshift-multus/multus-6kmdh" Jul 08 19:03:57.250275 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.250048 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.253602 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.250093 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3b8b33cc-c6e1-4b41-9fe3-4ca56a434892-etc-sysconfig\") pod \"tuned-qwcgr\" (UID: \"3b8b33cc-c6e1-4b41-9fe3-4ca56a434892\") " pod="openshift-cluster-node-tuning-operator/tuned-qwcgr" Jul 08 19:03:57.253602 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.250294 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fb6af782-3262-467f-95e4-31d549573cd6-system-cni-dir\") pod \"multus-additional-cni-plugins-mpz48\" (UID: \"fb6af782-3262-467f-95e4-31d549573cd6\") " pod="openshift-multus/multus-additional-cni-plugins-mpz48" Jul 08 19:03:57.253602 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.250561 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/001f0989-80a3-4b9d-8a57-9f41846bef7b-cnibin\") pod \"multus-6kmdh\" (UID: \"001f0989-80a3-4b9d-8a57-9f41846bef7b\") " pod="openshift-multus/multus-6kmdh" Jul 08 19:03:57.253602 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.250604 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/dcd11726-94ec-45c8-980e-9e362ab8c537-iptables-alerter-script\") pod \"iptables-alerter-mq9pr\" (UID: \"dcd11726-94ec-45c8-980e-9e362ab8c537\") " pod="openshift-network-operator/iptables-alerter-mq9pr" Jul 08 19:03:57.253602 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.250731 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-ovnkube-config\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.253602 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.250885 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-host-kubelet\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.253602 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.250968 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-env-overrides\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.253602 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.250999 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8a5d7413-ff2a-47c9-a084-2b8e1939522d-device-dir\") pod \"aws-ebs-csi-driver-node-xh2z8\" (UID: \"8a5d7413-ff2a-47c9-a084-2b8e1939522d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xh2z8" Jul 08 19:03:57.253602 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.251037 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8a5d7413-ff2a-47c9-a084-2b8e1939522d-registration-dir\") pod \"aws-ebs-csi-driver-node-xh2z8\" (UID: \"8a5d7413-ff2a-47c9-a084-2b8e1939522d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xh2z8" Jul 08 19:03:57.253602 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.251046 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/001f0989-80a3-4b9d-8a57-9f41846bef7b-host-var-lib-cni-multus\") pod \"multus-6kmdh\" (UID: \"001f0989-80a3-4b9d-8a57-9f41846bef7b\") " pod="openshift-multus/multus-6kmdh" Jul 08 19:03:57.253602 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.251074 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3b8b33cc-c6e1-4b41-9fe3-4ca56a434892-host\") pod \"tuned-qwcgr\" (UID: \"3b8b33cc-c6e1-4b41-9fe3-4ca56a434892\") " pod="openshift-cluster-node-tuning-operator/tuned-qwcgr" Jul 08 19:03:57.253602 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.251079 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/001f0989-80a3-4b9d-8a57-9f41846bef7b-host-var-lib-kubelet\") pod \"multus-6kmdh\" (UID: \"001f0989-80a3-4b9d-8a57-9f41846bef7b\") " pod="openshift-multus/multus-6kmdh" Jul 08 19:03:57.253602 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.251105 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-run-ovn\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.253602 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.251124 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/001f0989-80a3-4b9d-8a57-9f41846bef7b-os-release\") pod \"multus-6kmdh\" (UID: \"001f0989-80a3-4b9d-8a57-9f41846bef7b\") " pod="openshift-multus/multus-6kmdh" Jul 08 19:03:57.253602 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.251150 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-systemd-units\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.253602 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.251175 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8a5d7413-ff2a-47c9-a084-2b8e1939522d-socket-dir\") pod \"aws-ebs-csi-driver-node-xh2z8\" (UID: \"8a5d7413-ff2a-47c9-a084-2b8e1939522d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xh2z8" Jul 08 19:03:57.253602 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.251222 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/001f0989-80a3-4b9d-8a57-9f41846bef7b-cni-binary-copy\") pod \"multus-6kmdh\" (UID: \"001f0989-80a3-4b9d-8a57-9f41846bef7b\") " pod="openshift-multus/multus-6kmdh" Jul 08 19:03:57.253602 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.251355 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31a4f57f-1bb0-4b8f-bb58-9a78ef1565c6-tmp-dir\") pod \"node-resolver-zjgc5\" (UID: \"31a4f57f-1bb0-4b8f-bb58-9a78ef1565c6\") " pod="openshift-dns/node-resolver-zjgc5" Jul 08 19:03:57.254711 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.251603 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-run-openvswitch\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.254711 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.251650 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fb6af782-3262-467f-95e4-31d549573cd6-cnibin\") pod \"multus-additional-cni-plugins-mpz48\" (UID: \"fb6af782-3262-467f-95e4-31d549573cd6\") " pod="openshift-multus/multus-additional-cni-plugins-mpz48" Jul 08 19:03:57.254711 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.251749 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dcd11726-94ec-45c8-980e-9e362ab8c537-host-slash\") pod \"iptables-alerter-mq9pr\" (UID: \"dcd11726-94ec-45c8-980e-9e362ab8c537\") " pod="openshift-network-operator/iptables-alerter-mq9pr" Jul 08 19:03:57.254711 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.251790 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-host-run-netns\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.254711 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.251797 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fb6af782-3262-467f-95e4-31d549573cd6-cni-binary-copy\") pod \"multus-additional-cni-plugins-mpz48\" (UID: \"fb6af782-3262-467f-95e4-31d549573cd6\") " pod="openshift-multus/multus-additional-cni-plugins-mpz48" Jul 08 19:03:57.254711 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.251826 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/001f0989-80a3-4b9d-8a57-9f41846bef7b-host-var-lib-cni-bin\") pod \"multus-6kmdh\" (UID: \"001f0989-80a3-4b9d-8a57-9f41846bef7b\") " pod="openshift-multus/multus-6kmdh" Jul 08 19:03:57.254711 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.251841 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-etc-openvswitch\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.254711 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.251906 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3b8b33cc-c6e1-4b41-9fe3-4ca56a434892-etc-systemd\") pod \"tuned-qwcgr\" (UID: \"3b8b33cc-c6e1-4b41-9fe3-4ca56a434892\") " pod="openshift-cluster-node-tuning-operator/tuned-qwcgr" Jul 08 19:03:57.254711 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.251911 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3b8b33cc-c6e1-4b41-9fe3-4ca56a434892-etc-sysctl-d\") pod \"tuned-qwcgr\" (UID: \"3b8b33cc-c6e1-4b41-9fe3-4ca56a434892\") " pod="openshift-cluster-node-tuning-operator/tuned-qwcgr" Jul 08 19:03:57.254711 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.251960 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/31a4f57f-1bb0-4b8f-bb58-9a78ef1565c6-hosts-file\") pod \"node-resolver-zjgc5\" (UID: \"31a4f57f-1bb0-4b8f-bb58-9a78ef1565c6\") " pod="openshift-dns/node-resolver-zjgc5" Jul 08 19:03:57.254711 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.251965 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3b8b33cc-c6e1-4b41-9fe3-4ca56a434892-run\") pod \"tuned-qwcgr\" (UID: \"3b8b33cc-c6e1-4b41-9fe3-4ca56a434892\") " pod="openshift-cluster-node-tuning-operator/tuned-qwcgr" Jul 08 19:03:57.254711 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.252016 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/001f0989-80a3-4b9d-8a57-9f41846bef7b-system-cni-dir\") pod \"multus-6kmdh\" (UID: \"001f0989-80a3-4b9d-8a57-9f41846bef7b\") " pod="openshift-multus/multus-6kmdh" Jul 08 19:03:57.254711 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.252085 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/001f0989-80a3-4b9d-8a57-9f41846bef7b-multus-socket-dir-parent\") pod \"multus-6kmdh\" (UID: \"001f0989-80a3-4b9d-8a57-9f41846bef7b\") " pod="openshift-multus/multus-6kmdh" Jul 08 19:03:57.254711 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.252128 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fb6af782-3262-467f-95e4-31d549573cd6-os-release\") pod \"multus-additional-cni-plugins-mpz48\" (UID: \"fb6af782-3262-467f-95e4-31d549573cd6\") " pod="openshift-multus/multus-additional-cni-plugins-mpz48" Jul 08 19:03:57.254711 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.252172 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-node-log\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.254711 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.252198 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/001f0989-80a3-4b9d-8a57-9f41846bef7b-host-run-k8s-cni-cncf-io\") pod \"multus-6kmdh\" (UID: \"001f0989-80a3-4b9d-8a57-9f41846bef7b\") " pod="openshift-multus/multus-6kmdh" Jul 08 19:03:57.254711 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.252234 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/001f0989-80a3-4b9d-8a57-9f41846bef7b-multus-cni-dir\") pod \"multus-6kmdh\" (UID: \"001f0989-80a3-4b9d-8a57-9f41846bef7b\") " pod="openshift-multus/multus-6kmdh" Jul 08 19:03:57.254711 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.252245 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-var-lib-openvswitch\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.255528 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.252290 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-host-run-ovn-kubernetes\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.255528 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.252295 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-host-cni-netd\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.255528 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.252331 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8a5d7413-ff2a-47c9-a084-2b8e1939522d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xh2z8\" (UID: \"8a5d7413-ff2a-47c9-a084-2b8e1939522d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xh2z8" Jul 08 19:03:57.255528 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.252389 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fb6af782-3262-467f-95e4-31d549573cd6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mpz48\" (UID: \"fb6af782-3262-467f-95e4-31d549573cd6\") " pod="openshift-multus/multus-additional-cni-plugins-mpz48" Jul 08 19:03:57.255528 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.252418 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8a5d7413-ff2a-47c9-a084-2b8e1939522d-etc-selinux\") pod \"aws-ebs-csi-driver-node-xh2z8\" (UID: \"8a5d7413-ff2a-47c9-a084-2b8e1939522d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xh2z8" Jul 08 19:03:57.255528 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.252488 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fb6af782-3262-467f-95e4-31d549573cd6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mpz48\" (UID: \"fb6af782-3262-467f-95e4-31d549573cd6\") " pod="openshift-multus/multus-additional-cni-plugins-mpz48" Jul 08 19:03:57.255528 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.252595 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1033c4ea-5be7-4c7c-9c6c-0bbb663fad60-konnectivity-ca\") pod \"konnectivity-agent-n8prt\" (UID: \"1033c4ea-5be7-4c7c-9c6c-0bbb663fad60\") " pod="kube-system/konnectivity-agent-n8prt" Jul 08 19:03:57.255528 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.251895 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3b8b33cc-c6e1-4b41-9fe3-4ca56a434892-etc-kubernetes\") pod \"tuned-qwcgr\" (UID: \"3b8b33cc-c6e1-4b41-9fe3-4ca56a434892\") " pod="openshift-cluster-node-tuning-operator/tuned-qwcgr" Jul 08 19:03:57.255528 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.252596 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3b8b33cc-c6e1-4b41-9fe3-4ca56a434892-etc-modprobe-d\") pod \"tuned-qwcgr\" (UID: \"3b8b33cc-c6e1-4b41-9fe3-4ca56a434892\") " pod="openshift-cluster-node-tuning-operator/tuned-qwcgr" Jul 08 19:03:57.255528 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.252745 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3b8b33cc-c6e1-4b41-9fe3-4ca56a434892-etc-sysctl-conf\") pod \"tuned-qwcgr\" (UID: \"3b8b33cc-c6e1-4b41-9fe3-4ca56a434892\") " pod="openshift-cluster-node-tuning-operator/tuned-qwcgr" Jul 08 19:03:57.255528 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.252785 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/001f0989-80a3-4b9d-8a57-9f41846bef7b-multus-conf-dir\") pod \"multus-6kmdh\" (UID: \"001f0989-80a3-4b9d-8a57-9f41846bef7b\") " pod="openshift-multus/multus-6kmdh" Jul 08 19:03:57.255528 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.252865 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/001f0989-80a3-4b9d-8a57-9f41846bef7b-multus-daemon-config\") pod \"multus-6kmdh\" (UID: \"001f0989-80a3-4b9d-8a57-9f41846bef7b\") " pod="openshift-multus/multus-6kmdh" Jul 08 19:03:57.255528 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.252887 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-ovnkube-script-lib\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.255528 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.253731 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3b8b33cc-c6e1-4b41-9fe3-4ca56a434892-tmp\") pod \"tuned-qwcgr\" (UID: \"3b8b33cc-c6e1-4b41-9fe3-4ca56a434892\") " pod="openshift-cluster-node-tuning-operator/tuned-qwcgr" Jul 08 19:03:57.255528 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.253765 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-ovn-node-metrics-cert\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.256272 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.256245 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3b8b33cc-c6e1-4b41-9fe3-4ca56a434892-etc-tuned\") pod \"tuned-qwcgr\" (UID: \"3b8b33cc-c6e1-4b41-9fe3-4ca56a434892\") " pod="openshift-cluster-node-tuning-operator/tuned-qwcgr" Jul 08 19:03:57.256534 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.256516 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1033c4ea-5be7-4c7c-9c6c-0bbb663fad60-agent-certs\") pod \"konnectivity-agent-n8prt\" (UID: \"1033c4ea-5be7-4c7c-9c6c-0bbb663fad60\") " pod="kube-system/konnectivity-agent-n8prt" Jul 08 19:03:57.263056 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.262999 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tq9tm\" (UniqueName: \"kubernetes.io/projected/dcd11726-94ec-45c8-980e-9e362ab8c537-kube-api-access-tq9tm\") pod \"iptables-alerter-mq9pr\" (UID: \"dcd11726-94ec-45c8-980e-9e362ab8c537\") " pod="openshift-network-operator/iptables-alerter-mq9pr" Jul 08 19:03:57.263289 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:03:57.263074 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 19:03:57.263289 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:03:57.263097 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 19:03:57.263289 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:03:57.263110 2583 projected.go:196] Error preparing data for projected volume kube-api-access-wqh5r for pod openshift-network-diagnostics/network-check-target-z4hnv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:03:57.263289 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:03:57.263182 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7f193dea-aa4a-47c5-8cb2-8f34029b1355-kube-api-access-wqh5r podName:7f193dea-aa4a-47c5-8cb2-8f34029b1355 nodeName:}" failed. No retries permitted until 2026-07-08 19:03:57.763162613 +0000 UTC m=+3.042250228 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-wqh5r" (UniqueName: "kubernetes.io/projected/7f193dea-aa4a-47c5-8cb2-8f34029b1355-kube-api-access-wqh5r") pod "network-check-target-z4hnv" (UID: "7f193dea-aa4a-47c5-8cb2-8f34029b1355") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:03:57.265049 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.265025 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ms7dw\" (UniqueName: \"kubernetes.io/projected/3b8b33cc-c6e1-4b41-9fe3-4ca56a434892-kube-api-access-ms7dw\") pod \"tuned-qwcgr\" (UID: \"3b8b33cc-c6e1-4b41-9fe3-4ca56a434892\") " pod="openshift-cluster-node-tuning-operator/tuned-qwcgr" Jul 08 19:03:57.268987 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.268967 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pb5d\" (UniqueName: \"kubernetes.io/projected/001f0989-80a3-4b9d-8a57-9f41846bef7b-kube-api-access-8pb5d\") pod \"multus-6kmdh\" (UID: \"001f0989-80a3-4b9d-8a57-9f41846bef7b\") " pod="openshift-multus/multus-6kmdh" Jul 08 19:03:57.271094 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.271071 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvfdn\" (UniqueName: \"kubernetes.io/projected/31a4f57f-1bb0-4b8f-bb58-9a78ef1565c6-kube-api-access-pvfdn\") pod \"node-resolver-zjgc5\" (UID: \"31a4f57f-1bb0-4b8f-bb58-9a78ef1565c6\") " pod="openshift-dns/node-resolver-zjgc5" Jul 08 19:03:57.271189 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.271074 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ctl7\" (UniqueName: \"kubernetes.io/projected/fb6af782-3262-467f-95e4-31d549573cd6-kube-api-access-2ctl7\") pod \"multus-additional-cni-plugins-mpz48\" (UID: \"fb6af782-3262-467f-95e4-31d549573cd6\") " pod="openshift-multus/multus-additional-cni-plugins-mpz48" Jul 08 19:03:57.272320 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.272298 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-76xj5\" (UniqueName: \"kubernetes.io/projected/37e62f02-e16f-45b7-8c48-0a2e3b3bd66c-kube-api-access-76xj5\") pod \"ovnkube-node-2t8gf\" (UID: \"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c\") " pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.273207 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.273184 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5bfd\" (UniqueName: \"kubernetes.io/projected/8a5d7413-ff2a-47c9-a084-2b8e1939522d-kube-api-access-c5bfd\") pod \"aws-ebs-csi-driver-node-xh2z8\" (UID: \"8a5d7413-ff2a-47c9-a084-2b8e1939522d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xh2z8" Jul 08 19:03:57.342230 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.342151 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jw4sj" Jul 08 19:03:57.353129 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:57.353097 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb8c1f0b_7584_4d24_aa13_c73cad10576c.slice/crio-0ae74e362ddcc968ecbc4108e72d84577427415f3fa832788769e39007acf236 WatchSource:0}: Error finding container 0ae74e362ddcc968ecbc4108e72d84577427415f3fa832788769e39007acf236: Status 404 returned error can't find the container with id 0ae74e362ddcc968ecbc4108e72d84577427415f3fa832788769e39007acf236 Jul 08 19:03:57.359399 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.359362 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:03:57.369989 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.369968 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-n8prt" Jul 08 19:03:57.370360 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:57.370327 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37e62f02_e16f_45b7_8c48_0a2e3b3bd66c.slice/crio-3cac4471e2f8848532b363d2363f4dd330617ad4de4ab593c36070a9f2a91de7 WatchSource:0}: Error finding container 3cac4471e2f8848532b363d2363f4dd330617ad4de4ab593c36070a9f2a91de7: Status 404 returned error can't find the container with id 3cac4471e2f8848532b363d2363f4dd330617ad4de4ab593c36070a9f2a91de7 Jul 08 19:03:57.379116 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.379093 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xh2z8" Jul 08 19:03:57.379679 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:57.379656 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1033c4ea_5be7_4c7c_9c6c_0bbb663fad60.slice/crio-136223b2aad466bf593828f9eb3a4b09d83b6fc0edec2a194962369ac9fe7a6a WatchSource:0}: Error finding container 136223b2aad466bf593828f9eb3a4b09d83b6fc0edec2a194962369ac9fe7a6a: Status 404 returned error can't find the container with id 136223b2aad466bf593828f9eb3a4b09d83b6fc0edec2a194962369ac9fe7a6a Jul 08 19:03:57.387909 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.387890 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6kmdh" Jul 08 19:03:57.388298 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:57.388272 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a5d7413_ff2a_47c9_a084_2b8e1939522d.slice/crio-21b7e3e07d06f4a4754c79111bd65a11cde802fdcca097de631e66a478033ff3 WatchSource:0}: Error finding container 21b7e3e07d06f4a4754c79111bd65a11cde802fdcca097de631e66a478033ff3: Status 404 returned error can't find the container with id 21b7e3e07d06f4a4754c79111bd65a11cde802fdcca097de631e66a478033ff3 Jul 08 19:03:57.394257 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.394233 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qwcgr" Jul 08 19:03:57.399630 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:57.397196 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod001f0989_80a3_4b9d_8a57_9f41846bef7b.slice/crio-9a7fa553a9154da74e8c54530535820f134e3083861412707c817d499c82b8d7 WatchSource:0}: Error finding container 9a7fa553a9154da74e8c54530535820f134e3083861412707c817d499c82b8d7: Status 404 returned error can't find the container with id 9a7fa553a9154da74e8c54530535820f134e3083861412707c817d499c82b8d7 Jul 08 19:03:57.402314 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.402219 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zjgc5" Jul 08 19:03:57.404377 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:57.404342 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b8b33cc_c6e1_4b41_9fe3_4ca56a434892.slice/crio-7e6f62633cb72f1761ba1dcbb964de22faec42dfb4400ef6f89f2692873b56d1 WatchSource:0}: Error finding container 7e6f62633cb72f1761ba1dcbb964de22faec42dfb4400ef6f89f2692873b56d1: Status 404 returned error can't find the container with id 7e6f62633cb72f1761ba1dcbb964de22faec42dfb4400ef6f89f2692873b56d1 Jul 08 19:03:57.411029 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.411007 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mpz48" Jul 08 19:03:57.412000 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:57.411963 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31a4f57f_1bb0_4b8f_bb58_9a78ef1565c6.slice/crio-806d9eae1322ef23644c019aa161da527c646c790c3fecbdbb0606910f67a97e WatchSource:0}: Error finding container 806d9eae1322ef23644c019aa161da527c646c790c3fecbdbb0606910f67a97e: Status 404 returned error can't find the container with id 806d9eae1322ef23644c019aa161da527c646c790c3fecbdbb0606910f67a97e Jul 08 19:03:57.418208 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.418183 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mq9pr" Jul 08 19:03:57.420694 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:57.420670 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb6af782_3262_467f_95e4_31d549573cd6.slice/crio-89b055639a6b33dd426b5ff0e39104031f59f1356c21bbecfd4b8b1b81ffb257 WatchSource:0}: Error finding container 89b055639a6b33dd426b5ff0e39104031f59f1356c21bbecfd4b8b1b81ffb257: Status 404 returned error can't find the container with id 89b055639a6b33dd426b5ff0e39104031f59f1356c21bbecfd4b8b1b81ffb257 Jul 08 19:03:57.428223 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:03:57.428200 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcd11726_94ec_45c8_980e_9e362ab8c537.slice/crio-b59722348bef30a12a8ac4237199fdda91f47edd011dce33c500ec62262b3c82 WatchSource:0}: Error finding container b59722348bef30a12a8ac4237199fdda91f47edd011dce33c500ec62262b3c82: Status 404 returned error can't find the container with id b59722348bef30a12a8ac4237199fdda91f47edd011dce33c500ec62262b3c82 Jul 08 19:03:57.651983 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.651890 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9a902ad1-6161-490d-afb2-006966897633-original-pull-secret\") pod \"global-pull-secret-syncer-qx98w\" (UID: \"9a902ad1-6161-490d-afb2-006966897633\") " pod="kube-system/global-pull-secret-syncer-qx98w" Jul 08 19:03:57.651983 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.651930 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6701701d-6ce5-4028-a9ac-9fabc3925522-metrics-certs\") pod \"network-metrics-daemon-hs752\" (UID: \"6701701d-6ce5-4028-a9ac-9fabc3925522\") " pod="openshift-multus/network-metrics-daemon-hs752" Jul 08 19:03:57.652179 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:03:57.652040 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:03:57.652179 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:03:57.652043 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 19:03:57.652179 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:03:57.652107 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6701701d-6ce5-4028-a9ac-9fabc3925522-metrics-certs podName:6701701d-6ce5-4028-a9ac-9fabc3925522 nodeName:}" failed. No retries permitted until 2026-07-08 19:03:58.652089266 +0000 UTC m=+3.931176886 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6701701d-6ce5-4028-a9ac-9fabc3925522-metrics-certs") pod "network-metrics-daemon-hs752" (UID: "6701701d-6ce5-4028-a9ac-9fabc3925522") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:03:57.652179 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:03:57.652124 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a902ad1-6161-490d-afb2-006966897633-original-pull-secret podName:9a902ad1-6161-490d-afb2-006966897633 nodeName:}" failed. No retries permitted until 2026-07-08 19:03:58.652116528 +0000 UTC m=+3.931204125 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9a902ad1-6161-490d-afb2-006966897633-original-pull-secret") pod "global-pull-secret-syncer-qx98w" (UID: "9a902ad1-6161-490d-afb2-006966897633") : object "kube-system"/"original-pull-secret" not registered Jul 08 19:03:57.853222 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:57.853189 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqh5r\" (UniqueName: \"kubernetes.io/projected/7f193dea-aa4a-47c5-8cb2-8f34029b1355-kube-api-access-wqh5r\") pod \"network-check-target-z4hnv\" (UID: \"7f193dea-aa4a-47c5-8cb2-8f34029b1355\") " pod="openshift-network-diagnostics/network-check-target-z4hnv" Jul 08 19:03:57.853402 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:03:57.853315 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 19:03:57.853402 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:03:57.853332 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 19:03:57.853402 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:03:57.853345 2583 projected.go:196] Error preparing data for projected volume kube-api-access-wqh5r for pod openshift-network-diagnostics/network-check-target-z4hnv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:03:57.853574 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:03:57.853409 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7f193dea-aa4a-47c5-8cb2-8f34029b1355-kube-api-access-wqh5r podName:7f193dea-aa4a-47c5-8cb2-8f34029b1355 nodeName:}" failed. No retries permitted until 2026-07-08 19:03:58.853390231 +0000 UTC m=+4.132478023 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-wqh5r" (UniqueName: "kubernetes.io/projected/7f193dea-aa4a-47c5-8cb2-8f34029b1355-kube-api-access-wqh5r") pod "network-check-target-z4hnv" (UID: "7f193dea-aa4a-47c5-8cb2-8f34029b1355") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:03:58.093250 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:58.093186 2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 18:58:57 +0000 UTC" deadline="2028-04-11 08:25:17.385250202 +0000 UTC" Jul 08 19:03:58.093250 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:58.093233 2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15421h21m19.292022251s" Jul 08 19:03:58.178128 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:58.178085 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mq9pr" event={"ID":"dcd11726-94ec-45c8-980e-9e362ab8c537","Type":"ContainerStarted","Data":"b59722348bef30a12a8ac4237199fdda91f47edd011dce33c500ec62262b3c82"} Jul 08 19:03:58.180096 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:58.180065 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mpz48" event={"ID":"fb6af782-3262-467f-95e4-31d549573cd6","Type":"ContainerStarted","Data":"89b055639a6b33dd426b5ff0e39104031f59f1356c21bbecfd4b8b1b81ffb257"} Jul 08 19:03:58.181420 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:58.181373 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qwcgr" event={"ID":"3b8b33cc-c6e1-4b41-9fe3-4ca56a434892","Type":"ContainerStarted","Data":"7e6f62633cb72f1761ba1dcbb964de22faec42dfb4400ef6f89f2692873b56d1"} Jul 08 19:03:58.182677 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:58.182632 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6kmdh" event={"ID":"001f0989-80a3-4b9d-8a57-9f41846bef7b","Type":"ContainerStarted","Data":"9a7fa553a9154da74e8c54530535820f134e3083861412707c817d499c82b8d7"} Jul 08 19:03:58.183990 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:58.183927 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xh2z8" event={"ID":"8a5d7413-ff2a-47c9-a084-2b8e1939522d","Type":"ContainerStarted","Data":"21b7e3e07d06f4a4754c79111bd65a11cde802fdcca097de631e66a478033ff3"} Jul 08 19:03:58.185289 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:58.185232 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-n8prt" event={"ID":"1033c4ea-5be7-4c7c-9c6c-0bbb663fad60","Type":"ContainerStarted","Data":"136223b2aad466bf593828f9eb3a4b09d83b6fc0edec2a194962369ac9fe7a6a"} Jul 08 19:03:58.186822 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:58.186758 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zjgc5" event={"ID":"31a4f57f-1bb0-4b8f-bb58-9a78ef1565c6","Type":"ContainerStarted","Data":"806d9eae1322ef23644c019aa161da527c646c790c3fecbdbb0606910f67a97e"} Jul 08 19:03:58.188953 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:58.188854 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" event={"ID":"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c","Type":"ContainerStarted","Data":"3cac4471e2f8848532b363d2363f4dd330617ad4de4ab593c36070a9f2a91de7"} Jul 08 19:03:58.190265 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:58.190209 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jw4sj" event={"ID":"bb8c1f0b-7584-4d24-aa13-c73cad10576c","Type":"ContainerStarted","Data":"0ae74e362ddcc968ecbc4108e72d84577427415f3fa832788769e39007acf236"} Jul 08 19:03:58.657958 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:58.657916 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9a902ad1-6161-490d-afb2-006966897633-original-pull-secret\") pod \"global-pull-secret-syncer-qx98w\" (UID: \"9a902ad1-6161-490d-afb2-006966897633\") " pod="kube-system/global-pull-secret-syncer-qx98w" Jul 08 19:03:58.658123 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:58.657971 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6701701d-6ce5-4028-a9ac-9fabc3925522-metrics-certs\") pod \"network-metrics-daemon-hs752\" (UID: \"6701701d-6ce5-4028-a9ac-9fabc3925522\") " pod="openshift-multus/network-metrics-daemon-hs752" Jul 08 19:03:58.658191 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:03:58.658141 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:03:58.658243 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:03:58.658202 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6701701d-6ce5-4028-a9ac-9fabc3925522-metrics-certs podName:6701701d-6ce5-4028-a9ac-9fabc3925522 nodeName:}" failed. No retries permitted until 2026-07-08 19:04:00.658184162 +0000 UTC m=+5.937271776 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6701701d-6ce5-4028-a9ac-9fabc3925522-metrics-certs") pod "network-metrics-daemon-hs752" (UID: "6701701d-6ce5-4028-a9ac-9fabc3925522") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:03:58.658644 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:03:58.658624 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 19:03:58.658741 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:03:58.658679 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a902ad1-6161-490d-afb2-006966897633-original-pull-secret podName:9a902ad1-6161-490d-afb2-006966897633 nodeName:}" failed. No retries permitted until 2026-07-08 19:04:00.658662469 +0000 UTC m=+5.937750084 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9a902ad1-6161-490d-afb2-006966897633-original-pull-secret") pod "global-pull-secret-syncer-qx98w" (UID: "9a902ad1-6161-490d-afb2-006966897633") : object "kube-system"/"original-pull-secret" not registered Jul 08 19:03:58.860171 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:58.860134 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqh5r\" (UniqueName: \"kubernetes.io/projected/7f193dea-aa4a-47c5-8cb2-8f34029b1355-kube-api-access-wqh5r\") pod \"network-check-target-z4hnv\" (UID: \"7f193dea-aa4a-47c5-8cb2-8f34029b1355\") " pod="openshift-network-diagnostics/network-check-target-z4hnv" Jul 08 19:03:58.860806 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:03:58.860334 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 19:03:58.860806 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:03:58.860370 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 19:03:58.860806 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:03:58.860385 2583 projected.go:196] Error preparing data for projected volume kube-api-access-wqh5r for pod openshift-network-diagnostics/network-check-target-z4hnv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:03:58.860806 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:03:58.860470 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7f193dea-aa4a-47c5-8cb2-8f34029b1355-kube-api-access-wqh5r podName:7f193dea-aa4a-47c5-8cb2-8f34029b1355 nodeName:}" failed. No retries permitted until 2026-07-08 19:04:00.860450237 +0000 UTC m=+6.139537851 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-wqh5r" (UniqueName: "kubernetes.io/projected/7f193dea-aa4a-47c5-8cb2-8f34029b1355-kube-api-access-wqh5r") pod "network-check-target-z4hnv" (UID: "7f193dea-aa4a-47c5-8cb2-8f34029b1355") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:03:59.093964 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:59.093846 2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 18:58:57 +0000 UTC" deadline="2028-02-08 10:09:03.113450711 +0000 UTC" Jul 08 19:03:59.093964 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:59.093913 2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13911h5m4.01954194s" Jul 08 19:03:59.171043 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:59.171013 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4hnv" Jul 08 19:03:59.171220 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:03:59.171136 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z4hnv" podUID="7f193dea-aa4a-47c5-8cb2-8f34029b1355" Jul 08 19:03:59.171289 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:59.171223 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qx98w" Jul 08 19:03:59.171341 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:03:59.171299 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qx98w" podUID="9a902ad1-6161-490d-afb2-006966897633" Jul 08 19:03:59.171393 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:03:59.171354 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hs752" Jul 08 19:03:59.171469 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:03:59.171450 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hs752" podUID="6701701d-6ce5-4028-a9ac-9fabc3925522" Jul 08 19:04:00.199336 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:00.199292 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-244.ec2.internal" event={"ID":"09c7821b3b1a8a9c6dd9709ff325da43","Type":"ContainerStarted","Data":"556a3897feea3e123c5899daacd35eeda5bc321c67f8d81b08f9b82d98f25f1d"} Jul 08 19:04:00.203807 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:00.203776 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-244.ec2.internal" event={"ID":"2bde17ed4dbbb226aeb25d822491cdfc","Type":"ContainerStarted","Data":"2fdba673de2526d2691fc3124583a28b1a57d814eaf57aed1da859276c78dc20"} Jul 08 19:04:00.237546 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:00.237491 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-244.ec2.internal" podStartSLOduration=4.237468283 podStartE2EDuration="4.237468283s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:03:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:04:00.219490768 +0000 UTC m=+5.498578388" watchObservedRunningTime="2026-07-08 19:04:00.237468283 +0000 UTC m=+5.516555905" Jul 08 19:04:00.679050 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:00.679011 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9a902ad1-6161-490d-afb2-006966897633-original-pull-secret\") pod \"global-pull-secret-syncer-qx98w\" (UID: \"9a902ad1-6161-490d-afb2-006966897633\") " pod="kube-system/global-pull-secret-syncer-qx98w" Jul 08 19:04:00.679242 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:00.679062 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6701701d-6ce5-4028-a9ac-9fabc3925522-metrics-certs\") pod \"network-metrics-daemon-hs752\" (UID: \"6701701d-6ce5-4028-a9ac-9fabc3925522\") " pod="openshift-multus/network-metrics-daemon-hs752" Jul 08 19:04:00.679242 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:00.679219 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:04:00.679344 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:00.679283 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6701701d-6ce5-4028-a9ac-9fabc3925522-metrics-certs podName:6701701d-6ce5-4028-a9ac-9fabc3925522 nodeName:}" failed. No retries permitted until 2026-07-08 19:04:04.679263788 +0000 UTC m=+9.958351386 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6701701d-6ce5-4028-a9ac-9fabc3925522-metrics-certs") pod "network-metrics-daemon-hs752" (UID: "6701701d-6ce5-4028-a9ac-9fabc3925522") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:04:00.679689 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:00.679669 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 19:04:00.679773 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:00.679729 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a902ad1-6161-490d-afb2-006966897633-original-pull-secret podName:9a902ad1-6161-490d-afb2-006966897633 nodeName:}" failed. No retries permitted until 2026-07-08 19:04:04.679712787 +0000 UTC m=+9.958800388 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9a902ad1-6161-490d-afb2-006966897633-original-pull-secret") pod "global-pull-secret-syncer-qx98w" (UID: "9a902ad1-6161-490d-afb2-006966897633") : object "kube-system"/"original-pull-secret" not registered Jul 08 19:04:00.880708 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:00.880669 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqh5r\" (UniqueName: \"kubernetes.io/projected/7f193dea-aa4a-47c5-8cb2-8f34029b1355-kube-api-access-wqh5r\") pod \"network-check-target-z4hnv\" (UID: \"7f193dea-aa4a-47c5-8cb2-8f34029b1355\") " pod="openshift-network-diagnostics/network-check-target-z4hnv" Jul 08 19:04:00.880898 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:00.880888 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 19:04:00.880970 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:00.880911 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 19:04:00.880970 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:00.880923 2583 projected.go:196] Error preparing data for projected volume kube-api-access-wqh5r for pod openshift-network-diagnostics/network-check-target-z4hnv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:04:00.881072 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:00.880976 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7f193dea-aa4a-47c5-8cb2-8f34029b1355-kube-api-access-wqh5r podName:7f193dea-aa4a-47c5-8cb2-8f34029b1355 nodeName:}" failed. No retries permitted until 2026-07-08 19:04:04.880959022 +0000 UTC m=+10.160046627 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-wqh5r" (UniqueName: "kubernetes.io/projected/7f193dea-aa4a-47c5-8cb2-8f34029b1355-kube-api-access-wqh5r") pod "network-check-target-z4hnv" (UID: "7f193dea-aa4a-47c5-8cb2-8f34029b1355") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:04:01.170471 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:01.169556 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4hnv" Jul 08 19:04:01.170471 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:01.169556 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qx98w" Jul 08 19:04:01.170471 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:01.169665 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z4hnv" podUID="7f193dea-aa4a-47c5-8cb2-8f34029b1355" Jul 08 19:04:01.170471 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:01.169781 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qx98w" podUID="9a902ad1-6161-490d-afb2-006966897633" Jul 08 19:04:01.170471 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:01.169830 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hs752" Jul 08 19:04:01.170471 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:01.169893 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hs752" podUID="6701701d-6ce5-4028-a9ac-9fabc3925522" Jul 08 19:04:01.212732 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:01.212695 2583 generic.go:356] "Generic (PLEG): container finished" podID="2bde17ed4dbbb226aeb25d822491cdfc" containerID="2fdba673de2526d2691fc3124583a28b1a57d814eaf57aed1da859276c78dc20" exitCode=0 Jul 08 19:04:01.213643 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:01.213613 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-244.ec2.internal" event={"ID":"2bde17ed4dbbb226aeb25d822491cdfc","Type":"ContainerDied","Data":"2fdba673de2526d2691fc3124583a28b1a57d814eaf57aed1da859276c78dc20"} Jul 08 19:04:03.170415 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:03.170387 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hs752" Jul 08 19:04:03.170865 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:03.170534 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hs752" podUID="6701701d-6ce5-4028-a9ac-9fabc3925522" Jul 08 19:04:03.170930 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:03.170906 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4hnv" Jul 08 19:04:03.171015 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:03.170996 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z4hnv" podUID="7f193dea-aa4a-47c5-8cb2-8f34029b1355" Jul 08 19:04:03.171089 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:03.171076 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qx98w" Jul 08 19:04:03.171163 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:03.171147 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qx98w" podUID="9a902ad1-6161-490d-afb2-006966897633" Jul 08 19:04:04.711297 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:04.711214 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9a902ad1-6161-490d-afb2-006966897633-original-pull-secret\") pod \"global-pull-secret-syncer-qx98w\" (UID: \"9a902ad1-6161-490d-afb2-006966897633\") " pod="kube-system/global-pull-secret-syncer-qx98w" Jul 08 19:04:04.711297 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:04.711267 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6701701d-6ce5-4028-a9ac-9fabc3925522-metrics-certs\") pod \"network-metrics-daemon-hs752\" (UID: \"6701701d-6ce5-4028-a9ac-9fabc3925522\") " pod="openshift-multus/network-metrics-daemon-hs752" Jul 08 19:04:04.711763 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:04.711392 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 19:04:04.711763 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:04.711409 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:04:04.711763 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:04.711474 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a902ad1-6161-490d-afb2-006966897633-original-pull-secret podName:9a902ad1-6161-490d-afb2-006966897633 nodeName:}" failed. No retries permitted until 2026-07-08 19:04:12.71145915 +0000 UTC m=+17.990546748 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9a902ad1-6161-490d-afb2-006966897633-original-pull-secret") pod "global-pull-secret-syncer-qx98w" (UID: "9a902ad1-6161-490d-afb2-006966897633") : object "kube-system"/"original-pull-secret" not registered Jul 08 19:04:04.711763 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:04.711491 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6701701d-6ce5-4028-a9ac-9fabc3925522-metrics-certs podName:6701701d-6ce5-4028-a9ac-9fabc3925522 nodeName:}" failed. No retries permitted until 2026-07-08 19:04:12.711484008 +0000 UTC m=+17.990571605 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6701701d-6ce5-4028-a9ac-9fabc3925522-metrics-certs") pod "network-metrics-daemon-hs752" (UID: "6701701d-6ce5-4028-a9ac-9fabc3925522") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:04:04.912991 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:04.912953 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqh5r\" (UniqueName: \"kubernetes.io/projected/7f193dea-aa4a-47c5-8cb2-8f34029b1355-kube-api-access-wqh5r\") pod \"network-check-target-z4hnv\" (UID: \"7f193dea-aa4a-47c5-8cb2-8f34029b1355\") " pod="openshift-network-diagnostics/network-check-target-z4hnv" Jul 08 19:04:04.913181 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:04.913144 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 19:04:04.913181 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:04.913162 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 19:04:04.913181 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:04.913176 2583 projected.go:196] Error preparing data for projected volume kube-api-access-wqh5r for pod openshift-network-diagnostics/network-check-target-z4hnv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:04:04.913349 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:04.913224 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7f193dea-aa4a-47c5-8cb2-8f34029b1355-kube-api-access-wqh5r podName:7f193dea-aa4a-47c5-8cb2-8f34029b1355 nodeName:}" failed. No retries permitted until 2026-07-08 19:04:12.913211086 +0000 UTC m=+18.192298684 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-wqh5r" (UniqueName: "kubernetes.io/projected/7f193dea-aa4a-47c5-8cb2-8f34029b1355-kube-api-access-wqh5r") pod "network-check-target-z4hnv" (UID: "7f193dea-aa4a-47c5-8cb2-8f34029b1355") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:04:05.171387 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:05.171314 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4hnv" Jul 08 19:04:05.171387 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:05.171323 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qx98w" Jul 08 19:04:05.171387 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:05.171314 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hs752" Jul 08 19:04:05.171622 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:05.171419 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qx98w" podUID="9a902ad1-6161-490d-afb2-006966897633" Jul 08 19:04:05.171622 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:05.171497 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hs752" podUID="6701701d-6ce5-4028-a9ac-9fabc3925522" Jul 08 19:04:05.171622 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:05.171570 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z4hnv" podUID="7f193dea-aa4a-47c5-8cb2-8f34029b1355" Jul 08 19:04:07.169208 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:07.169180 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4hnv" Jul 08 19:04:07.169208 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:07.169197 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hs752" Jul 08 19:04:07.169590 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:07.169195 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qx98w" Jul 08 19:04:07.169590 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:07.169292 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z4hnv" podUID="7f193dea-aa4a-47c5-8cb2-8f34029b1355" Jul 08 19:04:07.169590 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:07.169392 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hs752" podUID="6701701d-6ce5-4028-a9ac-9fabc3925522" Jul 08 19:04:07.169590 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:07.169495 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qx98w" podUID="9a902ad1-6161-490d-afb2-006966897633" Jul 08 19:04:08.229779 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:08.229709 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mpz48" event={"ID":"fb6af782-3262-467f-95e4-31d549573cd6","Type":"ContainerStarted","Data":"d24a499e377187ec53ff96d72b0a614e731167a49cfb30c4932b350a548b0bdd"} Jul 08 19:04:08.232062 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:08.231954 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qwcgr" event={"ID":"3b8b33cc-c6e1-4b41-9fe3-4ca56a434892","Type":"ContainerStarted","Data":"e93937f9f3b82bf51606c791fa2316acdfd257d01da591d8b475d3328f6f506f"} Jul 08 19:04:08.238397 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:08.238368 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-n8prt" event={"ID":"1033c4ea-5be7-4c7c-9c6c-0bbb663fad60","Type":"ContainerStarted","Data":"86b54212e5e1e65b719e33f58027938ca670a1692ee58ed73c582e9fa53701d5"} Jul 08 19:04:08.244231 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:08.244198 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-244.ec2.internal" event={"ID":"2bde17ed4dbbb226aeb25d822491cdfc","Type":"ContainerStarted","Data":"dbaa3884136975228d35345966ca662d91a9db8010c3bed8c9ce54dd35cec032"} Jul 08 19:04:08.249163 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:08.249140 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zjgc5" event={"ID":"31a4f57f-1bb0-4b8f-bb58-9a78ef1565c6","Type":"ContainerStarted","Data":"5527f506fa4736a88c08cccd57a0f2f4f4aff834c6ad3b9022b0bce1b2925b2a"} Jul 08 19:04:08.252634 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:08.252602 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jw4sj" event={"ID":"bb8c1f0b-7584-4d24-aa13-c73cad10576c","Type":"ContainerStarted","Data":"d6d874f8cc9643e29dddcf792483567fa77bd165e5f57d81f4b33d5e4742fdfe"} Jul 08 19:04:08.281900 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:08.281830 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-244.ec2.internal" podStartSLOduration=12.281814316 podStartE2EDuration="12.281814316s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:03:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:04:08.281635145 +0000 UTC m=+13.560722766" watchObservedRunningTime="2026-07-08 19:04:08.281814316 +0000 UTC m=+13.560901926" Jul 08 19:04:08.301454 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:08.301390 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-jw4sj" podStartSLOduration=2.8171542179999998 podStartE2EDuration="13.301376057s" totalImagesPullingTime="10.484221839s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:03:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:04:08.300899429 +0000 UTC m=+13.579987053" watchObservedRunningTime="2026-07-08 19:04:08.301376057 +0000 UTC m=+13.580463668" Jul 08 19:04:08.317228 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:08.317181 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-n8prt" podStartSLOduration=2.859361352 podStartE2EDuration="13.317164805s" totalImagesPullingTime="10.457803453s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:03:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:04:08.316582376 +0000 UTC m=+13.595669998" watchObservedRunningTime="2026-07-08 19:04:08.317164805 +0000 UTC m=+13.596252424" Jul 08 19:04:08.335689 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:08.335600 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-qwcgr" podStartSLOduration=2.646023146 podStartE2EDuration="13.335586985s" totalImagesPullingTime="10.689563839s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:03:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:04:08.335031438 +0000 UTC m=+13.614119059" watchObservedRunningTime="2026-07-08 19:04:08.335586985 +0000 UTC m=+13.614674605" Jul 08 19:04:09.169308 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:09.169270 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4hnv" Jul 08 19:04:09.169519 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:09.169405 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z4hnv" podUID="7f193dea-aa4a-47c5-8cb2-8f34029b1355" Jul 08 19:04:09.169854 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:09.169835 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qx98w" Jul 08 19:04:09.169953 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:09.169935 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qx98w" podUID="9a902ad1-6161-490d-afb2-006966897633" Jul 08 19:04:09.170060 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:09.170040 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hs752" Jul 08 19:04:09.170186 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:09.170137 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hs752" podUID="6701701d-6ce5-4028-a9ac-9fabc3925522" Jul 08 19:04:09.256758 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:09.256731 2583 generic.go:356] "Generic (PLEG): container finished" podID="fb6af782-3262-467f-95e4-31d549573cd6" containerID="d24a499e377187ec53ff96d72b0a614e731167a49cfb30c4932b350a548b0bdd" exitCode=0 Jul 08 19:04:09.257468 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:09.256815 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mpz48" event={"ID":"fb6af782-3262-467f-95e4-31d549573cd6","Type":"ContainerDied","Data":"d24a499e377187ec53ff96d72b0a614e731167a49cfb30c4932b350a548b0bdd"} Jul 08 19:04:09.259233 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:09.259203 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xh2z8" event={"ID":"8a5d7413-ff2a-47c9-a084-2b8e1939522d","Type":"ContainerStarted","Data":"18cf035cad81c1feb0ccade73e2afee743863414ca7771b44145a0b5e173d37f"} Jul 08 19:04:09.282751 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:09.282695 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-zjgc5" podStartSLOduration=3.856708162 podStartE2EDuration="14.282675114s" totalImagesPullingTime="10.425966952s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:03:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:04:08.35198908 +0000 UTC m=+13.631076701" watchObservedRunningTime="2026-07-08 19:04:09.282675114 +0000 UTC m=+14.561762736" Jul 08 19:04:10.262925 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:10.262884 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mq9pr" event={"ID":"dcd11726-94ec-45c8-980e-9e362ab8c537","Type":"ContainerStarted","Data":"d5c9582598245f57bfaec51ce3f57906bd149f19a578b63980dc38e9df57b04c"} Jul 08 19:04:10.672122 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:10.672066 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-n8prt" Jul 08 19:04:10.672748 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:10.672724 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-n8prt" Jul 08 19:04:10.698447 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:10.698374 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-mq9pr" podStartSLOduration=5.287685955 podStartE2EDuration="15.698352184s" totalImagesPullingTime="10.410666229s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:03:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:04:10.289950409 +0000 UTC m=+15.569038030" watchObservedRunningTime="2026-07-08 19:04:10.698352184 +0000 UTC m=+15.977439805" Jul 08 19:04:11.169122 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:11.169080 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qx98w" Jul 08 19:04:11.169305 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:11.169122 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4hnv" Jul 08 19:04:11.169305 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:11.169212 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qx98w" podUID="9a902ad1-6161-490d-afb2-006966897633" Jul 08 19:04:11.169417 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:11.169296 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z4hnv" podUID="7f193dea-aa4a-47c5-8cb2-8f34029b1355" Jul 08 19:04:11.169417 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:11.169083 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hs752" Jul 08 19:04:11.169417 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:11.169403 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hs752" podUID="6701701d-6ce5-4028-a9ac-9fabc3925522" Jul 08 19:04:11.265474 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:11.265426 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-n8prt" Jul 08 19:04:11.265979 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:11.265846 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-n8prt" Jul 08 19:04:12.772329 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:12.772295 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9a902ad1-6161-490d-afb2-006966897633-original-pull-secret\") pod \"global-pull-secret-syncer-qx98w\" (UID: \"9a902ad1-6161-490d-afb2-006966897633\") " pod="kube-system/global-pull-secret-syncer-qx98w" Jul 08 19:04:12.772329 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:12.772330 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6701701d-6ce5-4028-a9ac-9fabc3925522-metrics-certs\") pod \"network-metrics-daemon-hs752\" (UID: \"6701701d-6ce5-4028-a9ac-9fabc3925522\") " pod="openshift-multus/network-metrics-daemon-hs752" Jul 08 19:04:12.772886 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:12.772463 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 19:04:12.772886 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:12.772528 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a902ad1-6161-490d-afb2-006966897633-original-pull-secret podName:9a902ad1-6161-490d-afb2-006966897633 nodeName:}" failed. No retries permitted until 2026-07-08 19:04:28.77251227 +0000 UTC m=+34.051599873 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9a902ad1-6161-490d-afb2-006966897633-original-pull-secret") pod "global-pull-secret-syncer-qx98w" (UID: "9a902ad1-6161-490d-afb2-006966897633") : object "kube-system"/"original-pull-secret" not registered Jul 08 19:04:12.772886 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:12.772463 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:04:12.772886 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:12.772602 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6701701d-6ce5-4028-a9ac-9fabc3925522-metrics-certs podName:6701701d-6ce5-4028-a9ac-9fabc3925522 nodeName:}" failed. No retries permitted until 2026-07-08 19:04:28.772587151 +0000 UTC m=+34.051674749 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6701701d-6ce5-4028-a9ac-9fabc3925522-metrics-certs") pod "network-metrics-daemon-hs752" (UID: "6701701d-6ce5-4028-a9ac-9fabc3925522") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:04:12.973579 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:12.973534 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqh5r\" (UniqueName: \"kubernetes.io/projected/7f193dea-aa4a-47c5-8cb2-8f34029b1355-kube-api-access-wqh5r\") pod \"network-check-target-z4hnv\" (UID: \"7f193dea-aa4a-47c5-8cb2-8f34029b1355\") " pod="openshift-network-diagnostics/network-check-target-z4hnv" Jul 08 19:04:12.973754 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:12.973730 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 19:04:12.973816 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:12.973755 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 19:04:12.973816 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:12.973770 2583 projected.go:196] Error preparing data for projected volume kube-api-access-wqh5r for pod openshift-network-diagnostics/network-check-target-z4hnv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:04:12.973889 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:12.973836 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7f193dea-aa4a-47c5-8cb2-8f34029b1355-kube-api-access-wqh5r podName:7f193dea-aa4a-47c5-8cb2-8f34029b1355 nodeName:}" failed. No retries permitted until 2026-07-08 19:04:28.973819116 +0000 UTC m=+34.252906736 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-wqh5r" (UniqueName: "kubernetes.io/projected/7f193dea-aa4a-47c5-8cb2-8f34029b1355-kube-api-access-wqh5r") pod "network-check-target-z4hnv" (UID: "7f193dea-aa4a-47c5-8cb2-8f34029b1355") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:04:13.169349 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:13.169134 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hs752" Jul 08 19:04:13.169556 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:13.169139 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4hnv" Jul 08 19:04:13.169556 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:13.169484 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hs752" podUID="6701701d-6ce5-4028-a9ac-9fabc3925522" Jul 08 19:04:13.169556 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:13.169532 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z4hnv" podUID="7f193dea-aa4a-47c5-8cb2-8f34029b1355" Jul 08 19:04:13.169556 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:13.169191 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qx98w" Jul 08 19:04:13.169740 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:13.169613 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qx98w" podUID="9a902ad1-6161-490d-afb2-006966897633" Jul 08 19:04:15.169482 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:15.169428 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4hnv" Jul 08 19:04:15.169482 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:15.169470 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qx98w" Jul 08 19:04:15.170075 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:15.169557 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z4hnv" podUID="7f193dea-aa4a-47c5-8cb2-8f34029b1355" Jul 08 19:04:15.170075 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:15.169946 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qx98w" podUID="9a902ad1-6161-490d-afb2-006966897633" Jul 08 19:04:15.170075 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:15.170004 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hs752" Jul 08 19:04:15.170075 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:15.170084 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hs752" podUID="6701701d-6ce5-4028-a9ac-9fabc3925522" Jul 08 19:04:17.168583 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:17.168545 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qx98w" Jul 08 19:04:17.169117 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:17.168552 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hs752" Jul 08 19:04:17.169117 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:17.168678 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qx98w" podUID="9a902ad1-6161-490d-afb2-006966897633" Jul 08 19:04:17.169117 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:17.168688 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4hnv" Jul 08 19:04:17.169117 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:17.168781 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hs752" podUID="6701701d-6ce5-4028-a9ac-9fabc3925522" Jul 08 19:04:17.169117 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:17.168870 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z4hnv" podUID="7f193dea-aa4a-47c5-8cb2-8f34029b1355" Jul 08 19:04:18.082474 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:18.082450 2583 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 08 19:04:18.107595 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:18.107496 2583 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-08T19:04:18.082470992Z","UUID":"cfaf924c-8c45-4dba-9f03-2075fdcd719e","Handler":null,"Name":"","Endpoint":""} Jul 08 19:04:18.109107 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:18.109083 2583 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 08 19:04:18.109191 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:18.109136 2583 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 08 19:04:18.282355 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:18.282100 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6kmdh" event={"ID":"001f0989-80a3-4b9d-8a57-9f41846bef7b","Type":"ContainerStarted","Data":"7786e88228a731e8bc455dab29d5f0a81a7e1eaad6d6d24d21bce902fcfa0c1b"} Jul 08 19:04:18.284702 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:18.284679 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xh2z8" event={"ID":"8a5d7413-ff2a-47c9-a084-2b8e1939522d","Type":"ContainerStarted","Data":"125633de0cf25a8bb1cf89d050e5aa9091cfbd86c60ca542b49cfb5331304f11"} Jul 08 19:04:18.286987 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:18.286953 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" event={"ID":"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c","Type":"ContainerStarted","Data":"c894c5a89eeeb0c4576f57a045875524f88c9ff70122b29f986373768e9f7553"} Jul 08 19:04:18.287063 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:18.287023 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" event={"ID":"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c","Type":"ContainerStarted","Data":"4418d9278749168ef811a4615ff5cca7cfdd61b98a7c30ae7b9e024a15a7883e"} Jul 08 19:04:18.287063 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:18.287053 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" event={"ID":"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c","Type":"ContainerStarted","Data":"41b1c7b68f51ee7392d7c5f9ef95644b1d236ed34ca22ca4a9cc7d821ec9d9ce"} Jul 08 19:04:18.309715 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:18.309672 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-6kmdh" podStartSLOduration=2.723635217 podStartE2EDuration="23.309655584s" totalImagesPullingTime="20.586020367s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:03:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:04:18.308872497 +0000 UTC m=+23.587960117" watchObservedRunningTime="2026-07-08 19:04:18.309655584 +0000 UTC m=+23.588743210" Jul 08 19:04:19.169197 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:19.169167 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qx98w" Jul 08 19:04:19.169365 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:19.169276 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qx98w" podUID="9a902ad1-6161-490d-afb2-006966897633" Jul 08 19:04:19.169365 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:19.169167 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4hnv" Jul 08 19:04:19.169365 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:19.169308 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hs752" Jul 08 19:04:19.169512 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:19.169403 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z4hnv" podUID="7f193dea-aa4a-47c5-8cb2-8f34029b1355" Jul 08 19:04:19.169554 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:19.169529 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hs752" podUID="6701701d-6ce5-4028-a9ac-9fabc3925522" Jul 08 19:04:19.293245 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:19.293212 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" event={"ID":"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c","Type":"ContainerStarted","Data":"3442fe124f357e9dcab23adf4c6c33f2847d6a0996e8300422650af965f061c9"} Jul 08 19:04:19.293720 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:19.293282 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" event={"ID":"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c","Type":"ContainerStarted","Data":"94f53efae08a7be95770cf925d43df825fb6187a00e52ff462b7f5eb1b7aa359"} Jul 08 19:04:19.293720 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:19.293329 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" event={"ID":"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c","Type":"ContainerStarted","Data":"131aeeeb0b00c9585bdd0d5d96fbea1174c95f5d1fd57c4464d1a8656d4b1513"} Jul 08 19:04:21.168684 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:21.168656 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hs752" Jul 08 19:04:21.169389 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:21.168656 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4hnv" Jul 08 19:04:21.169389 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:21.168774 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hs752" podUID="6701701d-6ce5-4028-a9ac-9fabc3925522" Jul 08 19:04:21.169389 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:21.168886 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z4hnv" podUID="7f193dea-aa4a-47c5-8cb2-8f34029b1355" Jul 08 19:04:21.169389 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:21.168965 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qx98w" Jul 08 19:04:21.169389 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:21.169059 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qx98w" podUID="9a902ad1-6161-490d-afb2-006966897633" Jul 08 19:04:21.300403 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:21.300326 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xh2z8" event={"ID":"8a5d7413-ff2a-47c9-a084-2b8e1939522d","Type":"ContainerStarted","Data":"1692a54e2f1bd7325f9f184ae6aabac709b2f9c042084622fc4e485a12f3dfe7"} Jul 08 19:04:21.303596 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:21.303570 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" event={"ID":"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c","Type":"ContainerStarted","Data":"a8ecee511561c66e9c00b8eee1752a57201b801cec586ff52516c9c70067fcbd"} Jul 08 19:04:21.323530 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:21.323484 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xh2z8" podStartSLOduration=3.188391143 podStartE2EDuration="26.323470943s" totalImagesPullingTime="23.1350798s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:03:55 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:04:21.323259777 +0000 UTC m=+26.602347397" watchObservedRunningTime="2026-07-08 19:04:21.323470943 +0000 UTC m=+26.602558562" Jul 08 19:04:22.307882 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:22.307851 2583 generic.go:356] "Generic (PLEG): container finished" podID="fb6af782-3262-467f-95e4-31d549573cd6" containerID="97a7c41a2efc60e1f1e1ef669c33cea1b7161287cf78e47e2039431919c56a1b" exitCode=0 Jul 08 19:04:22.308300 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:22.307956 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mpz48" event={"ID":"fb6af782-3262-467f-95e4-31d549573cd6","Type":"ContainerDied","Data":"97a7c41a2efc60e1f1e1ef669c33cea1b7161287cf78e47e2039431919c56a1b"} Jul 08 19:04:23.168942 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:23.168909 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qx98w" Jul 08 19:04:23.169090 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:23.169016 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qx98w" podUID="9a902ad1-6161-490d-afb2-006966897633" Jul 08 19:04:23.169090 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:23.168909 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hs752" Jul 08 19:04:23.169090 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:23.169053 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4hnv" Jul 08 19:04:23.169196 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:23.169105 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hs752" podUID="6701701d-6ce5-4028-a9ac-9fabc3925522" Jul 08 19:04:23.169196 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:23.169179 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z4hnv" podUID="7f193dea-aa4a-47c5-8cb2-8f34029b1355" Jul 08 19:04:24.315266 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:24.315066 2583 generic.go:356] "Generic (PLEG): container finished" podID="fb6af782-3262-467f-95e4-31d549573cd6" containerID="4668e6b8ec9d912f4d8fe29874d003ef24af971a97934c97da3f4c3c7b929887" exitCode=0 Jul 08 19:04:24.315854 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:24.315150 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mpz48" event={"ID":"fb6af782-3262-467f-95e4-31d549573cd6","Type":"ContainerDied","Data":"4668e6b8ec9d912f4d8fe29874d003ef24af971a97934c97da3f4c3c7b929887"} Jul 08 19:04:24.318871 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:24.318845 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" event={"ID":"37e62f02-e16f-45b7-8c48-0a2e3b3bd66c","Type":"ContainerStarted","Data":"5cbd46c0b98db521de6e04b1179c7b47fdfad937ecd0c020060486d6cd74db52"} Jul 08 19:04:24.319077 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:24.319058 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:04:24.319206 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:24.319122 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:04:24.335411 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:24.335389 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:04:24.374987 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:24.374934 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" podStartSLOduration=8.744875363 podStartE2EDuration="29.374917646s" totalImagesPullingTime="20.630042283s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:03:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:04:24.373195037 +0000 UTC m=+29.652282657" watchObservedRunningTime="2026-07-08 19:04:24.374917646 +0000 UTC m=+29.654005281" Jul 08 19:04:25.169798 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:25.169768 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4hnv" Jul 08 19:04:25.169980 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:25.169866 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z4hnv" podUID="7f193dea-aa4a-47c5-8cb2-8f34029b1355" Jul 08 19:04:25.169980 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:25.169876 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qx98w" Jul 08 19:04:25.169980 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:25.169902 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hs752" Jul 08 19:04:25.170144 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:25.169990 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qx98w" podUID="9a902ad1-6161-490d-afb2-006966897633" Jul 08 19:04:25.170144 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:25.170065 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hs752" podUID="6701701d-6ce5-4028-a9ac-9fabc3925522" Jul 08 19:04:25.321496 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:25.321315 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:04:25.336856 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:25.336835 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:04:25.468496 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:25.468468 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-z4hnv"] Jul 08 19:04:25.468653 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:25.468597 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4hnv" Jul 08 19:04:25.468724 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:25.468701 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z4hnv" podUID="7f193dea-aa4a-47c5-8cb2-8f34029b1355" Jul 08 19:04:25.471572 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:25.471544 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qx98w"] Jul 08 19:04:25.471667 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:25.471652 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qx98w" Jul 08 19:04:25.471813 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:25.471777 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qx98w" podUID="9a902ad1-6161-490d-afb2-006966897633" Jul 08 19:04:25.472343 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:25.472319 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hs752"] Jul 08 19:04:25.472467 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:25.472424 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hs752" Jul 08 19:04:25.472577 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:25.472558 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hs752" podUID="6701701d-6ce5-4028-a9ac-9fabc3925522" Jul 08 19:04:26.325704 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:26.325671 2583 generic.go:356] "Generic (PLEG): container finished" podID="fb6af782-3262-467f-95e4-31d549573cd6" containerID="a5cbeaeadda2048570670d68f12de8e6b03a0d37fb374240d86130c88f278b81" exitCode=0 Jul 08 19:04:26.326150 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:26.325752 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mpz48" event={"ID":"fb6af782-3262-467f-95e4-31d549573cd6","Type":"ContainerDied","Data":"a5cbeaeadda2048570670d68f12de8e6b03a0d37fb374240d86130c88f278b81"} Jul 08 19:04:27.168541 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:27.168505 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4hnv" Jul 08 19:04:27.168801 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:27.168552 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hs752" Jul 08 19:04:27.168801 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:27.168509 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qx98w" Jul 08 19:04:27.168801 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:27.168647 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z4hnv" podUID="7f193dea-aa4a-47c5-8cb2-8f34029b1355" Jul 08 19:04:27.168801 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:27.168713 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qx98w" podUID="9a902ad1-6161-490d-afb2-006966897633" Jul 08 19:04:27.168801 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:27.168791 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hs752" podUID="6701701d-6ce5-4028-a9ac-9fabc3925522" Jul 08 19:04:28.790595 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:28.790557 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9a902ad1-6161-490d-afb2-006966897633-original-pull-secret\") pod \"global-pull-secret-syncer-qx98w\" (UID: \"9a902ad1-6161-490d-afb2-006966897633\") " pod="kube-system/global-pull-secret-syncer-qx98w" Jul 08 19:04:28.790595 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:28.790595 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6701701d-6ce5-4028-a9ac-9fabc3925522-metrics-certs\") pod \"network-metrics-daemon-hs752\" (UID: \"6701701d-6ce5-4028-a9ac-9fabc3925522\") " pod="openshift-multus/network-metrics-daemon-hs752" Jul 08 19:04:28.791174 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:28.790698 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:04:28.791174 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:28.790699 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 19:04:28.791174 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:28.790749 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6701701d-6ce5-4028-a9ac-9fabc3925522-metrics-certs podName:6701701d-6ce5-4028-a9ac-9fabc3925522 nodeName:}" failed. No retries permitted until 2026-07-08 19:05:00.790735042 +0000 UTC m=+66.069822653 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6701701d-6ce5-4028-a9ac-9fabc3925522-metrics-certs") pod "network-metrics-daemon-hs752" (UID: "6701701d-6ce5-4028-a9ac-9fabc3925522") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:04:28.791174 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:28.790766 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a902ad1-6161-490d-afb2-006966897633-original-pull-secret podName:9a902ad1-6161-490d-afb2-006966897633 nodeName:}" failed. No retries permitted until 2026-07-08 19:05:00.790757484 +0000 UTC m=+66.069845093 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9a902ad1-6161-490d-afb2-006966897633-original-pull-secret") pod "global-pull-secret-syncer-qx98w" (UID: "9a902ad1-6161-490d-afb2-006966897633") : object "kube-system"/"original-pull-secret" not registered Jul 08 19:04:28.991717 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:28.991681 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqh5r\" (UniqueName: \"kubernetes.io/projected/7f193dea-aa4a-47c5-8cb2-8f34029b1355-kube-api-access-wqh5r\") pod \"network-check-target-z4hnv\" (UID: \"7f193dea-aa4a-47c5-8cb2-8f34029b1355\") " pod="openshift-network-diagnostics/network-check-target-z4hnv" Jul 08 19:04:28.991872 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:28.991808 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 19:04:28.991872 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:28.991823 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 19:04:28.991872 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:28.991833 2583 projected.go:196] Error preparing data for projected volume kube-api-access-wqh5r for pod openshift-network-diagnostics/network-check-target-z4hnv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:04:28.991977 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:28.991886 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7f193dea-aa4a-47c5-8cb2-8f34029b1355-kube-api-access-wqh5r podName:7f193dea-aa4a-47c5-8cb2-8f34029b1355 nodeName:}" failed. No retries permitted until 2026-07-08 19:05:00.991870842 +0000 UTC m=+66.270958457 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-wqh5r" (UniqueName: "kubernetes.io/projected/7f193dea-aa4a-47c5-8cb2-8f34029b1355-kube-api-access-wqh5r") pod "network-check-target-z4hnv" (UID: "7f193dea-aa4a-47c5-8cb2-8f34029b1355") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:04:29.168403 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:29.168377 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4hnv" Jul 08 19:04:29.168403 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:29.168503 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z4hnv" podUID="7f193dea-aa4a-47c5-8cb2-8f34029b1355" Jul 08 19:04:29.168724 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:29.168573 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hs752" Jul 08 19:04:29.168724 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:29.168705 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hs752" podUID="6701701d-6ce5-4028-a9ac-9fabc3925522" Jul 08 19:04:29.168804 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:29.168377 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qx98w" Jul 08 19:04:29.168848 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:29.168806 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qx98w" podUID="9a902ad1-6161-490d-afb2-006966897633" Jul 08 19:04:30.486632 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.486605 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-244.ec2.internal" event="NodeReady" Jul 08 19:04:30.487220 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.486759 2583 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 08 19:04:30.528183 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.528142 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6b9496b987-kfw7l"] Jul 08 19:04:30.533162 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.533139 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b9496b987-kfw7l" Jul 08 19:04:30.536661 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.536639 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 08 19:04:30.537019 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.536994 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-zm6q7\"" Jul 08 19:04:30.537135 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.537091 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 08 19:04:30.537856 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.537835 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 08 19:04:30.540052 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.539286 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-cz9s6"] Jul 08 19:04:30.542971 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.542950 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 08 19:04:30.543321 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.543296 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cz9s6" Jul 08 19:04:30.545945 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.545192 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-jd5m2"] Jul 08 19:04:30.546342 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.546325 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 08 19:04:30.547291 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.547066 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 08 19:04:30.547512 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.547490 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xm8m6\"" Jul 08 19:04:30.548583 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.548565 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jd5m2" Jul 08 19:04:30.551052 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.551031 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6b9496b987-kfw7l"] Jul 08 19:04:30.551448 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.551410 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 08 19:04:30.551588 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.551563 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 08 19:04:30.554457 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.553574 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-5l8cc\"" Jul 08 19:04:30.554457 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.553821 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 08 19:04:30.558528 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.558503 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cz9s6"] Jul 08 19:04:30.562172 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.562145 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jd5m2"] Jul 08 19:04:30.604274 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.604232 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/50770248-05eb-4ff9-946f-177c6111ce42-bound-sa-token\") pod \"image-registry-6b9496b987-kfw7l\" (UID: \"50770248-05eb-4ff9-946f-177c6111ce42\") " pod="openshift-image-registry/image-registry-6b9496b987-kfw7l" Jul 08 19:04:30.604463 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.604281 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/50770248-05eb-4ff9-946f-177c6111ce42-registry-certificates\") pod \"image-registry-6b9496b987-kfw7l\" (UID: \"50770248-05eb-4ff9-946f-177c6111ce42\") " pod="openshift-image-registry/image-registry-6b9496b987-kfw7l" Jul 08 19:04:30.604463 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.604333 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/50770248-05eb-4ff9-946f-177c6111ce42-registry-tls\") pod \"image-registry-6b9496b987-kfw7l\" (UID: \"50770248-05eb-4ff9-946f-177c6111ce42\") " pod="openshift-image-registry/image-registry-6b9496b987-kfw7l" Jul 08 19:04:30.604463 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.604361 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/50770248-05eb-4ff9-946f-177c6111ce42-ca-trust-extracted\") pod \"image-registry-6b9496b987-kfw7l\" (UID: \"50770248-05eb-4ff9-946f-177c6111ce42\") " pod="openshift-image-registry/image-registry-6b9496b987-kfw7l" Jul 08 19:04:30.604463 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.604385 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/50770248-05eb-4ff9-946f-177c6111ce42-trusted-ca\") pod \"image-registry-6b9496b987-kfw7l\" (UID: \"50770248-05eb-4ff9-946f-177c6111ce42\") " pod="openshift-image-registry/image-registry-6b9496b987-kfw7l" Jul 08 19:04:30.604463 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.604410 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/50770248-05eb-4ff9-946f-177c6111ce42-installation-pull-secrets\") pod \"image-registry-6b9496b987-kfw7l\" (UID: \"50770248-05eb-4ff9-946f-177c6111ce42\") " pod="openshift-image-registry/image-registry-6b9496b987-kfw7l" Jul 08 19:04:30.604659 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.604476 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/50770248-05eb-4ff9-946f-177c6111ce42-image-registry-private-configuration\") pod \"image-registry-6b9496b987-kfw7l\" (UID: \"50770248-05eb-4ff9-946f-177c6111ce42\") " pod="openshift-image-registry/image-registry-6b9496b987-kfw7l" Jul 08 19:04:30.604659 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.604528 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25rgg\" (UniqueName: \"kubernetes.io/projected/50770248-05eb-4ff9-946f-177c6111ce42-kube-api-access-25rgg\") pod \"image-registry-6b9496b987-kfw7l\" (UID: \"50770248-05eb-4ff9-946f-177c6111ce42\") " pod="openshift-image-registry/image-registry-6b9496b987-kfw7l" Jul 08 19:04:30.705598 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.705557 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25rgg\" (UniqueName: \"kubernetes.io/projected/50770248-05eb-4ff9-946f-177c6111ce42-kube-api-access-25rgg\") pod \"image-registry-6b9496b987-kfw7l\" (UID: \"50770248-05eb-4ff9-946f-177c6111ce42\") " pod="openshift-image-registry/image-registry-6b9496b987-kfw7l" Jul 08 19:04:30.705598 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.705617 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f875980c-42c6-47d0-995b-156a37813723-tmp-dir\") pod \"dns-default-cz9s6\" (UID: \"f875980c-42c6-47d0-995b-156a37813723\") " pod="openshift-dns/dns-default-cz9s6" Jul 08 19:04:30.705868 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.705643 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdz9f\" (UniqueName: \"kubernetes.io/projected/a1ace7c9-8aec-4fb8-992e-e9e2a10b2d77-kube-api-access-zdz9f\") pod \"ingress-canary-jd5m2\" (UID: \"a1ace7c9-8aec-4fb8-992e-e9e2a10b2d77\") " pod="openshift-ingress-canary/ingress-canary-jd5m2" Jul 08 19:04:30.705868 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.705670 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f875980c-42c6-47d0-995b-156a37813723-config-volume\") pod \"dns-default-cz9s6\" (UID: \"f875980c-42c6-47d0-995b-156a37813723\") " pod="openshift-dns/dns-default-cz9s6" Jul 08 19:04:30.705868 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.705826 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/50770248-05eb-4ff9-946f-177c6111ce42-bound-sa-token\") pod \"image-registry-6b9496b987-kfw7l\" (UID: \"50770248-05eb-4ff9-946f-177c6111ce42\") " pod="openshift-image-registry/image-registry-6b9496b987-kfw7l" Jul 08 19:04:30.705868 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.705860 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/50770248-05eb-4ff9-946f-177c6111ce42-registry-certificates\") pod \"image-registry-6b9496b987-kfw7l\" (UID: \"50770248-05eb-4ff9-946f-177c6111ce42\") " pod="openshift-image-registry/image-registry-6b9496b987-kfw7l" Jul 08 19:04:30.706044 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.705885 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f875980c-42c6-47d0-995b-156a37813723-metrics-tls\") pod \"dns-default-cz9s6\" (UID: \"f875980c-42c6-47d0-995b-156a37813723\") " pod="openshift-dns/dns-default-cz9s6" Jul 08 19:04:30.706044 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.705924 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/50770248-05eb-4ff9-946f-177c6111ce42-registry-tls\") pod \"image-registry-6b9496b987-kfw7l\" (UID: \"50770248-05eb-4ff9-946f-177c6111ce42\") " pod="openshift-image-registry/image-registry-6b9496b987-kfw7l" Jul 08 19:04:30.706044 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.705948 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/50770248-05eb-4ff9-946f-177c6111ce42-ca-trust-extracted\") pod \"image-registry-6b9496b987-kfw7l\" (UID: \"50770248-05eb-4ff9-946f-177c6111ce42\") " pod="openshift-image-registry/image-registry-6b9496b987-kfw7l" Jul 08 19:04:30.706044 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.705987 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/50770248-05eb-4ff9-946f-177c6111ce42-trusted-ca\") pod \"image-registry-6b9496b987-kfw7l\" (UID: \"50770248-05eb-4ff9-946f-177c6111ce42\") " pod="openshift-image-registry/image-registry-6b9496b987-kfw7l" Jul 08 19:04:30.706044 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.706011 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/50770248-05eb-4ff9-946f-177c6111ce42-installation-pull-secrets\") pod \"image-registry-6b9496b987-kfw7l\" (UID: \"50770248-05eb-4ff9-946f-177c6111ce42\") " pod="openshift-image-registry/image-registry-6b9496b987-kfw7l" Jul 08 19:04:30.706271 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:30.706051 2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 19:04:30.706271 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:30.706070 2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b9496b987-kfw7l: secret "image-registry-tls" not found Jul 08 19:04:30.706271 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.706071 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a1ace7c9-8aec-4fb8-992e-e9e2a10b2d77-cert\") pod \"ingress-canary-jd5m2\" (UID: \"a1ace7c9-8aec-4fb8-992e-e9e2a10b2d77\") " pod="openshift-ingress-canary/ingress-canary-jd5m2" Jul 08 19:04:30.706271 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.706097 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6qpf\" (UniqueName: \"kubernetes.io/projected/f875980c-42c6-47d0-995b-156a37813723-kube-api-access-d6qpf\") pod \"dns-default-cz9s6\" (UID: \"f875980c-42c6-47d0-995b-156a37813723\") " pod="openshift-dns/dns-default-cz9s6" Jul 08 19:04:30.706271 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:30.706140 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50770248-05eb-4ff9-946f-177c6111ce42-registry-tls podName:50770248-05eb-4ff9-946f-177c6111ce42 nodeName:}" failed. No retries permitted until 2026-07-08 19:04:31.206119304 +0000 UTC m=+36.485206919 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/50770248-05eb-4ff9-946f-177c6111ce42-registry-tls") pod "image-registry-6b9496b987-kfw7l" (UID: "50770248-05eb-4ff9-946f-177c6111ce42") : secret "image-registry-tls" not found Jul 08 19:04:30.706271 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.706163 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/50770248-05eb-4ff9-946f-177c6111ce42-image-registry-private-configuration\") pod \"image-registry-6b9496b987-kfw7l\" (UID: \"50770248-05eb-4ff9-946f-177c6111ce42\") " pod="openshift-image-registry/image-registry-6b9496b987-kfw7l" Jul 08 19:04:30.706559 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.706315 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/50770248-05eb-4ff9-946f-177c6111ce42-ca-trust-extracted\") pod \"image-registry-6b9496b987-kfw7l\" (UID: \"50770248-05eb-4ff9-946f-177c6111ce42\") " pod="openshift-image-registry/image-registry-6b9496b987-kfw7l" Jul 08 19:04:30.706611 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.706597 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/50770248-05eb-4ff9-946f-177c6111ce42-registry-certificates\") pod \"image-registry-6b9496b987-kfw7l\" (UID: \"50770248-05eb-4ff9-946f-177c6111ce42\") " pod="openshift-image-registry/image-registry-6b9496b987-kfw7l" Jul 08 19:04:30.706902 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.706885 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/50770248-05eb-4ff9-946f-177c6111ce42-trusted-ca\") pod \"image-registry-6b9496b987-kfw7l\" (UID: \"50770248-05eb-4ff9-946f-177c6111ce42\") " pod="openshift-image-registry/image-registry-6b9496b987-kfw7l" Jul 08 19:04:30.710877 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.710857 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/50770248-05eb-4ff9-946f-177c6111ce42-image-registry-private-configuration\") pod \"image-registry-6b9496b987-kfw7l\" (UID: \"50770248-05eb-4ff9-946f-177c6111ce42\") " pod="openshift-image-registry/image-registry-6b9496b987-kfw7l" Jul 08 19:04:30.710877 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.710866 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/50770248-05eb-4ff9-946f-177c6111ce42-installation-pull-secrets\") pod \"image-registry-6b9496b987-kfw7l\" (UID: \"50770248-05eb-4ff9-946f-177c6111ce42\") " pod="openshift-image-registry/image-registry-6b9496b987-kfw7l" Jul 08 19:04:30.721285 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.721260 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-dkzqj"] Jul 08 19:04:30.727183 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.727157 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/50770248-05eb-4ff9-946f-177c6111ce42-bound-sa-token\") pod \"image-registry-6b9496b987-kfw7l\" (UID: \"50770248-05eb-4ff9-946f-177c6111ce42\") " pod="openshift-image-registry/image-registry-6b9496b987-kfw7l" Jul 08 19:04:30.727689 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.727672 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-dkzqj" Jul 08 19:04:30.729178 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.729150 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-25rgg\" (UniqueName: \"kubernetes.io/projected/50770248-05eb-4ff9-946f-177c6111ce42-kube-api-access-25rgg\") pod \"image-registry-6b9496b987-kfw7l\" (UID: \"50770248-05eb-4ff9-946f-177c6111ce42\") " pod="openshift-image-registry/image-registry-6b9496b987-kfw7l" Jul 08 19:04:30.731126 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.730742 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6ts4z\"" Jul 08 19:04:30.731126 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.730875 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 08 19:04:30.731126 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.730972 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 08 19:04:30.731126 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.731000 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 08 19:04:30.731379 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.731157 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 08 19:04:30.742398 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.742374 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-dkzqj"] Jul 08 19:04:30.806925 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.806880 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f875980c-42c6-47d0-995b-156a37813723-metrics-tls\") pod \"dns-default-cz9s6\" (UID: \"f875980c-42c6-47d0-995b-156a37813723\") " pod="openshift-dns/dns-default-cz9s6" Jul 08 19:04:30.807114 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.806951 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/03c97b5b-1c6e-4461-9fed-e14c43705ddb-crio-socket\") pod \"insights-runtime-extractor-dkzqj\" (UID: \"03c97b5b-1c6e-4461-9fed-e14c43705ddb\") " pod="openshift-insights/insights-runtime-extractor-dkzqj" Jul 08 19:04:30.807114 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.806985 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b56lb\" (UniqueName: \"kubernetes.io/projected/03c97b5b-1c6e-4461-9fed-e14c43705ddb-kube-api-access-b56lb\") pod \"insights-runtime-extractor-dkzqj\" (UID: \"03c97b5b-1c6e-4461-9fed-e14c43705ddb\") " pod="openshift-insights/insights-runtime-extractor-dkzqj" Jul 08 19:04:30.807114 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.807025 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a1ace7c9-8aec-4fb8-992e-e9e2a10b2d77-cert\") pod \"ingress-canary-jd5m2\" (UID: \"a1ace7c9-8aec-4fb8-992e-e9e2a10b2d77\") " pod="openshift-ingress-canary/ingress-canary-jd5m2" Jul 08 19:04:30.807114 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:30.807026 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 19:04:30.807310 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:30.807155 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f875980c-42c6-47d0-995b-156a37813723-metrics-tls podName:f875980c-42c6-47d0-995b-156a37813723 nodeName:}" failed. No retries permitted until 2026-07-08 19:04:31.307130466 +0000 UTC m=+36.586218076 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f875980c-42c6-47d0-995b-156a37813723-metrics-tls") pod "dns-default-cz9s6" (UID: "f875980c-42c6-47d0-995b-156a37813723") : secret "dns-default-metrics-tls" not found Jul 08 19:04:30.807310 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.807053 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6qpf\" (UniqueName: \"kubernetes.io/projected/f875980c-42c6-47d0-995b-156a37813723-kube-api-access-d6qpf\") pod \"dns-default-cz9s6\" (UID: \"f875980c-42c6-47d0-995b-156a37813723\") " pod="openshift-dns/dns-default-cz9s6" Jul 08 19:04:30.807310 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.807221 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/03c97b5b-1c6e-4461-9fed-e14c43705ddb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dkzqj\" (UID: \"03c97b5b-1c6e-4461-9fed-e14c43705ddb\") " pod="openshift-insights/insights-runtime-extractor-dkzqj" Jul 08 19:04:30.807310 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.807263 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/03c97b5b-1c6e-4461-9fed-e14c43705ddb-data-volume\") pod \"insights-runtime-extractor-dkzqj\" (UID: \"03c97b5b-1c6e-4461-9fed-e14c43705ddb\") " pod="openshift-insights/insights-runtime-extractor-dkzqj" Jul 08 19:04:30.807310 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.807285 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f875980c-42c6-47d0-995b-156a37813723-tmp-dir\") pod \"dns-default-cz9s6\" (UID: \"f875980c-42c6-47d0-995b-156a37813723\") " pod="openshift-dns/dns-default-cz9s6" Jul 08 19:04:30.807600 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.807351 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdz9f\" (UniqueName: \"kubernetes.io/projected/a1ace7c9-8aec-4fb8-992e-e9e2a10b2d77-kube-api-access-zdz9f\") pod \"ingress-canary-jd5m2\" (UID: \"a1ace7c9-8aec-4fb8-992e-e9e2a10b2d77\") " pod="openshift-ingress-canary/ingress-canary-jd5m2" Jul 08 19:04:30.807600 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.807416 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f875980c-42c6-47d0-995b-156a37813723-config-volume\") pod \"dns-default-cz9s6\" (UID: \"f875980c-42c6-47d0-995b-156a37813723\") " pod="openshift-dns/dns-default-cz9s6" Jul 08 19:04:30.807600 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.807529 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/03c97b5b-1c6e-4461-9fed-e14c43705ddb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dkzqj\" (UID: \"03c97b5b-1c6e-4461-9fed-e14c43705ddb\") " pod="openshift-insights/insights-runtime-extractor-dkzqj" Jul 08 19:04:30.807735 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.807603 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f875980c-42c6-47d0-995b-156a37813723-tmp-dir\") pod \"dns-default-cz9s6\" (UID: \"f875980c-42c6-47d0-995b-156a37813723\") " pod="openshift-dns/dns-default-cz9s6" Jul 08 19:04:30.807997 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.807979 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f875980c-42c6-47d0-995b-156a37813723-config-volume\") pod \"dns-default-cz9s6\" (UID: \"f875980c-42c6-47d0-995b-156a37813723\") " pod="openshift-dns/dns-default-cz9s6" Jul 08 19:04:30.809879 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.809857 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a1ace7c9-8aec-4fb8-992e-e9e2a10b2d77-cert\") pod \"ingress-canary-jd5m2\" (UID: \"a1ace7c9-8aec-4fb8-992e-e9e2a10b2d77\") " pod="openshift-ingress-canary/ingress-canary-jd5m2" Jul 08 19:04:30.817933 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.817905 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6qpf\" (UniqueName: \"kubernetes.io/projected/f875980c-42c6-47d0-995b-156a37813723-kube-api-access-d6qpf\") pod \"dns-default-cz9s6\" (UID: \"f875980c-42c6-47d0-995b-156a37813723\") " pod="openshift-dns/dns-default-cz9s6" Jul 08 19:04:30.828739 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.828701 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdz9f\" (UniqueName: \"kubernetes.io/projected/a1ace7c9-8aec-4fb8-992e-e9e2a10b2d77-kube-api-access-zdz9f\") pod \"ingress-canary-jd5m2\" (UID: \"a1ace7c9-8aec-4fb8-992e-e9e2a10b2d77\") " pod="openshift-ingress-canary/ingress-canary-jd5m2" Jul 08 19:04:30.835605 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.835570 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-mvdjl"] Jul 08 19:04:30.841044 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.841025 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-mvdjl" Jul 08 19:04:30.843926 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.843908 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 08 19:04:30.844111 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.844084 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-mb77w\"" Jul 08 19:04:30.845222 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.845200 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 08 19:04:30.851264 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.851236 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-mvdjl"] Jul 08 19:04:30.867864 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.867842 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jd5m2" Jul 08 19:04:30.908834 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.908800 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/03c97b5b-1c6e-4461-9fed-e14c43705ddb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dkzqj\" (UID: \"03c97b5b-1c6e-4461-9fed-e14c43705ddb\") " pod="openshift-insights/insights-runtime-extractor-dkzqj" Jul 08 19:04:30.908989 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.908844 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/03c97b5b-1c6e-4461-9fed-e14c43705ddb-data-volume\") pod \"insights-runtime-extractor-dkzqj\" (UID: \"03c97b5b-1c6e-4461-9fed-e14c43705ddb\") " pod="openshift-insights/insights-runtime-extractor-dkzqj" Jul 08 19:04:30.909053 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:30.908989 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 19:04:30.909104 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:30.909072 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03c97b5b-1c6e-4461-9fed-e14c43705ddb-insights-runtime-extractor-tls podName:03c97b5b-1c6e-4461-9fed-e14c43705ddb nodeName:}" failed. No retries permitted until 2026-07-08 19:04:31.40905039 +0000 UTC m=+36.688137997 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/03c97b5b-1c6e-4461-9fed-e14c43705ddb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dkzqj" (UID: "03c97b5b-1c6e-4461-9fed-e14c43705ddb") : secret "insights-runtime-extractor-tls" not found Jul 08 19:04:30.909147 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.909117 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/03c97b5b-1c6e-4461-9fed-e14c43705ddb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dkzqj\" (UID: \"03c97b5b-1c6e-4461-9fed-e14c43705ddb\") " pod="openshift-insights/insights-runtime-extractor-dkzqj" Jul 08 19:04:30.909177 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.909157 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/03c97b5b-1c6e-4461-9fed-e14c43705ddb-data-volume\") pod \"insights-runtime-extractor-dkzqj\" (UID: \"03c97b5b-1c6e-4461-9fed-e14c43705ddb\") " pod="openshift-insights/insights-runtime-extractor-dkzqj" Jul 08 19:04:30.909263 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.909159 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/03c97b5b-1c6e-4461-9fed-e14c43705ddb-crio-socket\") pod \"insights-runtime-extractor-dkzqj\" (UID: \"03c97b5b-1c6e-4461-9fed-e14c43705ddb\") " pod="openshift-insights/insights-runtime-extractor-dkzqj" Jul 08 19:04:30.909263 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.909220 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b56lb\" (UniqueName: \"kubernetes.io/projected/03c97b5b-1c6e-4461-9fed-e14c43705ddb-kube-api-access-b56lb\") pod \"insights-runtime-extractor-dkzqj\" (UID: \"03c97b5b-1c6e-4461-9fed-e14c43705ddb\") " pod="openshift-insights/insights-runtime-extractor-dkzqj" Jul 08 19:04:30.909263 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:30.909232 2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 19:04:30.909538 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:30.909290 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/03c97b5b-1c6e-4461-9fed-e14c43705ddb-kube-rbac-proxy-cm podName:03c97b5b-1c6e-4461-9fed-e14c43705ddb nodeName:}" failed. No retries permitted until 2026-07-08 19:04:31.409275617 +0000 UTC m=+36.688363235 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/03c97b5b-1c6e-4461-9fed-e14c43705ddb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-dkzqj" (UID: "03c97b5b-1c6e-4461-9fed-e14c43705ddb") : configmap "kube-rbac-proxy" not found Jul 08 19:04:30.909538 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.909329 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/03c97b5b-1c6e-4461-9fed-e14c43705ddb-crio-socket\") pod \"insights-runtime-extractor-dkzqj\" (UID: \"03c97b5b-1c6e-4461-9fed-e14c43705ddb\") " pod="openshift-insights/insights-runtime-extractor-dkzqj" Jul 08 19:04:30.930017 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:30.929992 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b56lb\" (UniqueName: \"kubernetes.io/projected/03c97b5b-1c6e-4461-9fed-e14c43705ddb-kube-api-access-b56lb\") pod \"insights-runtime-extractor-dkzqj\" (UID: \"03c97b5b-1c6e-4461-9fed-e14c43705ddb\") " pod="openshift-insights/insights-runtime-extractor-dkzqj" Jul 08 19:04:31.010321 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:31.010237 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbwpq\" (UniqueName: \"kubernetes.io/projected/26c73a09-c9a6-4f28-b811-c3cd7f0e3fb6-kube-api-access-dbwpq\") pod \"migrator-d57d58cc9-mvdjl\" (UID: \"26c73a09-c9a6-4f28-b811-c3cd7f0e3fb6\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-mvdjl" Jul 08 19:04:31.110820 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:31.110784 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbwpq\" (UniqueName: \"kubernetes.io/projected/26c73a09-c9a6-4f28-b811-c3cd7f0e3fb6-kube-api-access-dbwpq\") pod \"migrator-d57d58cc9-mvdjl\" (UID: \"26c73a09-c9a6-4f28-b811-c3cd7f0e3fb6\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-mvdjl" Jul 08 19:04:31.121344 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:31.121307 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbwpq\" (UniqueName: \"kubernetes.io/projected/26c73a09-c9a6-4f28-b811-c3cd7f0e3fb6-kube-api-access-dbwpq\") pod \"migrator-d57d58cc9-mvdjl\" (UID: \"26c73a09-c9a6-4f28-b811-c3cd7f0e3fb6\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-mvdjl" Jul 08 19:04:31.152479 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:31.152444 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-mvdjl" Jul 08 19:04:31.168636 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:31.168567 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4hnv" Jul 08 19:04:31.168636 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:31.168582 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hs752" Jul 08 19:04:31.168636 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:31.168574 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qx98w" Jul 08 19:04:31.173653 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:31.173630 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-kpjrj\"" Jul 08 19:04:31.173779 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:31.173630 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 19:04:31.174128 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:31.174102 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 19:04:31.174214 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:31.174168 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gzqsf\"" Jul 08 19:04:31.174214 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:31.174200 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 19:04:31.174314 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:31.174201 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 19:04:31.211693 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:31.211666 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/50770248-05eb-4ff9-946f-177c6111ce42-registry-tls\") pod \"image-registry-6b9496b987-kfw7l\" (UID: \"50770248-05eb-4ff9-946f-177c6111ce42\") " pod="openshift-image-registry/image-registry-6b9496b987-kfw7l" Jul 08 19:04:31.211816 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:31.211782 2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 19:04:31.211816 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:31.211795 2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b9496b987-kfw7l: secret "image-registry-tls" not found Jul 08 19:04:31.211936 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:31.211845 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50770248-05eb-4ff9-946f-177c6111ce42-registry-tls podName:50770248-05eb-4ff9-946f-177c6111ce42 nodeName:}" failed. No retries permitted until 2026-07-08 19:04:32.211827582 +0000 UTC m=+37.490915196 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/50770248-05eb-4ff9-946f-177c6111ce42-registry-tls") pod "image-registry-6b9496b987-kfw7l" (UID: "50770248-05eb-4ff9-946f-177c6111ce42") : secret "image-registry-tls" not found Jul 08 19:04:31.313246 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:31.313209 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f875980c-42c6-47d0-995b-156a37813723-metrics-tls\") pod \"dns-default-cz9s6\" (UID: \"f875980c-42c6-47d0-995b-156a37813723\") " pod="openshift-dns/dns-default-cz9s6" Jul 08 19:04:31.313420 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:31.313369 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 19:04:31.313522 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:31.313465 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f875980c-42c6-47d0-995b-156a37813723-metrics-tls podName:f875980c-42c6-47d0-995b-156a37813723 nodeName:}" failed. No retries permitted until 2026-07-08 19:04:32.31342051 +0000 UTC m=+37.592508122 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f875980c-42c6-47d0-995b-156a37813723-metrics-tls") pod "dns-default-cz9s6" (UID: "f875980c-42c6-47d0-995b-156a37813723") : secret "dns-default-metrics-tls" not found Jul 08 19:04:31.414049 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:31.414021 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/03c97b5b-1c6e-4461-9fed-e14c43705ddb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dkzqj\" (UID: \"03c97b5b-1c6e-4461-9fed-e14c43705ddb\") " pod="openshift-insights/insights-runtime-extractor-dkzqj" Jul 08 19:04:31.414170 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:31.414102 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/03c97b5b-1c6e-4461-9fed-e14c43705ddb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dkzqj\" (UID: \"03c97b5b-1c6e-4461-9fed-e14c43705ddb\") " pod="openshift-insights/insights-runtime-extractor-dkzqj" Jul 08 19:04:31.414170 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:31.414147 2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 19:04:31.414274 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:31.414228 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/03c97b5b-1c6e-4461-9fed-e14c43705ddb-kube-rbac-proxy-cm podName:03c97b5b-1c6e-4461-9fed-e14c43705ddb nodeName:}" failed. No retries permitted until 2026-07-08 19:04:32.414203967 +0000 UTC m=+37.693291572 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/03c97b5b-1c6e-4461-9fed-e14c43705ddb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-dkzqj" (UID: "03c97b5b-1c6e-4461-9fed-e14c43705ddb") : configmap "kube-rbac-proxy" not found Jul 08 19:04:31.414491 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:31.414475 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 19:04:31.414550 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:31.414536 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03c97b5b-1c6e-4461-9fed-e14c43705ddb-insights-runtime-extractor-tls podName:03c97b5b-1c6e-4461-9fed-e14c43705ddb nodeName:}" failed. No retries permitted until 2026-07-08 19:04:32.414517881 +0000 UTC m=+37.693605480 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/03c97b5b-1c6e-4461-9fed-e14c43705ddb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dkzqj" (UID: "03c97b5b-1c6e-4461-9fed-e14c43705ddb") : secret "insights-runtime-extractor-tls" not found Jul 08 19:04:31.424864 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:31.424844 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jd5m2"] Jul 08 19:04:31.429468 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:31.429448 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-mvdjl"] Jul 08 19:04:31.431839 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:04:31.431147 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1ace7c9_8aec_4fb8_992e_e9e2a10b2d77.slice/crio-6ea84790fc8a64e9604c9d1f696bbc5ba45366cd502bf1894b282bcb55686e70 WatchSource:0}: Error finding container 6ea84790fc8a64e9604c9d1f696bbc5ba45366cd502bf1894b282bcb55686e70: Status 404 returned error can't find the container with id 6ea84790fc8a64e9604c9d1f696bbc5ba45366cd502bf1894b282bcb55686e70 Jul 08 19:04:31.434704 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:04:31.434665 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26c73a09_c9a6_4f28_b811_c3cd7f0e3fb6.slice/crio-abb0c91d217bf6c9e0ab30bf6f509f0c2a49c626baf7e80ef1f7aa51cf6e4cf5 WatchSource:0}: Error finding container abb0c91d217bf6c9e0ab30bf6f509f0c2a49c626baf7e80ef1f7aa51cf6e4cf5: Status 404 returned error can't find the container with id abb0c91d217bf6c9e0ab30bf6f509f0c2a49c626baf7e80ef1f7aa51cf6e4cf5 Jul 08 19:04:32.220139 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:32.220100 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/50770248-05eb-4ff9-946f-177c6111ce42-registry-tls\") pod \"image-registry-6b9496b987-kfw7l\" (UID: \"50770248-05eb-4ff9-946f-177c6111ce42\") " pod="openshift-image-registry/image-registry-6b9496b987-kfw7l" Jul 08 19:04:32.220549 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:32.220265 2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 19:04:32.220549 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:32.220285 2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b9496b987-kfw7l: secret "image-registry-tls" not found Jul 08 19:04:32.220549 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:32.220347 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50770248-05eb-4ff9-946f-177c6111ce42-registry-tls podName:50770248-05eb-4ff9-946f-177c6111ce42 nodeName:}" failed. No retries permitted until 2026-07-08 19:04:34.220327299 +0000 UTC m=+39.499414911 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/50770248-05eb-4ff9-946f-177c6111ce42-registry-tls") pod "image-registry-6b9496b987-kfw7l" (UID: "50770248-05eb-4ff9-946f-177c6111ce42") : secret "image-registry-tls" not found Jul 08 19:04:32.320975 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:32.320916 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f875980c-42c6-47d0-995b-156a37813723-metrics-tls\") pod \"dns-default-cz9s6\" (UID: \"f875980c-42c6-47d0-995b-156a37813723\") " pod="openshift-dns/dns-default-cz9s6" Jul 08 19:04:32.321122 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:32.321077 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 19:04:32.321177 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:32.321157 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f875980c-42c6-47d0-995b-156a37813723-metrics-tls podName:f875980c-42c6-47d0-995b-156a37813723 nodeName:}" failed. No retries permitted until 2026-07-08 19:04:34.321140307 +0000 UTC m=+39.600227909 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f875980c-42c6-47d0-995b-156a37813723-metrics-tls") pod "dns-default-cz9s6" (UID: "f875980c-42c6-47d0-995b-156a37813723") : secret "dns-default-metrics-tls" not found Jul 08 19:04:32.345491 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:32.345452 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jd5m2" event={"ID":"a1ace7c9-8aec-4fb8-992e-e9e2a10b2d77","Type":"ContainerStarted","Data":"6ea84790fc8a64e9604c9d1f696bbc5ba45366cd502bf1894b282bcb55686e70"} Jul 08 19:04:32.348672 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:32.348640 2583 generic.go:356] "Generic (PLEG): container finished" podID="fb6af782-3262-467f-95e4-31d549573cd6" containerID="3da9e75dded67a97f749d676a70eff9929b1bd54e8a86a0ba6d71bbc9f22369c" exitCode=0 Jul 08 19:04:32.348793 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:32.348720 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mpz48" event={"ID":"fb6af782-3262-467f-95e4-31d549573cd6","Type":"ContainerDied","Data":"3da9e75dded67a97f749d676a70eff9929b1bd54e8a86a0ba6d71bbc9f22369c"} Jul 08 19:04:32.349935 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:32.349913 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-mvdjl" event={"ID":"26c73a09-c9a6-4f28-b811-c3cd7f0e3fb6","Type":"ContainerStarted","Data":"abb0c91d217bf6c9e0ab30bf6f509f0c2a49c626baf7e80ef1f7aa51cf6e4cf5"} Jul 08 19:04:32.421375 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:32.421344 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/03c97b5b-1c6e-4461-9fed-e14c43705ddb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dkzqj\" (UID: \"03c97b5b-1c6e-4461-9fed-e14c43705ddb\") " pod="openshift-insights/insights-runtime-extractor-dkzqj" Jul 08 19:04:32.421546 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:32.421467 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/03c97b5b-1c6e-4461-9fed-e14c43705ddb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dkzqj\" (UID: \"03c97b5b-1c6e-4461-9fed-e14c43705ddb\") " pod="openshift-insights/insights-runtime-extractor-dkzqj" Jul 08 19:04:32.421546 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:32.421529 2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 19:04:32.421669 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:32.421603 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/03c97b5b-1c6e-4461-9fed-e14c43705ddb-kube-rbac-proxy-cm podName:03c97b5b-1c6e-4461-9fed-e14c43705ddb nodeName:}" failed. No retries permitted until 2026-07-08 19:04:34.421582355 +0000 UTC m=+39.700669969 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/03c97b5b-1c6e-4461-9fed-e14c43705ddb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-dkzqj" (UID: "03c97b5b-1c6e-4461-9fed-e14c43705ddb") : configmap "kube-rbac-proxy" not found Jul 08 19:04:32.421974 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:32.421948 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 19:04:32.422086 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:32.422013 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03c97b5b-1c6e-4461-9fed-e14c43705ddb-insights-runtime-extractor-tls podName:03c97b5b-1c6e-4461-9fed-e14c43705ddb nodeName:}" failed. No retries permitted until 2026-07-08 19:04:34.421996178 +0000 UTC m=+39.701083805 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/03c97b5b-1c6e-4461-9fed-e14c43705ddb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dkzqj" (UID: "03c97b5b-1c6e-4461-9fed-e14c43705ddb") : secret "insights-runtime-extractor-tls" not found Jul 08 19:04:32.502041 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:32.502016 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zjgc5_31a4f57f-1bb0-4b8f-bb58-9a78ef1565c6/dns-node-resolver/0.log" Jul 08 19:04:32.938069 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:32.938038 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-swjnd"] Jul 08 19:04:32.959539 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:32.959512 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-swjnd"] Jul 08 19:04:32.959684 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:32.959619 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-swjnd" Jul 08 19:04:32.962682 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:32.962654 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 08 19:04:32.962817 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:32.962781 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 08 19:04:32.964417 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:32.963693 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 08 19:04:32.964417 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:32.963712 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 08 19:04:32.964417 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:32.963687 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-52jcx\"" Jul 08 19:04:33.126754 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:33.126719 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/0ecaa5ea-9362-419c-8ec6-60a84115d94a-signing-cabundle\") pod \"service-ca-758f599c5-swjnd\" (UID: \"0ecaa5ea-9362-419c-8ec6-60a84115d94a\") " pod="openshift-service-ca/service-ca-758f599c5-swjnd" Jul 08 19:04:33.126919 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:33.126786 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/0ecaa5ea-9362-419c-8ec6-60a84115d94a-signing-key\") pod \"service-ca-758f599c5-swjnd\" (UID: \"0ecaa5ea-9362-419c-8ec6-60a84115d94a\") " pod="openshift-service-ca/service-ca-758f599c5-swjnd" Jul 08 19:04:33.126919 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:33.126904 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l56l9\" (UniqueName: \"kubernetes.io/projected/0ecaa5ea-9362-419c-8ec6-60a84115d94a-kube-api-access-l56l9\") pod \"service-ca-758f599c5-swjnd\" (UID: \"0ecaa5ea-9362-419c-8ec6-60a84115d94a\") " pod="openshift-service-ca/service-ca-758f599c5-swjnd" Jul 08 19:04:33.227343 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:33.227246 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l56l9\" (UniqueName: \"kubernetes.io/projected/0ecaa5ea-9362-419c-8ec6-60a84115d94a-kube-api-access-l56l9\") pod \"service-ca-758f599c5-swjnd\" (UID: \"0ecaa5ea-9362-419c-8ec6-60a84115d94a\") " pod="openshift-service-ca/service-ca-758f599c5-swjnd" Jul 08 19:04:33.227761 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:33.227344 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/0ecaa5ea-9362-419c-8ec6-60a84115d94a-signing-cabundle\") pod \"service-ca-758f599c5-swjnd\" (UID: \"0ecaa5ea-9362-419c-8ec6-60a84115d94a\") " pod="openshift-service-ca/service-ca-758f599c5-swjnd" Jul 08 19:04:33.227761 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:33.227386 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/0ecaa5ea-9362-419c-8ec6-60a84115d94a-signing-key\") pod \"service-ca-758f599c5-swjnd\" (UID: \"0ecaa5ea-9362-419c-8ec6-60a84115d94a\") " pod="openshift-service-ca/service-ca-758f599c5-swjnd" Jul 08 19:04:33.228079 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:33.228057 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/0ecaa5ea-9362-419c-8ec6-60a84115d94a-signing-cabundle\") pod \"service-ca-758f599c5-swjnd\" (UID: \"0ecaa5ea-9362-419c-8ec6-60a84115d94a\") " pod="openshift-service-ca/service-ca-758f599c5-swjnd" Jul 08 19:04:33.231923 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:33.231900 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/0ecaa5ea-9362-419c-8ec6-60a84115d94a-signing-key\") pod \"service-ca-758f599c5-swjnd\" (UID: \"0ecaa5ea-9362-419c-8ec6-60a84115d94a\") " pod="openshift-service-ca/service-ca-758f599c5-swjnd" Jul 08 19:04:33.237063 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:33.237043 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l56l9\" (UniqueName: \"kubernetes.io/projected/0ecaa5ea-9362-419c-8ec6-60a84115d94a-kube-api-access-l56l9\") pod \"service-ca-758f599c5-swjnd\" (UID: \"0ecaa5ea-9362-419c-8ec6-60a84115d94a\") " pod="openshift-service-ca/service-ca-758f599c5-swjnd" Jul 08 19:04:33.271272 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:33.271236 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-swjnd" Jul 08 19:04:33.356876 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:33.356848 2583 generic.go:356] "Generic (PLEG): container finished" podID="fb6af782-3262-467f-95e4-31d549573cd6" containerID="28d7399ea78bd883d358e76afad5d4f7cc9ed98f1750d6a1a9047d491b452750" exitCode=0 Jul 08 19:04:33.357054 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:33.356909 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mpz48" event={"ID":"fb6af782-3262-467f-95e4-31d549573cd6","Type":"ContainerDied","Data":"28d7399ea78bd883d358e76afad5d4f7cc9ed98f1750d6a1a9047d491b452750"} Jul 08 19:04:33.417580 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:33.417546 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-swjnd"] Jul 08 19:04:33.500313 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:33.500247 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jw4sj_bb8c1f0b-7584-4d24-aa13-c73cad10576c/node-ca/0.log" Jul 08 19:04:33.512164 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:04:33.512132 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ecaa5ea_9362_419c_8ec6_60a84115d94a.slice/crio-6793ba40e32eef0df2005ffa9837128ac05200d4421ee8deb0b26af57cb73827 WatchSource:0}: Error finding container 6793ba40e32eef0df2005ffa9837128ac05200d4421ee8deb0b26af57cb73827: Status 404 returned error can't find the container with id 6793ba40e32eef0df2005ffa9837128ac05200d4421ee8deb0b26af57cb73827 Jul 08 19:04:34.236702 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:34.236635 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/50770248-05eb-4ff9-946f-177c6111ce42-registry-tls\") pod \"image-registry-6b9496b987-kfw7l\" (UID: \"50770248-05eb-4ff9-946f-177c6111ce42\") " pod="openshift-image-registry/image-registry-6b9496b987-kfw7l" Jul 08 19:04:34.237382 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:34.236819 2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 19:04:34.237382 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:34.236853 2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b9496b987-kfw7l: secret "image-registry-tls" not found Jul 08 19:04:34.237382 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:34.236904 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50770248-05eb-4ff9-946f-177c6111ce42-registry-tls podName:50770248-05eb-4ff9-946f-177c6111ce42 nodeName:}" failed. No retries permitted until 2026-07-08 19:04:38.236888652 +0000 UTC m=+43.515976250 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/50770248-05eb-4ff9-946f-177c6111ce42-registry-tls") pod "image-registry-6b9496b987-kfw7l" (UID: "50770248-05eb-4ff9-946f-177c6111ce42") : secret "image-registry-tls" not found Jul 08 19:04:34.338051 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:34.338021 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f875980c-42c6-47d0-995b-156a37813723-metrics-tls\") pod \"dns-default-cz9s6\" (UID: \"f875980c-42c6-47d0-995b-156a37813723\") " pod="openshift-dns/dns-default-cz9s6" Jul 08 19:04:34.338237 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:34.338149 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 19:04:34.338237 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:34.338219 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f875980c-42c6-47d0-995b-156a37813723-metrics-tls podName:f875980c-42c6-47d0-995b-156a37813723 nodeName:}" failed. No retries permitted until 2026-07-08 19:04:38.338200731 +0000 UTC m=+43.617288329 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f875980c-42c6-47d0-995b-156a37813723-metrics-tls") pod "dns-default-cz9s6" (UID: "f875980c-42c6-47d0-995b-156a37813723") : secret "dns-default-metrics-tls" not found Jul 08 19:04:34.364330 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:34.364293 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mpz48" event={"ID":"fb6af782-3262-467f-95e4-31d549573cd6","Type":"ContainerStarted","Data":"ac5fee8001c244ed28eb05fb754e28ba6365efc2a2864eca6a5d9d6f8012e828"} Jul 08 19:04:34.365675 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:34.365644 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-swjnd" event={"ID":"0ecaa5ea-9362-419c-8ec6-60a84115d94a","Type":"ContainerStarted","Data":"6793ba40e32eef0df2005ffa9837128ac05200d4421ee8deb0b26af57cb73827"} Jul 08 19:04:34.367302 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:34.367272 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-mvdjl" event={"ID":"26c73a09-c9a6-4f28-b811-c3cd7f0e3fb6","Type":"ContainerStarted","Data":"ea03f7def11e0ae2021ebc6385aa01d3ed8d6436def2f8b7265aec4d076a69b2"} Jul 08 19:04:34.392833 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:34.392781 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-mpz48" podStartSLOduration=9.493702096 podStartE2EDuration="39.3927655s" totalImagesPullingTime="29.899063404s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:03:55 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:04:34.391261456 +0000 UTC m=+39.670349075" watchObservedRunningTime="2026-07-08 19:04:34.3927655 +0000 UTC m=+39.671853119" Jul 08 19:04:34.439243 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:34.439211 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/03c97b5b-1c6e-4461-9fed-e14c43705ddb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dkzqj\" (UID: \"03c97b5b-1c6e-4461-9fed-e14c43705ddb\") " pod="openshift-insights/insights-runtime-extractor-dkzqj" Jul 08 19:04:34.439399 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:34.439299 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/03c97b5b-1c6e-4461-9fed-e14c43705ddb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dkzqj\" (UID: \"03c97b5b-1c6e-4461-9fed-e14c43705ddb\") " pod="openshift-insights/insights-runtime-extractor-dkzqj" Jul 08 19:04:34.439399 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:34.439394 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 19:04:34.439537 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:34.439401 2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 19:04:34.439537 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:34.439464 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03c97b5b-1c6e-4461-9fed-e14c43705ddb-insights-runtime-extractor-tls podName:03c97b5b-1c6e-4461-9fed-e14c43705ddb nodeName:}" failed. No retries permitted until 2026-07-08 19:04:38.439447536 +0000 UTC m=+43.718535153 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/03c97b5b-1c6e-4461-9fed-e14c43705ddb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dkzqj" (UID: "03c97b5b-1c6e-4461-9fed-e14c43705ddb") : secret "insights-runtime-extractor-tls" not found Jul 08 19:04:34.439537 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:34.439492 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/03c97b5b-1c6e-4461-9fed-e14c43705ddb-kube-rbac-proxy-cm podName:03c97b5b-1c6e-4461-9fed-e14c43705ddb nodeName:}" failed. No retries permitted until 2026-07-08 19:04:38.439471351 +0000 UTC m=+43.718558951 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/03c97b5b-1c6e-4461-9fed-e14c43705ddb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-dkzqj" (UID: "03c97b5b-1c6e-4461-9fed-e14c43705ddb") : configmap "kube-rbac-proxy" not found Jul 08 19:04:35.371699 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:35.371369 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-mvdjl" event={"ID":"26c73a09-c9a6-4f28-b811-c3cd7f0e3fb6","Type":"ContainerStarted","Data":"8c6b5fc1cb4ea7e2f67d4a6e958c767e8c5192c9a1c0f90d9c82464ee75cc12d"} Jul 08 19:04:35.373223 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:35.373192 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jd5m2" event={"ID":"a1ace7c9-8aec-4fb8-992e-e9e2a10b2d77","Type":"ContainerStarted","Data":"035b0a3d4cd6cf1d48901b60d2695c5f259ae7960fa7ff71ed2a1cc4bb7cf3b5"} Jul 08 19:04:35.412448 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:35.412380 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-mvdjl" podStartSLOduration=3.305086189 podStartE2EDuration="5.412356234s" totalImagesPullingTime="2.107270045s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:04:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:04:35.390611573 +0000 UTC m=+40.669699193" watchObservedRunningTime="2026-07-08 19:04:35.412356234 +0000 UTC m=+40.691443855" Jul 08 19:04:35.413153 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:35.413118 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-jd5m2" podStartSLOduration=2.105894026 podStartE2EDuration="5.41310683s" totalImagesPullingTime="3.307212804s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:04:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:04:35.411964146 +0000 UTC m=+40.691051769" watchObservedRunningTime="2026-07-08 19:04:35.41310683 +0000 UTC m=+40.692194451" Jul 08 19:04:38.269191 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:38.269109 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/50770248-05eb-4ff9-946f-177c6111ce42-registry-tls\") pod \"image-registry-6b9496b987-kfw7l\" (UID: \"50770248-05eb-4ff9-946f-177c6111ce42\") " pod="openshift-image-registry/image-registry-6b9496b987-kfw7l" Jul 08 19:04:38.269539 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:38.269229 2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 19:04:38.269539 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:38.269240 2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b9496b987-kfw7l: secret "image-registry-tls" not found Jul 08 19:04:38.269539 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:38.269289 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50770248-05eb-4ff9-946f-177c6111ce42-registry-tls podName:50770248-05eb-4ff9-946f-177c6111ce42 nodeName:}" failed. No retries permitted until 2026-07-08 19:04:46.269275524 +0000 UTC m=+51.548363125 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/50770248-05eb-4ff9-946f-177c6111ce42-registry-tls") pod "image-registry-6b9496b987-kfw7l" (UID: "50770248-05eb-4ff9-946f-177c6111ce42") : secret "image-registry-tls" not found Jul 08 19:04:38.370181 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:38.370147 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f875980c-42c6-47d0-995b-156a37813723-metrics-tls\") pod \"dns-default-cz9s6\" (UID: \"f875980c-42c6-47d0-995b-156a37813723\") " pod="openshift-dns/dns-default-cz9s6" Jul 08 19:04:38.370325 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:38.370298 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 19:04:38.370375 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:38.370365 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f875980c-42c6-47d0-995b-156a37813723-metrics-tls podName:f875980c-42c6-47d0-995b-156a37813723 nodeName:}" failed. No retries permitted until 2026-07-08 19:04:46.37035015 +0000 UTC m=+51.649437748 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f875980c-42c6-47d0-995b-156a37813723-metrics-tls") pod "dns-default-cz9s6" (UID: "f875980c-42c6-47d0-995b-156a37813723") : secret "dns-default-metrics-tls" not found Jul 08 19:04:38.382631 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:38.382600 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-swjnd" event={"ID":"0ecaa5ea-9362-419c-8ec6-60a84115d94a","Type":"ContainerStarted","Data":"5539b3f9db796930efea46b05b4156e0204286d9498ab999cb3c88ce032e607d"} Jul 08 19:04:38.398925 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:38.398885 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-swjnd" podStartSLOduration=2.028933977 podStartE2EDuration="6.398868201s" totalImagesPullingTime="4.369934224s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:04:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:04:38.397861972 +0000 UTC m=+43.676949592" watchObservedRunningTime="2026-07-08 19:04:38.398868201 +0000 UTC m=+43.677955837" Jul 08 19:04:38.471317 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:38.471278 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/03c97b5b-1c6e-4461-9fed-e14c43705ddb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dkzqj\" (UID: \"03c97b5b-1c6e-4461-9fed-e14c43705ddb\") " pod="openshift-insights/insights-runtime-extractor-dkzqj" Jul 08 19:04:38.471512 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:38.471373 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/03c97b5b-1c6e-4461-9fed-e14c43705ddb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dkzqj\" (UID: \"03c97b5b-1c6e-4461-9fed-e14c43705ddb\") " pod="openshift-insights/insights-runtime-extractor-dkzqj" Jul 08 19:04:38.471512 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:38.471419 2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 19:04:38.471512 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:38.471496 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 19:04:38.471512 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:38.471511 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/03c97b5b-1c6e-4461-9fed-e14c43705ddb-kube-rbac-proxy-cm podName:03c97b5b-1c6e-4461-9fed-e14c43705ddb nodeName:}" failed. No retries permitted until 2026-07-08 19:04:46.471492446 +0000 UTC m=+51.750580063 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/03c97b5b-1c6e-4461-9fed-e14c43705ddb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-dkzqj" (UID: "03c97b5b-1c6e-4461-9fed-e14c43705ddb") : configmap "kube-rbac-proxy" not found Jul 08 19:04:38.471698 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:38.471543 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03c97b5b-1c6e-4461-9fed-e14c43705ddb-insights-runtime-extractor-tls podName:03c97b5b-1c6e-4461-9fed-e14c43705ddb nodeName:}" failed. No retries permitted until 2026-07-08 19:04:46.47153015 +0000 UTC m=+51.750617748 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/03c97b5b-1c6e-4461-9fed-e14c43705ddb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dkzqj" (UID: "03c97b5b-1c6e-4461-9fed-e14c43705ddb") : secret "insights-runtime-extractor-tls" not found Jul 08 19:04:46.340194 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:46.340159 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/50770248-05eb-4ff9-946f-177c6111ce42-registry-tls\") pod \"image-registry-6b9496b987-kfw7l\" (UID: \"50770248-05eb-4ff9-946f-177c6111ce42\") " pod="openshift-image-registry/image-registry-6b9496b987-kfw7l" Jul 08 19:04:46.344056 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:46.344026 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/50770248-05eb-4ff9-946f-177c6111ce42-registry-tls\") pod \"image-registry-6b9496b987-kfw7l\" (UID: \"50770248-05eb-4ff9-946f-177c6111ce42\") " pod="openshift-image-registry/image-registry-6b9496b987-kfw7l" Jul 08 19:04:46.441356 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:46.441323 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f875980c-42c6-47d0-995b-156a37813723-metrics-tls\") pod \"dns-default-cz9s6\" (UID: \"f875980c-42c6-47d0-995b-156a37813723\") " pod="openshift-dns/dns-default-cz9s6" Jul 08 19:04:46.443685 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:46.443665 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f875980c-42c6-47d0-995b-156a37813723-metrics-tls\") pod \"dns-default-cz9s6\" (UID: \"f875980c-42c6-47d0-995b-156a37813723\") " pod="openshift-dns/dns-default-cz9s6" Jul 08 19:04:46.446745 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:46.446724 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b9496b987-kfw7l" Jul 08 19:04:46.460856 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:46.460831 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cz9s6" Jul 08 19:04:46.542364 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:46.542086 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/03c97b5b-1c6e-4461-9fed-e14c43705ddb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dkzqj\" (UID: \"03c97b5b-1c6e-4461-9fed-e14c43705ddb\") " pod="openshift-insights/insights-runtime-extractor-dkzqj" Jul 08 19:04:46.542364 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:46.542181 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/03c97b5b-1c6e-4461-9fed-e14c43705ddb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dkzqj\" (UID: \"03c97b5b-1c6e-4461-9fed-e14c43705ddb\") " pod="openshift-insights/insights-runtime-extractor-dkzqj" Jul 08 19:04:46.542364 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:46.542309 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 19:04:46.542622 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:04:46.542385 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03c97b5b-1c6e-4461-9fed-e14c43705ddb-insights-runtime-extractor-tls podName:03c97b5b-1c6e-4461-9fed-e14c43705ddb nodeName:}" failed. No retries permitted until 2026-07-08 19:05:02.542365027 +0000 UTC m=+67.821452630 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/03c97b5b-1c6e-4461-9fed-e14c43705ddb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dkzqj" (UID: "03c97b5b-1c6e-4461-9fed-e14c43705ddb") : secret "insights-runtime-extractor-tls" not found Jul 08 19:04:46.542885 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:46.542852 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/03c97b5b-1c6e-4461-9fed-e14c43705ddb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dkzqj\" (UID: \"03c97b5b-1c6e-4461-9fed-e14c43705ddb\") " pod="openshift-insights/insights-runtime-extractor-dkzqj" Jul 08 19:04:46.620030 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:46.619947 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6b9496b987-kfw7l"] Jul 08 19:04:46.622810 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:04:46.622784 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50770248_05eb_4ff9_946f_177c6111ce42.slice/crio-980bcc14815ad16c9011764720876768cf75d4684deac7b9855a4ebb6c0a485e WatchSource:0}: Error finding container 980bcc14815ad16c9011764720876768cf75d4684deac7b9855a4ebb6c0a485e: Status 404 returned error can't find the container with id 980bcc14815ad16c9011764720876768cf75d4684deac7b9855a4ebb6c0a485e Jul 08 19:04:46.627734 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:46.627713 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cz9s6"] Jul 08 19:04:46.630201 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:04:46.630178 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf875980c_42c6_47d0_995b_156a37813723.slice/crio-65d82e960f40ec88065e258cabdc97b25c3822dd79999440957aad1c86759f4d WatchSource:0}: Error finding container 65d82e960f40ec88065e258cabdc97b25c3822dd79999440957aad1c86759f4d: Status 404 returned error can't find the container with id 65d82e960f40ec88065e258cabdc97b25c3822dd79999440957aad1c86759f4d Jul 08 19:04:47.407901 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:47.407862 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cz9s6" event={"ID":"f875980c-42c6-47d0-995b-156a37813723","Type":"ContainerStarted","Data":"65d82e960f40ec88065e258cabdc97b25c3822dd79999440957aad1c86759f4d"} Jul 08 19:04:47.409687 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:47.409655 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b9496b987-kfw7l" event={"ID":"50770248-05eb-4ff9-946f-177c6111ce42","Type":"ContainerStarted","Data":"02075c872262255afd8b29cc3cd0594241ffb982133e35e8a68469d7694ba2ab"} Jul 08 19:04:47.409855 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:47.409705 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b9496b987-kfw7l" event={"ID":"50770248-05eb-4ff9-946f-177c6111ce42","Type":"ContainerStarted","Data":"980bcc14815ad16c9011764720876768cf75d4684deac7b9855a4ebb6c0a485e"} Jul 08 19:04:47.409855 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:47.409807 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6b9496b987-kfw7l" Jul 08 19:04:47.434903 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:47.434854 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6b9496b987-kfw7l" podStartSLOduration=25.434838657 podStartE2EDuration="25.434838657s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:04:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:04:47.433217265 +0000 UTC m=+52.712304887" watchObservedRunningTime="2026-07-08 19:04:47.434838657 +0000 UTC m=+52.713926276" Jul 08 19:04:49.418412 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:49.418376 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cz9s6" event={"ID":"f875980c-42c6-47d0-995b-156a37813723","Type":"ContainerStarted","Data":"db7a4fd7fb427a6edd010b9cfc57035d1206114799d8bdc4a4d2a61625ee0716"} Jul 08 19:04:49.418917 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:49.418446 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cz9s6" event={"ID":"f875980c-42c6-47d0-995b-156a37813723","Type":"ContainerStarted","Data":"32a31bed55fc0cb1847a9501bfe29c5bf574cc564658e9ab6fd625cf487797f8"} Jul 08 19:04:49.418917 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:49.418500 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-cz9s6" Jul 08 19:04:49.445263 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:49.445213 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-cz9s6" podStartSLOduration=17.262613874 podStartE2EDuration="19.445197711s" totalImagesPullingTime="2.182583837s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:04:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:04:49.444106482 +0000 UTC m=+54.723194147" watchObservedRunningTime="2026-07-08 19:04:49.445197711 +0000 UTC m=+54.724285330" Jul 08 19:04:54.563149 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.563112 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5584454cc4-ctnk9"] Jul 08 19:04:54.583269 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.583226 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fd887cfbb-z2n2w"] Jul 08 19:04:54.583425 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.583357 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5584454cc4-ctnk9" Jul 08 19:04:54.587172 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.587146 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 08 19:04:54.587471 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.587420 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 08 19:04:54.587664 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.587527 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 08 19:04:54.587664 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.587528 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 08 19:04:54.597120 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.597102 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57fcfb596c-vktvj"] Jul 08 19:04:54.597245 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.597200 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fd887cfbb-z2n2w" Jul 08 19:04:54.599895 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.599867 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/4066b2c9-fb26-481e-80c2-54a73a1faa90-klusterlet-config\") pod \"klusterlet-addon-workmgr-5584454cc4-ctnk9\" (UID: \"4066b2c9-fb26-481e-80c2-54a73a1faa90\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5584454cc4-ctnk9" Jul 08 19:04:54.600010 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.599954 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4066b2c9-fb26-481e-80c2-54a73a1faa90-tmp\") pod \"klusterlet-addon-workmgr-5584454cc4-ctnk9\" (UID: \"4066b2c9-fb26-481e-80c2-54a73a1faa90\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5584454cc4-ctnk9" Jul 08 19:04:54.600010 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.599980 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frbxz\" (UniqueName: \"kubernetes.io/projected/4066b2c9-fb26-481e-80c2-54a73a1faa90-kube-api-access-frbxz\") pod \"klusterlet-addon-workmgr-5584454cc4-ctnk9\" (UID: \"4066b2c9-fb26-481e-80c2-54a73a1faa90\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5584454cc4-ctnk9" Jul 08 19:04:54.600398 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.600384 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 08 19:04:54.601943 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.601921 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-8k2hk\"" Jul 08 19:04:54.618177 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.618156 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fd887cfbb-z2n2w"] Jul 08 19:04:54.618247 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.618194 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5584454cc4-ctnk9"] Jul 08 19:04:54.618247 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.618213 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57fcfb596c-vktvj"] Jul 08 19:04:54.618247 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.618227 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6b9496b987-kfw7l"] Jul 08 19:04:54.618247 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.618167 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57fcfb596c-vktvj" Jul 08 19:04:54.623008 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.622991 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 08 19:04:54.623742 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.623728 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 08 19:04:54.624055 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.624040 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 08 19:04:54.624098 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.624061 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 08 19:04:54.700687 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.700658 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/4066b2c9-fb26-481e-80c2-54a73a1faa90-klusterlet-config\") pod \"klusterlet-addon-workmgr-5584454cc4-ctnk9\" (UID: \"4066b2c9-fb26-481e-80c2-54a73a1faa90\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5584454cc4-ctnk9" Jul 08 19:04:54.700812 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.700696 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/39558e38-5b50-4f2d-883e-60ea7a91dad3-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7fd887cfbb-z2n2w\" (UID: \"39558e38-5b50-4f2d-883e-60ea7a91dad3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fd887cfbb-z2n2w" Jul 08 19:04:54.700812 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.700727 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/8ea0145c-6dc6-4508-80b5-bb131e7e0c8b-hub\") pod \"cluster-proxy-proxy-agent-57fcfb596c-vktvj\" (UID: \"8ea0145c-6dc6-4508-80b5-bb131e7e0c8b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57fcfb596c-vktvj" Jul 08 19:04:54.700812 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.700765 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hqbx\" (UniqueName: \"kubernetes.io/projected/8ea0145c-6dc6-4508-80b5-bb131e7e0c8b-kube-api-access-8hqbx\") pod \"cluster-proxy-proxy-agent-57fcfb596c-vktvj\" (UID: \"8ea0145c-6dc6-4508-80b5-bb131e7e0c8b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57fcfb596c-vktvj" Jul 08 19:04:54.700914 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.700834 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4h7x\" (UniqueName: \"kubernetes.io/projected/39558e38-5b50-4f2d-883e-60ea7a91dad3-kube-api-access-n4h7x\") pod \"managed-serviceaccount-addon-agent-7fd887cfbb-z2n2w\" (UID: \"39558e38-5b50-4f2d-883e-60ea7a91dad3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fd887cfbb-z2n2w" Jul 08 19:04:54.700914 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.700864 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ea0145c-6dc6-4508-80b5-bb131e7e0c8b-additional-service-ca\") pod \"cluster-proxy-proxy-agent-57fcfb596c-vktvj\" (UID: \"8ea0145c-6dc6-4508-80b5-bb131e7e0c8b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57fcfb596c-vktvj" Jul 08 19:04:54.700914 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.700887 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/8ea0145c-6dc6-4508-80b5-bb131e7e0c8b-ca\") pod \"cluster-proxy-proxy-agent-57fcfb596c-vktvj\" (UID: \"8ea0145c-6dc6-4508-80b5-bb131e7e0c8b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57fcfb596c-vktvj" Jul 08 19:04:54.700999 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.700927 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frbxz\" (UniqueName: \"kubernetes.io/projected/4066b2c9-fb26-481e-80c2-54a73a1faa90-kube-api-access-frbxz\") pod \"klusterlet-addon-workmgr-5584454cc4-ctnk9\" (UID: \"4066b2c9-fb26-481e-80c2-54a73a1faa90\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5584454cc4-ctnk9" Jul 08 19:04:54.700999 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.700957 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4066b2c9-fb26-481e-80c2-54a73a1faa90-tmp\") pod \"klusterlet-addon-workmgr-5584454cc4-ctnk9\" (UID: \"4066b2c9-fb26-481e-80c2-54a73a1faa90\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5584454cc4-ctnk9" Jul 08 19:04:54.700999 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.700984 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8ea0145c-6dc6-4508-80b5-bb131e7e0c8b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-57fcfb596c-vktvj\" (UID: \"8ea0145c-6dc6-4508-80b5-bb131e7e0c8b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57fcfb596c-vktvj" Jul 08 19:04:54.701120 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.701011 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/8ea0145c-6dc6-4508-80b5-bb131e7e0c8b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-57fcfb596c-vktvj\" (UID: \"8ea0145c-6dc6-4508-80b5-bb131e7e0c8b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57fcfb596c-vktvj" Jul 08 19:04:54.701340 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.701323 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4066b2c9-fb26-481e-80c2-54a73a1faa90-tmp\") pod \"klusterlet-addon-workmgr-5584454cc4-ctnk9\" (UID: \"4066b2c9-fb26-481e-80c2-54a73a1faa90\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5584454cc4-ctnk9" Jul 08 19:04:54.703150 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.703133 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/4066b2c9-fb26-481e-80c2-54a73a1faa90-klusterlet-config\") pod \"klusterlet-addon-workmgr-5584454cc4-ctnk9\" (UID: \"4066b2c9-fb26-481e-80c2-54a73a1faa90\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5584454cc4-ctnk9" Jul 08 19:04:54.728745 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.728717 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-76d95b5874-7rxrt"] Jul 08 19:04:54.737969 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.737945 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-frbxz\" (UniqueName: \"kubernetes.io/projected/4066b2c9-fb26-481e-80c2-54a73a1faa90-kube-api-access-frbxz\") pod \"klusterlet-addon-workmgr-5584454cc4-ctnk9\" (UID: \"4066b2c9-fb26-481e-80c2-54a73a1faa90\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5584454cc4-ctnk9" Jul 08 19:04:54.753079 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.753025 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-76d95b5874-7rxrt"] Jul 08 19:04:54.753523 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.753503 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76d95b5874-7rxrt" Jul 08 19:04:54.802137 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.802113 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/39558e38-5b50-4f2d-883e-60ea7a91dad3-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7fd887cfbb-z2n2w\" (UID: \"39558e38-5b50-4f2d-883e-60ea7a91dad3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fd887cfbb-z2n2w" Jul 08 19:04:54.802272 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.802145 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vgwb\" (UniqueName: \"kubernetes.io/projected/f19196fd-6a0a-49f7-81b3-8a14637104a0-kube-api-access-6vgwb\") pod \"image-registry-76d95b5874-7rxrt\" (UID: \"f19196fd-6a0a-49f7-81b3-8a14637104a0\") " pod="openshift-image-registry/image-registry-76d95b5874-7rxrt" Jul 08 19:04:54.802272 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.802166 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f19196fd-6a0a-49f7-81b3-8a14637104a0-registry-certificates\") pod \"image-registry-76d95b5874-7rxrt\" (UID: \"f19196fd-6a0a-49f7-81b3-8a14637104a0\") " pod="openshift-image-registry/image-registry-76d95b5874-7rxrt" Jul 08 19:04:54.802272 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.802183 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/8ea0145c-6dc6-4508-80b5-bb131e7e0c8b-hub\") pod \"cluster-proxy-proxy-agent-57fcfb596c-vktvj\" (UID: \"8ea0145c-6dc6-4508-80b5-bb131e7e0c8b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57fcfb596c-vktvj" Jul 08 19:04:54.802272 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.802211 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f19196fd-6a0a-49f7-81b3-8a14637104a0-ca-trust-extracted\") pod \"image-registry-76d95b5874-7rxrt\" (UID: \"f19196fd-6a0a-49f7-81b3-8a14637104a0\") " pod="openshift-image-registry/image-registry-76d95b5874-7rxrt" Jul 08 19:04:54.802272 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.802255 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hqbx\" (UniqueName: \"kubernetes.io/projected/8ea0145c-6dc6-4508-80b5-bb131e7e0c8b-kube-api-access-8hqbx\") pod \"cluster-proxy-proxy-agent-57fcfb596c-vktvj\" (UID: \"8ea0145c-6dc6-4508-80b5-bb131e7e0c8b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57fcfb596c-vktvj" Jul 08 19:04:54.802618 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.802278 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f19196fd-6a0a-49f7-81b3-8a14637104a0-trusted-ca\") pod \"image-registry-76d95b5874-7rxrt\" (UID: \"f19196fd-6a0a-49f7-81b3-8a14637104a0\") " pod="openshift-image-registry/image-registry-76d95b5874-7rxrt" Jul 08 19:04:54.802618 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.802344 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4h7x\" (UniqueName: \"kubernetes.io/projected/39558e38-5b50-4f2d-883e-60ea7a91dad3-kube-api-access-n4h7x\") pod \"managed-serviceaccount-addon-agent-7fd887cfbb-z2n2w\" (UID: \"39558e38-5b50-4f2d-883e-60ea7a91dad3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fd887cfbb-z2n2w" Jul 08 19:04:54.802618 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.802372 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ea0145c-6dc6-4508-80b5-bb131e7e0c8b-additional-service-ca\") pod \"cluster-proxy-proxy-agent-57fcfb596c-vktvj\" (UID: \"8ea0145c-6dc6-4508-80b5-bb131e7e0c8b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57fcfb596c-vktvj" Jul 08 19:04:54.802618 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.802400 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f19196fd-6a0a-49f7-81b3-8a14637104a0-bound-sa-token\") pod \"image-registry-76d95b5874-7rxrt\" (UID: \"f19196fd-6a0a-49f7-81b3-8a14637104a0\") " pod="openshift-image-registry/image-registry-76d95b5874-7rxrt" Jul 08 19:04:54.802618 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.802455 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f19196fd-6a0a-49f7-81b3-8a14637104a0-image-registry-private-configuration\") pod \"image-registry-76d95b5874-7rxrt\" (UID: \"f19196fd-6a0a-49f7-81b3-8a14637104a0\") " pod="openshift-image-registry/image-registry-76d95b5874-7rxrt" Jul 08 19:04:54.802618 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.802485 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/8ea0145c-6dc6-4508-80b5-bb131e7e0c8b-ca\") pod \"cluster-proxy-proxy-agent-57fcfb596c-vktvj\" (UID: \"8ea0145c-6dc6-4508-80b5-bb131e7e0c8b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57fcfb596c-vktvj" Jul 08 19:04:54.802618 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.802536 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f19196fd-6a0a-49f7-81b3-8a14637104a0-registry-tls\") pod \"image-registry-76d95b5874-7rxrt\" (UID: \"f19196fd-6a0a-49f7-81b3-8a14637104a0\") " pod="openshift-image-registry/image-registry-76d95b5874-7rxrt" Jul 08 19:04:54.802618 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.802590 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8ea0145c-6dc6-4508-80b5-bb131e7e0c8b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-57fcfb596c-vktvj\" (UID: \"8ea0145c-6dc6-4508-80b5-bb131e7e0c8b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57fcfb596c-vktvj" Jul 08 19:04:54.803123 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.802624 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/8ea0145c-6dc6-4508-80b5-bb131e7e0c8b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-57fcfb596c-vktvj\" (UID: \"8ea0145c-6dc6-4508-80b5-bb131e7e0c8b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57fcfb596c-vktvj" Jul 08 19:04:54.803123 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.802656 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f19196fd-6a0a-49f7-81b3-8a14637104a0-installation-pull-secrets\") pod \"image-registry-76d95b5874-7rxrt\" (UID: \"f19196fd-6a0a-49f7-81b3-8a14637104a0\") " pod="openshift-image-registry/image-registry-76d95b5874-7rxrt" Jul 08 19:04:54.803319 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.803295 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ea0145c-6dc6-4508-80b5-bb131e7e0c8b-additional-service-ca\") pod \"cluster-proxy-proxy-agent-57fcfb596c-vktvj\" (UID: \"8ea0145c-6dc6-4508-80b5-bb131e7e0c8b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57fcfb596c-vktvj" Jul 08 19:04:54.804844 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.804823 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/39558e38-5b50-4f2d-883e-60ea7a91dad3-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7fd887cfbb-z2n2w\" (UID: \"39558e38-5b50-4f2d-883e-60ea7a91dad3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fd887cfbb-z2n2w" Jul 08 19:04:54.805357 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.805331 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/8ea0145c-6dc6-4508-80b5-bb131e7e0c8b-ca\") pod \"cluster-proxy-proxy-agent-57fcfb596c-vktvj\" (UID: \"8ea0145c-6dc6-4508-80b5-bb131e7e0c8b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57fcfb596c-vktvj" Jul 08 19:04:54.805479 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.805365 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/8ea0145c-6dc6-4508-80b5-bb131e7e0c8b-hub\") pod \"cluster-proxy-proxy-agent-57fcfb596c-vktvj\" (UID: \"8ea0145c-6dc6-4508-80b5-bb131e7e0c8b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57fcfb596c-vktvj" Jul 08 19:04:54.805614 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.805597 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8ea0145c-6dc6-4508-80b5-bb131e7e0c8b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-57fcfb596c-vktvj\" (UID: \"8ea0145c-6dc6-4508-80b5-bb131e7e0c8b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57fcfb596c-vktvj" Jul 08 19:04:54.805696 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.805674 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/8ea0145c-6dc6-4508-80b5-bb131e7e0c8b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-57fcfb596c-vktvj\" (UID: \"8ea0145c-6dc6-4508-80b5-bb131e7e0c8b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57fcfb596c-vktvj" Jul 08 19:04:54.822385 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.822326 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hqbx\" (UniqueName: \"kubernetes.io/projected/8ea0145c-6dc6-4508-80b5-bb131e7e0c8b-kube-api-access-8hqbx\") pod \"cluster-proxy-proxy-agent-57fcfb596c-vktvj\" (UID: \"8ea0145c-6dc6-4508-80b5-bb131e7e0c8b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57fcfb596c-vktvj" Jul 08 19:04:54.824280 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.824262 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4h7x\" (UniqueName: \"kubernetes.io/projected/39558e38-5b50-4f2d-883e-60ea7a91dad3-kube-api-access-n4h7x\") pod \"managed-serviceaccount-addon-agent-7fd887cfbb-z2n2w\" (UID: \"39558e38-5b50-4f2d-883e-60ea7a91dad3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fd887cfbb-z2n2w" Jul 08 19:04:54.894577 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.894545 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5584454cc4-ctnk9" Jul 08 19:04:54.903832 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.903810 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vgwb\" (UniqueName: \"kubernetes.io/projected/f19196fd-6a0a-49f7-81b3-8a14637104a0-kube-api-access-6vgwb\") pod \"image-registry-76d95b5874-7rxrt\" (UID: \"f19196fd-6a0a-49f7-81b3-8a14637104a0\") " pod="openshift-image-registry/image-registry-76d95b5874-7rxrt" Jul 08 19:04:54.903934 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.903840 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f19196fd-6a0a-49f7-81b3-8a14637104a0-registry-certificates\") pod \"image-registry-76d95b5874-7rxrt\" (UID: \"f19196fd-6a0a-49f7-81b3-8a14637104a0\") " pod="openshift-image-registry/image-registry-76d95b5874-7rxrt" Jul 08 19:04:54.903934 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.903860 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f19196fd-6a0a-49f7-81b3-8a14637104a0-ca-trust-extracted\") pod \"image-registry-76d95b5874-7rxrt\" (UID: \"f19196fd-6a0a-49f7-81b3-8a14637104a0\") " pod="openshift-image-registry/image-registry-76d95b5874-7rxrt" Jul 08 19:04:54.903934 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.903890 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f19196fd-6a0a-49f7-81b3-8a14637104a0-trusted-ca\") pod \"image-registry-76d95b5874-7rxrt\" (UID: \"f19196fd-6a0a-49f7-81b3-8a14637104a0\") " pod="openshift-image-registry/image-registry-76d95b5874-7rxrt" Jul 08 19:04:54.904108 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.904065 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f19196fd-6a0a-49f7-81b3-8a14637104a0-bound-sa-token\") pod \"image-registry-76d95b5874-7rxrt\" (UID: \"f19196fd-6a0a-49f7-81b3-8a14637104a0\") " pod="openshift-image-registry/image-registry-76d95b5874-7rxrt" Jul 08 19:04:54.904108 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.904107 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f19196fd-6a0a-49f7-81b3-8a14637104a0-image-registry-private-configuration\") pod \"image-registry-76d95b5874-7rxrt\" (UID: \"f19196fd-6a0a-49f7-81b3-8a14637104a0\") " pod="openshift-image-registry/image-registry-76d95b5874-7rxrt" Jul 08 19:04:54.904243 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.904145 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f19196fd-6a0a-49f7-81b3-8a14637104a0-registry-tls\") pod \"image-registry-76d95b5874-7rxrt\" (UID: \"f19196fd-6a0a-49f7-81b3-8a14637104a0\") " pod="openshift-image-registry/image-registry-76d95b5874-7rxrt" Jul 08 19:04:54.904243 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.904194 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f19196fd-6a0a-49f7-81b3-8a14637104a0-installation-pull-secrets\") pod \"image-registry-76d95b5874-7rxrt\" (UID: \"f19196fd-6a0a-49f7-81b3-8a14637104a0\") " pod="openshift-image-registry/image-registry-76d95b5874-7rxrt" Jul 08 19:04:54.904342 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.904317 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f19196fd-6a0a-49f7-81b3-8a14637104a0-ca-trust-extracted\") pod \"image-registry-76d95b5874-7rxrt\" (UID: \"f19196fd-6a0a-49f7-81b3-8a14637104a0\") " pod="openshift-image-registry/image-registry-76d95b5874-7rxrt" Jul 08 19:04:54.904779 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.904753 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f19196fd-6a0a-49f7-81b3-8a14637104a0-registry-certificates\") pod \"image-registry-76d95b5874-7rxrt\" (UID: \"f19196fd-6a0a-49f7-81b3-8a14637104a0\") " pod="openshift-image-registry/image-registry-76d95b5874-7rxrt" Jul 08 19:04:54.904896 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.904873 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f19196fd-6a0a-49f7-81b3-8a14637104a0-trusted-ca\") pod \"image-registry-76d95b5874-7rxrt\" (UID: \"f19196fd-6a0a-49f7-81b3-8a14637104a0\") " pod="openshift-image-registry/image-registry-76d95b5874-7rxrt" Jul 08 19:04:54.906570 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.906551 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f19196fd-6a0a-49f7-81b3-8a14637104a0-registry-tls\") pod \"image-registry-76d95b5874-7rxrt\" (UID: \"f19196fd-6a0a-49f7-81b3-8a14637104a0\") " pod="openshift-image-registry/image-registry-76d95b5874-7rxrt" Jul 08 19:04:54.906667 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.906630 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f19196fd-6a0a-49f7-81b3-8a14637104a0-installation-pull-secrets\") pod \"image-registry-76d95b5874-7rxrt\" (UID: \"f19196fd-6a0a-49f7-81b3-8a14637104a0\") " pod="openshift-image-registry/image-registry-76d95b5874-7rxrt" Jul 08 19:04:54.906757 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.906738 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f19196fd-6a0a-49f7-81b3-8a14637104a0-image-registry-private-configuration\") pod \"image-registry-76d95b5874-7rxrt\" (UID: \"f19196fd-6a0a-49f7-81b3-8a14637104a0\") " pod="openshift-image-registry/image-registry-76d95b5874-7rxrt" Jul 08 19:04:54.914094 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.914075 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vgwb\" (UniqueName: \"kubernetes.io/projected/f19196fd-6a0a-49f7-81b3-8a14637104a0-kube-api-access-6vgwb\") pod \"image-registry-76d95b5874-7rxrt\" (UID: \"f19196fd-6a0a-49f7-81b3-8a14637104a0\") " pod="openshift-image-registry/image-registry-76d95b5874-7rxrt" Jul 08 19:04:54.916743 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.916542 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f19196fd-6a0a-49f7-81b3-8a14637104a0-bound-sa-token\") pod \"image-registry-76d95b5874-7rxrt\" (UID: \"f19196fd-6a0a-49f7-81b3-8a14637104a0\") " pod="openshift-image-registry/image-registry-76d95b5874-7rxrt" Jul 08 19:04:54.931771 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.931750 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fd887cfbb-z2n2w" Jul 08 19:04:54.938923 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:54.938903 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57fcfb596c-vktvj" Jul 08 19:04:55.058341 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:55.058256 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5584454cc4-ctnk9"] Jul 08 19:04:55.060147 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:04:55.060118 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4066b2c9_fb26_481e_80c2_54a73a1faa90.slice/crio-ecf18e3f26885e0593069c81610e7109237a6c39990d5e60166f7434ed241a90 WatchSource:0}: Error finding container ecf18e3f26885e0593069c81610e7109237a6c39990d5e60166f7434ed241a90: Status 404 returned error can't find the container with id ecf18e3f26885e0593069c81610e7109237a6c39990d5e60166f7434ed241a90 Jul 08 19:04:55.080854 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:55.080827 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76d95b5874-7rxrt" Jul 08 19:04:55.106650 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:55.106621 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fd887cfbb-z2n2w"] Jul 08 19:04:55.110644 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:04:55.110616 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod39558e38_5b50_4f2d_883e_60ea7a91dad3.slice/crio-80d843602c1c214f7bb2136ed5e87b34f55850d25e5e111bc5d7465d22ddb59f WatchSource:0}: Error finding container 80d843602c1c214f7bb2136ed5e87b34f55850d25e5e111bc5d7465d22ddb59f: Status 404 returned error can't find the container with id 80d843602c1c214f7bb2136ed5e87b34f55850d25e5e111bc5d7465d22ddb59f Jul 08 19:04:55.131476 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:55.131446 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57fcfb596c-vktvj"] Jul 08 19:04:55.134359 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:04:55.134331 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8ea0145c_6dc6_4508_80b5_bb131e7e0c8b.slice/crio-841fb242c0c01630aea8513894bcd5f828a2a478cf40b833d713c7957b15961d WatchSource:0}: Error finding container 841fb242c0c01630aea8513894bcd5f828a2a478cf40b833d713c7957b15961d: Status 404 returned error can't find the container with id 841fb242c0c01630aea8513894bcd5f828a2a478cf40b833d713c7957b15961d Jul 08 19:04:55.238887 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:55.238857 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-76d95b5874-7rxrt"] Jul 08 19:04:55.241353 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:04:55.241328 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf19196fd_6a0a_49f7_81b3_8a14637104a0.slice/crio-1ab15bf9567b079a0e1a576e11dbf51e4df58a109433476fc8ba01c16a8b3ad0 WatchSource:0}: Error finding container 1ab15bf9567b079a0e1a576e11dbf51e4df58a109433476fc8ba01c16a8b3ad0: Status 404 returned error can't find the container with id 1ab15bf9567b079a0e1a576e11dbf51e4df58a109433476fc8ba01c16a8b3ad0 Jul 08 19:04:55.438248 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:55.438203 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57fcfb596c-vktvj" event={"ID":"8ea0145c-6dc6-4508-80b5-bb131e7e0c8b","Type":"ContainerStarted","Data":"841fb242c0c01630aea8513894bcd5f828a2a478cf40b833d713c7957b15961d"} Jul 08 19:04:55.439448 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:55.439405 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5584454cc4-ctnk9" event={"ID":"4066b2c9-fb26-481e-80c2-54a73a1faa90","Type":"ContainerStarted","Data":"ecf18e3f26885e0593069c81610e7109237a6c39990d5e60166f7434ed241a90"} Jul 08 19:04:55.440915 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:55.440890 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76d95b5874-7rxrt" event={"ID":"f19196fd-6a0a-49f7-81b3-8a14637104a0","Type":"ContainerStarted","Data":"551a8c361aa9e5ae75945d734f6fdf47ff4c33b33a6351c38c1b9318b1e676bd"} Jul 08 19:04:55.441029 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:55.440930 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76d95b5874-7rxrt" event={"ID":"f19196fd-6a0a-49f7-81b3-8a14637104a0","Type":"ContainerStarted","Data":"1ab15bf9567b079a0e1a576e11dbf51e4df58a109433476fc8ba01c16a8b3ad0"} Jul 08 19:04:55.441029 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:55.441009 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-76d95b5874-7rxrt" Jul 08 19:04:55.442218 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:55.442178 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fd887cfbb-z2n2w" event={"ID":"39558e38-5b50-4f2d-883e-60ea7a91dad3","Type":"ContainerStarted","Data":"80d843602c1c214f7bb2136ed5e87b34f55850d25e5e111bc5d7465d22ddb59f"} Jul 08 19:04:55.463908 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:55.463862 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-76d95b5874-7rxrt" podStartSLOduration=1.463848616 podStartE2EDuration="1.463848616s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:04:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:04:55.463066778 +0000 UTC m=+60.742154395" watchObservedRunningTime="2026-07-08 19:04:55.463848616 +0000 UTC m=+60.742936226" Jul 08 19:04:57.344463 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:57.343326 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2t8gf" Jul 08 19:04:59.424026 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:04:59.423991 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-cz9s6" Jul 08 19:05:00.862602 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:00.862565 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9a902ad1-6161-490d-afb2-006966897633-original-pull-secret\") pod \"global-pull-secret-syncer-qx98w\" (UID: \"9a902ad1-6161-490d-afb2-006966897633\") " pod="kube-system/global-pull-secret-syncer-qx98w" Jul 08 19:05:00.863031 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:00.862668 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6701701d-6ce5-4028-a9ac-9fabc3925522-metrics-certs\") pod \"network-metrics-daemon-hs752\" (UID: \"6701701d-6ce5-4028-a9ac-9fabc3925522\") " pod="openshift-multus/network-metrics-daemon-hs752" Jul 08 19:05:00.865070 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:00.865046 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 19:05:00.865070 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:00.865058 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 19:05:00.876307 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:00.876029 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6701701d-6ce5-4028-a9ac-9fabc3925522-metrics-certs\") pod \"network-metrics-daemon-hs752\" (UID: \"6701701d-6ce5-4028-a9ac-9fabc3925522\") " pod="openshift-multus/network-metrics-daemon-hs752" Jul 08 19:05:00.876307 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:00.876164 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9a902ad1-6161-490d-afb2-006966897633-original-pull-secret\") pod \"global-pull-secret-syncer-qx98w\" (UID: \"9a902ad1-6161-490d-afb2-006966897633\") " pod="kube-system/global-pull-secret-syncer-qx98w" Jul 08 19:05:00.882831 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:00.882810 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-kpjrj\"" Jul 08 19:05:00.890223 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:00.890202 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hs752" Jul 08 19:05:00.893358 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:00.893342 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qx98w" Jul 08 19:05:01.065898 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:01.065871 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqh5r\" (UniqueName: \"kubernetes.io/projected/7f193dea-aa4a-47c5-8cb2-8f34029b1355-kube-api-access-wqh5r\") pod \"network-check-target-z4hnv\" (UID: \"7f193dea-aa4a-47c5-8cb2-8f34029b1355\") " pod="openshift-network-diagnostics/network-check-target-z4hnv" Jul 08 19:05:01.068884 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:01.068661 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 19:05:01.073812 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:01.073730 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hs752"] Jul 08 19:05:01.078987 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:05:01.078961 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6701701d_6ce5_4028_a9ac_9fabc3925522.slice/crio-187774333b077960f4f858398de265d50a9b9ef3266861cbdc1af9a3d6ed4ef7 WatchSource:0}: Error finding container 187774333b077960f4f858398de265d50a9b9ef3266861cbdc1af9a3d6ed4ef7: Status 404 returned error can't find the container with id 187774333b077960f4f858398de265d50a9b9ef3266861cbdc1af9a3d6ed4ef7 Jul 08 19:05:01.079129 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:01.079038 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 19:05:01.089505 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:01.089488 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqh5r\" (UniqueName: \"kubernetes.io/projected/7f193dea-aa4a-47c5-8cb2-8f34029b1355-kube-api-access-wqh5r\") pod \"network-check-target-z4hnv\" (UID: \"7f193dea-aa4a-47c5-8cb2-8f34029b1355\") " pod="openshift-network-diagnostics/network-check-target-z4hnv" Jul 08 19:05:01.095663 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:01.094356 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qx98w"] Jul 08 19:05:01.191003 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:01.190974 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gzqsf\"" Jul 08 19:05:01.198077 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:01.198054 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z4hnv" Jul 08 19:05:01.331262 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:01.331239 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-z4hnv"] Jul 08 19:05:01.333744 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:05:01.333720 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f193dea_aa4a_47c5_8cb2_8f34029b1355.slice/crio-a1439d3cbe00112eb8b47498757b5604d72e404ee3714edfa41d59a4f18b8c33 WatchSource:0}: Error finding container a1439d3cbe00112eb8b47498757b5604d72e404ee3714edfa41d59a4f18b8c33: Status 404 returned error can't find the container with id a1439d3cbe00112eb8b47498757b5604d72e404ee3714edfa41d59a4f18b8c33 Jul 08 19:05:01.468873 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:01.468831 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57fcfb596c-vktvj" event={"ID":"8ea0145c-6dc6-4508-80b5-bb131e7e0c8b","Type":"ContainerStarted","Data":"1831cd1dd6e44fe81a37e27752e8d8c2736c0571026d84446c645b4ecca316b4"} Jul 08 19:05:01.469073 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:01.468886 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57fcfb596c-vktvj" event={"ID":"8ea0145c-6dc6-4508-80b5-bb131e7e0c8b","Type":"ContainerStarted","Data":"d0a9ee4e32113f95a17a4b1d82cec919f78d16c22e03f9ad44d743012bfeb6b9"} Jul 08 19:05:01.469073 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:01.468907 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57fcfb596c-vktvj" event={"ID":"8ea0145c-6dc6-4508-80b5-bb131e7e0c8b","Type":"ContainerStarted","Data":"91d61d514637ef78566c03690a68e70d34cdcd3fbaa47a9c695e5a105b240d2a"} Jul 08 19:05:01.470566 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:01.470537 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5584454cc4-ctnk9" event={"ID":"4066b2c9-fb26-481e-80c2-54a73a1faa90","Type":"ContainerStarted","Data":"d299a5d33c64ea8e842b3f55b2fc850c1d0c90e7ebfb27c758a000c1804994a9"} Jul 08 19:05:01.470706 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:01.470669 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5584454cc4-ctnk9" Jul 08 19:05:01.471762 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:01.471740 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-z4hnv" event={"ID":"7f193dea-aa4a-47c5-8cb2-8f34029b1355","Type":"ContainerStarted","Data":"a1439d3cbe00112eb8b47498757b5604d72e404ee3714edfa41d59a4f18b8c33"} Jul 08 19:05:01.472581 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:01.472556 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5584454cc4-ctnk9" Jul 08 19:05:01.473050 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:01.473027 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qx98w" event={"ID":"9a902ad1-6161-490d-afb2-006966897633","Type":"ContainerStarted","Data":"a5e1184c13a06837043948933bc265a0105d4302d13b2a22dbed01d627191ef3"} Jul 08 19:05:01.474275 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:01.474250 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hs752" event={"ID":"6701701d-6ce5-4028-a9ac-9fabc3925522","Type":"ContainerStarted","Data":"187774333b077960f4f858398de265d50a9b9ef3266861cbdc1af9a3d6ed4ef7"} Jul 08 19:05:01.475935 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:01.475913 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fd887cfbb-z2n2w" event={"ID":"39558e38-5b50-4f2d-883e-60ea7a91dad3","Type":"ContainerStarted","Data":"370e34c0312664b98fa13c0a642965492513c5d13ecb7de6ea5b017388c61662"} Jul 08 19:05:01.496800 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:01.496751 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-57fcfb596c-vktvj" podStartSLOduration=1.7558217200000001 podStartE2EDuration="7.496734102s" totalImagesPullingTime="5.740912382s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:04:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:05:01.494940368 +0000 UTC m=+66.774027989" watchObservedRunningTime="2026-07-08 19:05:01.496734102 +0000 UTC m=+66.775821700" Jul 08 19:05:01.514891 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:01.514828 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fd887cfbb-z2n2w" podStartSLOduration=1.7652295329999999 podStartE2EDuration="7.514806837s" totalImagesPullingTime="5.749577304s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:04:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:05:01.513791302 +0000 UTC m=+66.792878924" watchObservedRunningTime="2026-07-08 19:05:01.514806837 +0000 UTC m=+66.793894459" Jul 08 19:05:01.540412 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:01.540358 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5584454cc4-ctnk9" podStartSLOduration=1.722572202 podStartE2EDuration="7.540342663s" totalImagesPullingTime="5.817770461s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:04:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:05:01.54009677 +0000 UTC m=+66.819184391" watchObservedRunningTime="2026-07-08 19:05:01.540342663 +0000 UTC m=+66.819430261" Jul 08 19:05:02.579947 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:02.579906 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/03c97b5b-1c6e-4461-9fed-e14c43705ddb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dkzqj\" (UID: \"03c97b5b-1c6e-4461-9fed-e14c43705ddb\") " pod="openshift-insights/insights-runtime-extractor-dkzqj" Jul 08 19:05:02.589932 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:02.589902 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/03c97b5b-1c6e-4461-9fed-e14c43705ddb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dkzqj\" (UID: \"03c97b5b-1c6e-4461-9fed-e14c43705ddb\") " pod="openshift-insights/insights-runtime-extractor-dkzqj" Jul 08 19:05:02.852208 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:02.852127 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6ts4z\"" Jul 08 19:05:02.860379 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:02.860353 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-dkzqj" Jul 08 19:05:03.131094 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:03.131069 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-dkzqj"] Jul 08 19:05:03.135786 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:05:03.135749 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03c97b5b_1c6e_4461_9fed_e14c43705ddb.slice/crio-b14046d462d5b999c4a0f108851f481f08d7f2dd6a59fd3bc31596bd3774755f WatchSource:0}: Error finding container b14046d462d5b999c4a0f108851f481f08d7f2dd6a59fd3bc31596bd3774755f: Status 404 returned error can't find the container with id b14046d462d5b999c4a0f108851f481f08d7f2dd6a59fd3bc31596bd3774755f Jul 08 19:05:03.487197 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:03.487161 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hs752" event={"ID":"6701701d-6ce5-4028-a9ac-9fabc3925522","Type":"ContainerStarted","Data":"c212214c5a81ef8ea3ef284b200b716773091bc4c70864b84e78ce551528270a"} Jul 08 19:05:03.487385 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:03.487220 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hs752" event={"ID":"6701701d-6ce5-4028-a9ac-9fabc3925522","Type":"ContainerStarted","Data":"57761caef0c98e1c8094881865cb08735f47c7a5a4f9e6bf00057eb78ffd6e6e"} Jul 08 19:05:03.488480 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:03.488452 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dkzqj" event={"ID":"03c97b5b-1c6e-4461-9fed-e14c43705ddb","Type":"ContainerStarted","Data":"b14046d462d5b999c4a0f108851f481f08d7f2dd6a59fd3bc31596bd3774755f"} Jul 08 19:05:03.507209 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:03.507137 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-hs752" podStartSLOduration=66.562038643 podStartE2EDuration="1m8.507120729s" totalImagesPullingTime="1.945082086s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:03:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:05:03.505710088 +0000 UTC m=+68.784797711" watchObservedRunningTime="2026-07-08 19:05:03.507120729 +0000 UTC m=+68.786208328" Jul 08 19:05:04.626325 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:04.626286 2583 patch_prober.go:28] interesting pod/image-registry-6b9496b987-kfw7l container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 08 19:05:04.626823 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:04.626350 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6b9496b987-kfw7l" podUID="50770248-05eb-4ff9-946f-177c6111ce42" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 19:05:09.516995 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:09.516956 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-z4hnv" event={"ID":"7f193dea-aa4a-47c5-8cb2-8f34029b1355","Type":"ContainerStarted","Data":"33bc11f0b9d718d6f382afd731f7f49e820642e891b9c53afa9151fce1dfde8a"} Jul 08 19:05:09.517418 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:09.517097 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-z4hnv" Jul 08 19:05:09.518623 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:09.518591 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qx98w" event={"ID":"9a902ad1-6161-490d-afb2-006966897633","Type":"ContainerStarted","Data":"f52d4a6a3ba5e6f9c23e6b248b257b13f3ed3a716176eee4eca7553770a8d9ea"} Jul 08 19:05:09.539224 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:09.539180 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-z4hnv" podStartSLOduration=66.569866616 podStartE2EDuration="1m14.539167566s" totalImagesPullingTime="7.96930095s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:03:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:05:09.53836442 +0000 UTC m=+74.817452039" watchObservedRunningTime="2026-07-08 19:05:09.539167566 +0000 UTC m=+74.818255186" Jul 08 19:05:09.560276 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:09.560177 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-qx98w" podStartSLOduration=66.364885968 podStartE2EDuration="1m14.560159358s" totalImagesPullingTime="8.19527339s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:03:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:05:09.558077322 +0000 UTC m=+74.837164940" watchObservedRunningTime="2026-07-08 19:05:09.560159358 +0000 UTC m=+74.839246978" Jul 08 19:05:10.524453 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:10.524390 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dkzqj" event={"ID":"03c97b5b-1c6e-4461-9fed-e14c43705ddb","Type":"ContainerStarted","Data":"3faad4fb12b5b122374490433827bbb3312a9c16c1cf3336bf69c6b4cac0eb4b"} Jul 08 19:05:11.529762 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:11.529722 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dkzqj" event={"ID":"03c97b5b-1c6e-4461-9fed-e14c43705ddb","Type":"ContainerStarted","Data":"b6a2ea4ff016e30a47893368e538ee0760d37edfc44089cb72708a1a35e9d24b"} Jul 08 19:05:12.762032 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:12.761996 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-vj8cc"] Jul 08 19:05:12.773156 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:12.772422 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-vj8cc" Jul 08 19:05:12.775630 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:12.775425 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 08 19:05:12.775630 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:12.775425 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 08 19:05:12.776377 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:12.776346 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 08 19:05:12.776711 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:12.776693 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 08 19:05:12.776916 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:12.776900 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 08 19:05:12.785866 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:12.780593 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-h87lk\"" Jul 08 19:05:12.797097 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:12.793836 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 08 19:05:12.870184 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:12.869743 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/54b8b171-50ea-4a73-bc26-5208007a513a-node-exporter-tls\") pod \"node-exporter-vj8cc\" (UID: \"54b8b171-50ea-4a73-bc26-5208007a513a\") " pod="openshift-monitoring/node-exporter-vj8cc" Jul 08 19:05:12.870184 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:12.869797 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/54b8b171-50ea-4a73-bc26-5208007a513a-node-exporter-textfile\") pod \"node-exporter-vj8cc\" (UID: \"54b8b171-50ea-4a73-bc26-5208007a513a\") " pod="openshift-monitoring/node-exporter-vj8cc" Jul 08 19:05:12.870184 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:12.869833 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/54b8b171-50ea-4a73-bc26-5208007a513a-sys\") pod \"node-exporter-vj8cc\" (UID: \"54b8b171-50ea-4a73-bc26-5208007a513a\") " pod="openshift-monitoring/node-exporter-vj8cc" Jul 08 19:05:12.870184 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:12.869859 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/54b8b171-50ea-4a73-bc26-5208007a513a-node-exporter-accelerators-collector-config\") pod \"node-exporter-vj8cc\" (UID: \"54b8b171-50ea-4a73-bc26-5208007a513a\") " pod="openshift-monitoring/node-exporter-vj8cc" Jul 08 19:05:12.870184 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:12.869900 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/54b8b171-50ea-4a73-bc26-5208007a513a-root\") pod \"node-exporter-vj8cc\" (UID: \"54b8b171-50ea-4a73-bc26-5208007a513a\") " pod="openshift-monitoring/node-exporter-vj8cc" Jul 08 19:05:12.870184 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:12.869926 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/54b8b171-50ea-4a73-bc26-5208007a513a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vj8cc\" (UID: \"54b8b171-50ea-4a73-bc26-5208007a513a\") " pod="openshift-monitoring/node-exporter-vj8cc" Jul 08 19:05:12.870184 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:12.869956 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/54b8b171-50ea-4a73-bc26-5208007a513a-metrics-client-ca\") pod \"node-exporter-vj8cc\" (UID: \"54b8b171-50ea-4a73-bc26-5208007a513a\") " pod="openshift-monitoring/node-exporter-vj8cc" Jul 08 19:05:12.870184 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:12.869983 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wc9fh\" (UniqueName: \"kubernetes.io/projected/54b8b171-50ea-4a73-bc26-5208007a513a-kube-api-access-wc9fh\") pod \"node-exporter-vj8cc\" (UID: \"54b8b171-50ea-4a73-bc26-5208007a513a\") " pod="openshift-monitoring/node-exporter-vj8cc" Jul 08 19:05:12.870184 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:12.870011 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/54b8b171-50ea-4a73-bc26-5208007a513a-node-exporter-wtmp\") pod \"node-exporter-vj8cc\" (UID: \"54b8b171-50ea-4a73-bc26-5208007a513a\") " pod="openshift-monitoring/node-exporter-vj8cc" Jul 08 19:05:12.970448 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:12.970388 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/54b8b171-50ea-4a73-bc26-5208007a513a-root\") pod \"node-exporter-vj8cc\" (UID: \"54b8b171-50ea-4a73-bc26-5208007a513a\") " pod="openshift-monitoring/node-exporter-vj8cc" Jul 08 19:05:12.970640 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:12.970461 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/54b8b171-50ea-4a73-bc26-5208007a513a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vj8cc\" (UID: \"54b8b171-50ea-4a73-bc26-5208007a513a\") " pod="openshift-monitoring/node-exporter-vj8cc" Jul 08 19:05:12.970640 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:12.970494 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/54b8b171-50ea-4a73-bc26-5208007a513a-metrics-client-ca\") pod \"node-exporter-vj8cc\" (UID: \"54b8b171-50ea-4a73-bc26-5208007a513a\") " pod="openshift-monitoring/node-exporter-vj8cc" Jul 08 19:05:12.970640 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:12.970522 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wc9fh\" (UniqueName: \"kubernetes.io/projected/54b8b171-50ea-4a73-bc26-5208007a513a-kube-api-access-wc9fh\") pod \"node-exporter-vj8cc\" (UID: \"54b8b171-50ea-4a73-bc26-5208007a513a\") " pod="openshift-monitoring/node-exporter-vj8cc" Jul 08 19:05:12.970640 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:12.970547 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/54b8b171-50ea-4a73-bc26-5208007a513a-node-exporter-wtmp\") pod \"node-exporter-vj8cc\" (UID: \"54b8b171-50ea-4a73-bc26-5208007a513a\") " pod="openshift-monitoring/node-exporter-vj8cc" Jul 08 19:05:12.970640 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:12.970614 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/54b8b171-50ea-4a73-bc26-5208007a513a-node-exporter-tls\") pod \"node-exporter-vj8cc\" (UID: \"54b8b171-50ea-4a73-bc26-5208007a513a\") " pod="openshift-monitoring/node-exporter-vj8cc" Jul 08 19:05:12.970640 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:12.970640 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/54b8b171-50ea-4a73-bc26-5208007a513a-node-exporter-textfile\") pod \"node-exporter-vj8cc\" (UID: \"54b8b171-50ea-4a73-bc26-5208007a513a\") " pod="openshift-monitoring/node-exporter-vj8cc" Jul 08 19:05:12.970948 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:12.970669 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/54b8b171-50ea-4a73-bc26-5208007a513a-sys\") pod \"node-exporter-vj8cc\" (UID: \"54b8b171-50ea-4a73-bc26-5208007a513a\") " pod="openshift-monitoring/node-exporter-vj8cc" Jul 08 19:05:12.970948 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:12.970694 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/54b8b171-50ea-4a73-bc26-5208007a513a-node-exporter-accelerators-collector-config\") pod \"node-exporter-vj8cc\" (UID: \"54b8b171-50ea-4a73-bc26-5208007a513a\") " pod="openshift-monitoring/node-exporter-vj8cc" Jul 08 19:05:12.971286 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:12.971262 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/54b8b171-50ea-4a73-bc26-5208007a513a-node-exporter-accelerators-collector-config\") pod \"node-exporter-vj8cc\" (UID: \"54b8b171-50ea-4a73-bc26-5208007a513a\") " pod="openshift-monitoring/node-exporter-vj8cc" Jul 08 19:05:12.971384 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:12.971348 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/54b8b171-50ea-4a73-bc26-5208007a513a-root\") pod \"node-exporter-vj8cc\" (UID: \"54b8b171-50ea-4a73-bc26-5208007a513a\") " pod="openshift-monitoring/node-exporter-vj8cc" Jul 08 19:05:12.972578 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:12.971839 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/54b8b171-50ea-4a73-bc26-5208007a513a-node-exporter-wtmp\") pod \"node-exporter-vj8cc\" (UID: \"54b8b171-50ea-4a73-bc26-5208007a513a\") " pod="openshift-monitoring/node-exporter-vj8cc" Jul 08 19:05:12.972578 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:12.972266 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/54b8b171-50ea-4a73-bc26-5208007a513a-metrics-client-ca\") pod \"node-exporter-vj8cc\" (UID: \"54b8b171-50ea-4a73-bc26-5208007a513a\") " pod="openshift-monitoring/node-exporter-vj8cc" Jul 08 19:05:12.973054 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:12.973004 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/54b8b171-50ea-4a73-bc26-5208007a513a-node-exporter-textfile\") pod \"node-exporter-vj8cc\" (UID: \"54b8b171-50ea-4a73-bc26-5208007a513a\") " pod="openshift-monitoring/node-exporter-vj8cc" Jul 08 19:05:12.973290 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:05:12.973239 2583 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 08 19:05:12.973453 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:05:12.973414 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54b8b171-50ea-4a73-bc26-5208007a513a-node-exporter-tls podName:54b8b171-50ea-4a73-bc26-5208007a513a nodeName:}" failed. No retries permitted until 2026-07-08 19:05:13.473394072 +0000 UTC m=+78.752481674 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/54b8b171-50ea-4a73-bc26-5208007a513a-node-exporter-tls") pod "node-exporter-vj8cc" (UID: "54b8b171-50ea-4a73-bc26-5208007a513a") : secret "node-exporter-tls" not found Jul 08 19:05:12.973890 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:12.973856 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/54b8b171-50ea-4a73-bc26-5208007a513a-sys\") pod \"node-exporter-vj8cc\" (UID: \"54b8b171-50ea-4a73-bc26-5208007a513a\") " pod="openshift-monitoring/node-exporter-vj8cc" Jul 08 19:05:12.974381 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:12.974346 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/54b8b171-50ea-4a73-bc26-5208007a513a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vj8cc\" (UID: \"54b8b171-50ea-4a73-bc26-5208007a513a\") " pod="openshift-monitoring/node-exporter-vj8cc" Jul 08 19:05:12.988104 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:12.987982 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wc9fh\" (UniqueName: \"kubernetes.io/projected/54b8b171-50ea-4a73-bc26-5208007a513a-kube-api-access-wc9fh\") pod \"node-exporter-vj8cc\" (UID: \"54b8b171-50ea-4a73-bc26-5208007a513a\") " pod="openshift-monitoring/node-exporter-vj8cc" Jul 08 19:05:13.474191 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:13.474153 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/54b8b171-50ea-4a73-bc26-5208007a513a-node-exporter-tls\") pod \"node-exporter-vj8cc\" (UID: \"54b8b171-50ea-4a73-bc26-5208007a513a\") " pod="openshift-monitoring/node-exporter-vj8cc" Jul 08 19:05:13.476614 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:13.476588 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/54b8b171-50ea-4a73-bc26-5208007a513a-node-exporter-tls\") pod \"node-exporter-vj8cc\" (UID: \"54b8b171-50ea-4a73-bc26-5208007a513a\") " pod="openshift-monitoring/node-exporter-vj8cc" Jul 08 19:05:13.537684 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:13.537595 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dkzqj" event={"ID":"03c97b5b-1c6e-4461-9fed-e14c43705ddb","Type":"ContainerStarted","Data":"80401ff0a52b2c7ef71cd40a14506f702dba5bc8be7d361d7dea63c53ff345e2"} Jul 08 19:05:13.559857 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:13.559810 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-dkzqj" podStartSLOduration=33.598478158 podStartE2EDuration="43.559794012s" totalImagesPullingTime="9.961315854s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:04:30 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:05:13.558410583 +0000 UTC m=+78.837498205" watchObservedRunningTime="2026-07-08 19:05:13.559794012 +0000 UTC m=+78.838881633" Jul 08 19:05:13.703105 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:13.703062 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-vj8cc" Jul 08 19:05:13.716589 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:05:13.716553 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54b8b171_50ea_4a73_bc26_5208007a513a.slice/crio-0b6e36bde37a0c4ba9e2f256f3711f2a96f99c6185878063e0369bb0338423b4 WatchSource:0}: Error finding container 0b6e36bde37a0c4ba9e2f256f3711f2a96f99c6185878063e0369bb0338423b4: Status 404 returned error can't find the container with id 0b6e36bde37a0c4ba9e2f256f3711f2a96f99c6185878063e0369bb0338423b4 Jul 08 19:05:14.541861 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:14.541828 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vj8cc" event={"ID":"54b8b171-50ea-4a73-bc26-5208007a513a","Type":"ContainerStarted","Data":"0b6e36bde37a0c4ba9e2f256f3711f2a96f99c6185878063e0369bb0338423b4"} Jul 08 19:05:14.628801 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:14.628771 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6b9496b987-kfw7l" Jul 08 19:05:15.546641 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:15.546525 2583 generic.go:356] "Generic (PLEG): container finished" podID="54b8b171-50ea-4a73-bc26-5208007a513a" containerID="dda4f936beba6306fc78d4db0f6feceaed07aeacc2112f4f0d49e7b57b98cd2f" exitCode=0 Jul 08 19:05:15.546641 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:15.546582 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vj8cc" event={"ID":"54b8b171-50ea-4a73-bc26-5208007a513a","Type":"ContainerDied","Data":"dda4f936beba6306fc78d4db0f6feceaed07aeacc2112f4f0d49e7b57b98cd2f"} Jul 08 19:05:16.453919 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:16.453882 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-76d95b5874-7rxrt" Jul 08 19:05:16.552058 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:16.552023 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vj8cc" event={"ID":"54b8b171-50ea-4a73-bc26-5208007a513a","Type":"ContainerStarted","Data":"65febf4e0a35854339514308979eef3e1958fec856c0a2692265a843ece6102d"} Jul 08 19:05:16.552452 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:16.552075 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vj8cc" event={"ID":"54b8b171-50ea-4a73-bc26-5208007a513a","Type":"ContainerStarted","Data":"6e4087d1c0afc7b1fc2be1e003cd5620eefae3f64786e773ceb529a87f9a2dfa"} Jul 08 19:05:16.574367 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:16.574314 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-vj8cc" podStartSLOduration=3.312708321 podStartE2EDuration="4.574285699s" totalImagesPullingTime="1.191032143s" totalInitContainerRuntime="70.545235ms" isStatefulPod=true podCreationTimestamp="2026-07-08 19:05:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:05:16.573113425 +0000 UTC m=+81.852201045" watchObservedRunningTime="2026-07-08 19:05:16.574285699 +0000 UTC m=+81.853373318" Jul 08 19:05:19.639092 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:19.639029 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6b9496b987-kfw7l" podUID="50770248-05eb-4ff9-946f-177c6111ce42" containerName="registry" containerID="cri-o://02075c872262255afd8b29cc3cd0594241ffb982133e35e8a68469d7694ba2ab" gracePeriod=30 Jul 08 19:05:19.874082 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:19.874055 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b9496b987-kfw7l" Jul 08 19:05:20.028196 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:20.028163 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/50770248-05eb-4ff9-946f-177c6111ce42-registry-certificates\") pod \"50770248-05eb-4ff9-946f-177c6111ce42\" (UID: \"50770248-05eb-4ff9-946f-177c6111ce42\") " Jul 08 19:05:20.028418 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:20.028212 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/50770248-05eb-4ff9-946f-177c6111ce42-registry-tls\") pod \"50770248-05eb-4ff9-946f-177c6111ce42\" (UID: \"50770248-05eb-4ff9-946f-177c6111ce42\") " Jul 08 19:05:20.028418 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:20.028239 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/50770248-05eb-4ff9-946f-177c6111ce42-trusted-ca\") pod \"50770248-05eb-4ff9-946f-177c6111ce42\" (UID: \"50770248-05eb-4ff9-946f-177c6111ce42\") " Jul 08 19:05:20.028418 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:20.028257 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/50770248-05eb-4ff9-946f-177c6111ce42-bound-sa-token\") pod \"50770248-05eb-4ff9-946f-177c6111ce42\" (UID: \"50770248-05eb-4ff9-946f-177c6111ce42\") " Jul 08 19:05:20.028418 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:20.028278 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/50770248-05eb-4ff9-946f-177c6111ce42-image-registry-private-configuration\") pod \"50770248-05eb-4ff9-946f-177c6111ce42\" (UID: \"50770248-05eb-4ff9-946f-177c6111ce42\") " Jul 08 19:05:20.028418 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:20.028317 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/50770248-05eb-4ff9-946f-177c6111ce42-ca-trust-extracted\") pod \"50770248-05eb-4ff9-946f-177c6111ce42\" (UID: \"50770248-05eb-4ff9-946f-177c6111ce42\") " Jul 08 19:05:20.028418 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:20.028336 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/50770248-05eb-4ff9-946f-177c6111ce42-installation-pull-secrets\") pod \"50770248-05eb-4ff9-946f-177c6111ce42\" (UID: \"50770248-05eb-4ff9-946f-177c6111ce42\") " Jul 08 19:05:20.028418 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:20.028369 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-25rgg\" (UniqueName: \"kubernetes.io/projected/50770248-05eb-4ff9-946f-177c6111ce42-kube-api-access-25rgg\") pod \"50770248-05eb-4ff9-946f-177c6111ce42\" (UID: \"50770248-05eb-4ff9-946f-177c6111ce42\") " Jul 08 19:05:20.028814 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:20.028575 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50770248-05eb-4ff9-946f-177c6111ce42-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "50770248-05eb-4ff9-946f-177c6111ce42" (UID: "50770248-05eb-4ff9-946f-177c6111ce42"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:05:20.030137 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:20.030105 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50770248-05eb-4ff9-946f-177c6111ce42-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "50770248-05eb-4ff9-946f-177c6111ce42" (UID: "50770248-05eb-4ff9-946f-177c6111ce42"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:05:20.031534 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:20.031507 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50770248-05eb-4ff9-946f-177c6111ce42-kube-api-access-25rgg" (OuterVolumeSpecName: "kube-api-access-25rgg") pod "50770248-05eb-4ff9-946f-177c6111ce42" (UID: "50770248-05eb-4ff9-946f-177c6111ce42"). InnerVolumeSpecName "kube-api-access-25rgg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:05:20.032319 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:20.032294 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50770248-05eb-4ff9-946f-177c6111ce42-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "50770248-05eb-4ff9-946f-177c6111ce42" (UID: "50770248-05eb-4ff9-946f-177c6111ce42"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:05:20.032481 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:20.032402 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50770248-05eb-4ff9-946f-177c6111ce42-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "50770248-05eb-4ff9-946f-177c6111ce42" (UID: "50770248-05eb-4ff9-946f-177c6111ce42"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:05:20.033786 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:20.033766 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50770248-05eb-4ff9-946f-177c6111ce42-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "50770248-05eb-4ff9-946f-177c6111ce42" (UID: "50770248-05eb-4ff9-946f-177c6111ce42"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:05:20.034517 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:20.034490 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50770248-05eb-4ff9-946f-177c6111ce42-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "50770248-05eb-4ff9-946f-177c6111ce42" (UID: "50770248-05eb-4ff9-946f-177c6111ce42"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:05:20.039503 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:20.039480 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50770248-05eb-4ff9-946f-177c6111ce42-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "50770248-05eb-4ff9-946f-177c6111ce42" (UID: "50770248-05eb-4ff9-946f-177c6111ce42"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:05:20.129601 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:20.129569 2583 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/50770248-05eb-4ff9-946f-177c6111ce42-trusted-ca\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:05:20.129601 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:20.129596 2583 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/50770248-05eb-4ff9-946f-177c6111ce42-bound-sa-token\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:05:20.129601 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:20.129605 2583 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/50770248-05eb-4ff9-946f-177c6111ce42-image-registry-private-configuration\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:05:20.129808 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:20.129614 2583 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/50770248-05eb-4ff9-946f-177c6111ce42-ca-trust-extracted\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:05:20.129808 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:20.129625 2583 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/50770248-05eb-4ff9-946f-177c6111ce42-installation-pull-secrets\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:05:20.129808 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:20.129638 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-25rgg\" (UniqueName: \"kubernetes.io/projected/50770248-05eb-4ff9-946f-177c6111ce42-kube-api-access-25rgg\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:05:20.129808 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:20.129646 2583 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/50770248-05eb-4ff9-946f-177c6111ce42-registry-certificates\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:05:20.129808 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:20.129655 2583 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/50770248-05eb-4ff9-946f-177c6111ce42-registry-tls\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:05:20.567848 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:20.567811 2583 generic.go:356] "Generic (PLEG): container finished" podID="50770248-05eb-4ff9-946f-177c6111ce42" containerID="02075c872262255afd8b29cc3cd0594241ffb982133e35e8a68469d7694ba2ab" exitCode=0 Jul 08 19:05:20.568001 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:20.567858 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b9496b987-kfw7l" Jul 08 19:05:20.568001 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:20.567899 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b9496b987-kfw7l" event={"ID":"50770248-05eb-4ff9-946f-177c6111ce42","Type":"ContainerDied","Data":"02075c872262255afd8b29cc3cd0594241ffb982133e35e8a68469d7694ba2ab"} Jul 08 19:05:20.568001 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:20.567957 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b9496b987-kfw7l" event={"ID":"50770248-05eb-4ff9-946f-177c6111ce42","Type":"ContainerDied","Data":"980bcc14815ad16c9011764720876768cf75d4684deac7b9855a4ebb6c0a485e"} Jul 08 19:05:20.568001 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:20.567992 2583 scope.go:117] "RemoveContainer" containerID="02075c872262255afd8b29cc3cd0594241ffb982133e35e8a68469d7694ba2ab" Jul 08 19:05:20.579129 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:20.579109 2583 scope.go:117] "RemoveContainer" containerID="02075c872262255afd8b29cc3cd0594241ffb982133e35e8a68469d7694ba2ab" Jul 08 19:05:20.579420 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:05:20.579397 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02075c872262255afd8b29cc3cd0594241ffb982133e35e8a68469d7694ba2ab\": container with ID starting with 02075c872262255afd8b29cc3cd0594241ffb982133e35e8a68469d7694ba2ab not found: ID does not exist" containerID="02075c872262255afd8b29cc3cd0594241ffb982133e35e8a68469d7694ba2ab" Jul 08 19:05:20.579483 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:20.579446 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02075c872262255afd8b29cc3cd0594241ffb982133e35e8a68469d7694ba2ab"} err="failed to get container status \"02075c872262255afd8b29cc3cd0594241ffb982133e35e8a68469d7694ba2ab\": rpc error: code = NotFound desc = could not find container \"02075c872262255afd8b29cc3cd0594241ffb982133e35e8a68469d7694ba2ab\": container with ID starting with 02075c872262255afd8b29cc3cd0594241ffb982133e35e8a68469d7694ba2ab not found: ID does not exist" Jul 08 19:05:20.596977 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:20.596949 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6b9496b987-kfw7l"] Jul 08 19:05:20.600801 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:20.600780 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6b9496b987-kfw7l"] Jul 08 19:05:21.173154 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:21.173125 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50770248-05eb-4ff9-946f-177c6111ce42" path="/var/lib/kubelet/pods/50770248-05eb-4ff9-946f-177c6111ce42/volumes" Jul 08 19:05:40.527490 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:05:40.527458 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-z4hnv" Jul 08 19:08:55.047459 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:08:55.047410 2583 kubelet.go:1648] "Image garbage collection succeeded" Jul 08 19:09:41.422605 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:09:41.422570 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-994cc9f69-7vjn8"] Jul 08 19:09:41.423082 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:09:41.422826 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50770248-05eb-4ff9-946f-177c6111ce42" containerName="registry" Jul 08 19:09:41.423082 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:09:41.422837 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="50770248-05eb-4ff9-946f-177c6111ce42" containerName="registry" Jul 08 19:09:41.423082 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:09:41.422899 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="50770248-05eb-4ff9-946f-177c6111ce42" containerName="registry" Jul 08 19:09:41.425746 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:09:41.425727 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-994cc9f69-7vjn8" Jul 08 19:09:41.434518 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:09:41.434497 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 08 19:09:41.438778 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:09:41.438760 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 08 19:09:41.438917 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:09:41.438900 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-dkxt2\"" Jul 08 19:09:41.439009 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:09:41.438994 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 08 19:09:41.439049 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:09:41.438996 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 08 19:09:41.455763 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:09:41.455732 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-994cc9f69-7vjn8"] Jul 08 19:09:41.485533 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:09:41.485500 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0c940ac7-574d-41be-b59b-45dfd4db0dba-tmp\") pod \"opendatahub-operator-controller-manager-994cc9f69-7vjn8\" (UID: \"0c940ac7-574d-41be-b59b-45dfd4db0dba\") " pod="opendatahub/opendatahub-operator-controller-manager-994cc9f69-7vjn8" Jul 08 19:09:41.485533 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:09:41.485534 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0c940ac7-574d-41be-b59b-45dfd4db0dba-manifests\") pod \"opendatahub-operator-controller-manager-994cc9f69-7vjn8\" (UID: \"0c940ac7-574d-41be-b59b-45dfd4db0dba\") " pod="opendatahub/opendatahub-operator-controller-manager-994cc9f69-7vjn8" Jul 08 19:09:41.485710 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:09:41.485575 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0c940ac7-574d-41be-b59b-45dfd4db0dba-apiservice-cert\") pod \"opendatahub-operator-controller-manager-994cc9f69-7vjn8\" (UID: \"0c940ac7-574d-41be-b59b-45dfd4db0dba\") " pod="opendatahub/opendatahub-operator-controller-manager-994cc9f69-7vjn8" Jul 08 19:09:41.485710 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:09:41.485632 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2p7p9\" (UniqueName: \"kubernetes.io/projected/0c940ac7-574d-41be-b59b-45dfd4db0dba-kube-api-access-2p7p9\") pod \"opendatahub-operator-controller-manager-994cc9f69-7vjn8\" (UID: \"0c940ac7-574d-41be-b59b-45dfd4db0dba\") " pod="opendatahub/opendatahub-operator-controller-manager-994cc9f69-7vjn8" Jul 08 19:09:41.485710 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:09:41.485704 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0c940ac7-574d-41be-b59b-45dfd4db0dba-webhook-cert\") pod \"opendatahub-operator-controller-manager-994cc9f69-7vjn8\" (UID: \"0c940ac7-574d-41be-b59b-45dfd4db0dba\") " pod="opendatahub/opendatahub-operator-controller-manager-994cc9f69-7vjn8" Jul 08 19:09:41.586401 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:09:41.586369 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0c940ac7-574d-41be-b59b-45dfd4db0dba-webhook-cert\") pod \"opendatahub-operator-controller-manager-994cc9f69-7vjn8\" (UID: \"0c940ac7-574d-41be-b59b-45dfd4db0dba\") " pod="opendatahub/opendatahub-operator-controller-manager-994cc9f69-7vjn8" Jul 08 19:09:41.586632 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:09:41.586422 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0c940ac7-574d-41be-b59b-45dfd4db0dba-tmp\") pod \"opendatahub-operator-controller-manager-994cc9f69-7vjn8\" (UID: \"0c940ac7-574d-41be-b59b-45dfd4db0dba\") " pod="opendatahub/opendatahub-operator-controller-manager-994cc9f69-7vjn8" Jul 08 19:09:41.586632 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:09:41.586476 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0c940ac7-574d-41be-b59b-45dfd4db0dba-manifests\") pod \"opendatahub-operator-controller-manager-994cc9f69-7vjn8\" (UID: \"0c940ac7-574d-41be-b59b-45dfd4db0dba\") " pod="opendatahub/opendatahub-operator-controller-manager-994cc9f69-7vjn8" Jul 08 19:09:41.586632 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:09:41.586514 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0c940ac7-574d-41be-b59b-45dfd4db0dba-apiservice-cert\") pod \"opendatahub-operator-controller-manager-994cc9f69-7vjn8\" (UID: \"0c940ac7-574d-41be-b59b-45dfd4db0dba\") " pod="opendatahub/opendatahub-operator-controller-manager-994cc9f69-7vjn8" Jul 08 19:09:41.586632 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:09:41.586547 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2p7p9\" (UniqueName: \"kubernetes.io/projected/0c940ac7-574d-41be-b59b-45dfd4db0dba-kube-api-access-2p7p9\") pod \"opendatahub-operator-controller-manager-994cc9f69-7vjn8\" (UID: \"0c940ac7-574d-41be-b59b-45dfd4db0dba\") " pod="opendatahub/opendatahub-operator-controller-manager-994cc9f69-7vjn8" Jul 08 19:09:41.586980 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:09:41.586953 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0c940ac7-574d-41be-b59b-45dfd4db0dba-manifests\") pod \"opendatahub-operator-controller-manager-994cc9f69-7vjn8\" (UID: \"0c940ac7-574d-41be-b59b-45dfd4db0dba\") " pod="opendatahub/opendatahub-operator-controller-manager-994cc9f69-7vjn8" Jul 08 19:09:41.588879 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:09:41.588847 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0c940ac7-574d-41be-b59b-45dfd4db0dba-tmp\") pod \"opendatahub-operator-controller-manager-994cc9f69-7vjn8\" (UID: \"0c940ac7-574d-41be-b59b-45dfd4db0dba\") " pod="opendatahub/opendatahub-operator-controller-manager-994cc9f69-7vjn8" Jul 08 19:09:41.589065 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:09:41.589047 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0c940ac7-574d-41be-b59b-45dfd4db0dba-apiservice-cert\") pod \"opendatahub-operator-controller-manager-994cc9f69-7vjn8\" (UID: \"0c940ac7-574d-41be-b59b-45dfd4db0dba\") " pod="opendatahub/opendatahub-operator-controller-manager-994cc9f69-7vjn8" Jul 08 19:09:41.589153 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:09:41.589135 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0c940ac7-574d-41be-b59b-45dfd4db0dba-webhook-cert\") pod \"opendatahub-operator-controller-manager-994cc9f69-7vjn8\" (UID: \"0c940ac7-574d-41be-b59b-45dfd4db0dba\") " pod="opendatahub/opendatahub-operator-controller-manager-994cc9f69-7vjn8" Jul 08 19:09:41.597036 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:09:41.597006 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2p7p9\" (UniqueName: \"kubernetes.io/projected/0c940ac7-574d-41be-b59b-45dfd4db0dba-kube-api-access-2p7p9\") pod \"opendatahub-operator-controller-manager-994cc9f69-7vjn8\" (UID: \"0c940ac7-574d-41be-b59b-45dfd4db0dba\") " pod="opendatahub/opendatahub-operator-controller-manager-994cc9f69-7vjn8" Jul 08 19:09:41.735415 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:09:41.735329 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-994cc9f69-7vjn8" Jul 08 19:09:41.885200 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:09:41.885169 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-994cc9f69-7vjn8"] Jul 08 19:09:41.887681 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:09:41.887645 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c940ac7_574d_41be_b59b_45dfd4db0dba.slice/crio-444bd0e4553cdbef1149f940d3753733c9c0a5c7686b2470918971a756550ff2 WatchSource:0}: Error finding container 444bd0e4553cdbef1149f940d3753733c9c0a5c7686b2470918971a756550ff2: Status 404 returned error can't find the container with id 444bd0e4553cdbef1149f940d3753733c9c0a5c7686b2470918971a756550ff2 Jul 08 19:09:41.889377 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:09:41.889356 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 19:09:42.427319 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:09:42.427279 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-994cc9f69-7vjn8" event={"ID":"0c940ac7-574d-41be-b59b-45dfd4db0dba","Type":"ContainerStarted","Data":"444bd0e4553cdbef1149f940d3753733c9c0a5c7686b2470918971a756550ff2"} Jul 08 19:09:45.439056 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:09:45.438976 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-994cc9f69-7vjn8" event={"ID":"0c940ac7-574d-41be-b59b-45dfd4db0dba","Type":"ContainerStarted","Data":"285fafb5b791f21e2bcd09e93b2abbd52dcd6253a9ef6812adc6b7c3c932a8d8"} Jul 08 19:09:46.443817 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:09:46.443782 2583 generic.go:356] "Generic (PLEG): container finished" podID="0c940ac7-574d-41be-b59b-45dfd4db0dba" containerID="285fafb5b791f21e2bcd09e93b2abbd52dcd6253a9ef6812adc6b7c3c932a8d8" exitCode=0 Jul 08 19:09:46.444231 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:09:46.443839 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-994cc9f69-7vjn8" event={"ID":"0c940ac7-574d-41be-b59b-45dfd4db0dba","Type":"ContainerDied","Data":"285fafb5b791f21e2bcd09e93b2abbd52dcd6253a9ef6812adc6b7c3c932a8d8"} Jul 08 19:09:47.449344 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:09:47.449303 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-994cc9f69-7vjn8" event={"ID":"0c940ac7-574d-41be-b59b-45dfd4db0dba","Type":"ContainerStarted","Data":"90405673ae25cc0bdcd43dc84c0e873356d62b9ddc665e70c50d2bf7d51ce719"} Jul 08 19:09:47.449739 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:09:47.449416 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-994cc9f69-7vjn8" Jul 08 19:09:47.500143 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:09:47.500087 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-994cc9f69-7vjn8" podStartSLOduration=2.349030425 podStartE2EDuration="6.500068328s" totalImagesPullingTime="3.097650635s" totalInitContainerRuntime="1.053387268s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:09:41 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:09:47.498886117 +0000 UTC m=+352.777973738" watchObservedRunningTime="2026-07-08 19:09:47.500068328 +0000 UTC m=+352.779155947" Jul 08 19:09:58.455554 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:09:58.455522 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-994cc9f69-7vjn8" Jul 08 19:10:01.197700 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:01.197667 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-qzg9x"] Jul 08 19:10:01.209387 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:01.209364 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-qzg9x" Jul 08 19:10:01.212235 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:01.212207 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 08 19:10:01.212564 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:01.212548 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-vx4gz\"" Jul 08 19:10:01.215144 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:01.215117 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-685cdb786f-4pcjp"] Jul 08 19:10:01.218703 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:01.218684 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-qzg9x"] Jul 08 19:10:01.218825 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:01.218787 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-685cdb786f-4pcjp" Jul 08 19:10:01.221897 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:01.221879 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 08 19:10:01.222032 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:01.222014 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 08 19:10:01.222114 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:01.222046 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 08 19:10:01.222175 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:01.222146 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 08 19:10:01.222720 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:01.222698 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-2j6hl\"" Jul 08 19:10:01.234477 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:01.234422 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-685cdb786f-4pcjp"] Jul 08 19:10:01.334003 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:01.333971 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dts8z\" (UniqueName: \"kubernetes.io/projected/02ef5a7c-9326-42c8-9933-c2e53a151ff2-kube-api-access-dts8z\") pod \"kube-auth-proxy-685cdb786f-4pcjp\" (UID: \"02ef5a7c-9326-42c8-9933-c2e53a151ff2\") " pod="openshift-ingress/kube-auth-proxy-685cdb786f-4pcjp" Jul 08 19:10:01.334003 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:01.334008 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02ef5a7c-9326-42c8-9933-c2e53a151ff2-tls-certs\") pod \"kube-auth-proxy-685cdb786f-4pcjp\" (UID: \"02ef5a7c-9326-42c8-9933-c2e53a151ff2\") " pod="openshift-ingress/kube-auth-proxy-685cdb786f-4pcjp" Jul 08 19:10:01.334228 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:01.334031 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxhtm\" (UniqueName: \"kubernetes.io/projected/4e80abce-1fd7-43ca-942a-d23acb4b1f87-kube-api-access-rxhtm\") pod \"odh-model-controller-7998c67dc4-qzg9x\" (UID: \"4e80abce-1fd7-43ca-942a-d23acb4b1f87\") " pod="opendatahub/odh-model-controller-7998c67dc4-qzg9x" Jul 08 19:10:01.334228 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:01.334098 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/02ef5a7c-9326-42c8-9933-c2e53a151ff2-tmp\") pod \"kube-auth-proxy-685cdb786f-4pcjp\" (UID: \"02ef5a7c-9326-42c8-9933-c2e53a151ff2\") " pod="openshift-ingress/kube-auth-proxy-685cdb786f-4pcjp" Jul 08 19:10:01.334228 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:01.334174 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4e80abce-1fd7-43ca-942a-d23acb4b1f87-cert\") pod \"odh-model-controller-7998c67dc4-qzg9x\" (UID: \"4e80abce-1fd7-43ca-942a-d23acb4b1f87\") " pod="opendatahub/odh-model-controller-7998c67dc4-qzg9x" Jul 08 19:10:01.434678 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:01.434645 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4e80abce-1fd7-43ca-942a-d23acb4b1f87-cert\") pod \"odh-model-controller-7998c67dc4-qzg9x\" (UID: \"4e80abce-1fd7-43ca-942a-d23acb4b1f87\") " pod="opendatahub/odh-model-controller-7998c67dc4-qzg9x" Jul 08 19:10:01.434860 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:01.434720 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dts8z\" (UniqueName: \"kubernetes.io/projected/02ef5a7c-9326-42c8-9933-c2e53a151ff2-kube-api-access-dts8z\") pod \"kube-auth-proxy-685cdb786f-4pcjp\" (UID: \"02ef5a7c-9326-42c8-9933-c2e53a151ff2\") " pod="openshift-ingress/kube-auth-proxy-685cdb786f-4pcjp" Jul 08 19:10:01.434860 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:01.434755 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02ef5a7c-9326-42c8-9933-c2e53a151ff2-tls-certs\") pod \"kube-auth-proxy-685cdb786f-4pcjp\" (UID: \"02ef5a7c-9326-42c8-9933-c2e53a151ff2\") " pod="openshift-ingress/kube-auth-proxy-685cdb786f-4pcjp" Jul 08 19:10:01.434860 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:01.434784 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxhtm\" (UniqueName: \"kubernetes.io/projected/4e80abce-1fd7-43ca-942a-d23acb4b1f87-kube-api-access-rxhtm\") pod \"odh-model-controller-7998c67dc4-qzg9x\" (UID: \"4e80abce-1fd7-43ca-942a-d23acb4b1f87\") " pod="opendatahub/odh-model-controller-7998c67dc4-qzg9x" Jul 08 19:10:01.434860 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:01.434810 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/02ef5a7c-9326-42c8-9933-c2e53a151ff2-tmp\") pod \"kube-auth-proxy-685cdb786f-4pcjp\" (UID: \"02ef5a7c-9326-42c8-9933-c2e53a151ff2\") " pod="openshift-ingress/kube-auth-proxy-685cdb786f-4pcjp" Jul 08 19:10:01.434860 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:10:01.434818 2583 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 08 19:10:01.435095 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:10:01.434890 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e80abce-1fd7-43ca-942a-d23acb4b1f87-cert podName:4e80abce-1fd7-43ca-942a-d23acb4b1f87 nodeName:}" failed. No retries permitted until 2026-07-08 19:10:01.934869107 +0000 UTC m=+367.213956725 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/4e80abce-1fd7-43ca-942a-d23acb4b1f87-cert") pod "odh-model-controller-7998c67dc4-qzg9x" (UID: "4e80abce-1fd7-43ca-942a-d23acb4b1f87") : secret "odh-model-controller-webhook-cert" not found Jul 08 19:10:01.437231 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:01.437207 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/02ef5a7c-9326-42c8-9933-c2e53a151ff2-tmp\") pod \"kube-auth-proxy-685cdb786f-4pcjp\" (UID: \"02ef5a7c-9326-42c8-9933-c2e53a151ff2\") " pod="openshift-ingress/kube-auth-proxy-685cdb786f-4pcjp" Jul 08 19:10:01.437417 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:01.437399 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02ef5a7c-9326-42c8-9933-c2e53a151ff2-tls-certs\") pod \"kube-auth-proxy-685cdb786f-4pcjp\" (UID: \"02ef5a7c-9326-42c8-9933-c2e53a151ff2\") " pod="openshift-ingress/kube-auth-proxy-685cdb786f-4pcjp" Jul 08 19:10:01.448826 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:01.448773 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dts8z\" (UniqueName: \"kubernetes.io/projected/02ef5a7c-9326-42c8-9933-c2e53a151ff2-kube-api-access-dts8z\") pod \"kube-auth-proxy-685cdb786f-4pcjp\" (UID: \"02ef5a7c-9326-42c8-9933-c2e53a151ff2\") " pod="openshift-ingress/kube-auth-proxy-685cdb786f-4pcjp" Jul 08 19:10:01.450404 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:01.450381 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxhtm\" (UniqueName: \"kubernetes.io/projected/4e80abce-1fd7-43ca-942a-d23acb4b1f87-kube-api-access-rxhtm\") pod \"odh-model-controller-7998c67dc4-qzg9x\" (UID: \"4e80abce-1fd7-43ca-942a-d23acb4b1f87\") " pod="opendatahub/odh-model-controller-7998c67dc4-qzg9x" Jul 08 19:10:01.531452 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:01.531404 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-685cdb786f-4pcjp" Jul 08 19:10:01.678217 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:01.678186 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-685cdb786f-4pcjp"] Jul 08 19:10:01.681750 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:10:01.681723 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02ef5a7c_9326_42c8_9933_c2e53a151ff2.slice/crio-013724619aefac41c3612ebba142f20f1b13d99311c35c03cfa4af9b97a83cd9 WatchSource:0}: Error finding container 013724619aefac41c3612ebba142f20f1b13d99311c35c03cfa4af9b97a83cd9: Status 404 returned error can't find the container with id 013724619aefac41c3612ebba142f20f1b13d99311c35c03cfa4af9b97a83cd9 Jul 08 19:10:01.939774 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:01.939735 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4e80abce-1fd7-43ca-942a-d23acb4b1f87-cert\") pod \"odh-model-controller-7998c67dc4-qzg9x\" (UID: \"4e80abce-1fd7-43ca-942a-d23acb4b1f87\") " pod="opendatahub/odh-model-controller-7998c67dc4-qzg9x" Jul 08 19:10:01.942256 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:01.942231 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4e80abce-1fd7-43ca-942a-d23acb4b1f87-cert\") pod \"odh-model-controller-7998c67dc4-qzg9x\" (UID: \"4e80abce-1fd7-43ca-942a-d23acb4b1f87\") " pod="opendatahub/odh-model-controller-7998c67dc4-qzg9x" Jul 08 19:10:02.121536 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:02.121492 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-qzg9x" Jul 08 19:10:02.200711 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:02.200652 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-t4v2b"] Jul 08 19:10:02.204326 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:02.204305 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-t4v2b" Jul 08 19:10:02.207275 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:02.207206 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 08 19:10:02.223547 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:02.223461 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-t4v2b"] Jul 08 19:10:02.269226 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:02.269195 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-qzg9x"] Jul 08 19:10:02.273019 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:10:02.272995 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e80abce_1fd7_43ca_942a_d23acb4b1f87.slice/crio-984a746788f7efbe6fe87df4d55998bca98d66433971e59c05c8a2ce204a88c6 WatchSource:0}: Error finding container 984a746788f7efbe6fe87df4d55998bca98d66433971e59c05c8a2ce204a88c6: Status 404 returned error can't find the container with id 984a746788f7efbe6fe87df4d55998bca98d66433971e59c05c8a2ce204a88c6 Jul 08 19:10:02.342308 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:02.342267 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbkg9\" (UniqueName: \"kubernetes.io/projected/23b0aacd-a6a4-49a0-8881-77974f6465b1-kube-api-access-rbkg9\") pod \"model-serving-api-54d89bd5fc-t4v2b\" (UID: \"23b0aacd-a6a4-49a0-8881-77974f6465b1\") " pod="opendatahub/model-serving-api-54d89bd5fc-t4v2b" Jul 08 19:10:02.342308 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:02.342309 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23b0aacd-a6a4-49a0-8881-77974f6465b1-tls-certs\") pod \"model-serving-api-54d89bd5fc-t4v2b\" (UID: \"23b0aacd-a6a4-49a0-8881-77974f6465b1\") " pod="opendatahub/model-serving-api-54d89bd5fc-t4v2b" Jul 08 19:10:02.443687 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:02.443651 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbkg9\" (UniqueName: \"kubernetes.io/projected/23b0aacd-a6a4-49a0-8881-77974f6465b1-kube-api-access-rbkg9\") pod \"model-serving-api-54d89bd5fc-t4v2b\" (UID: \"23b0aacd-a6a4-49a0-8881-77974f6465b1\") " pod="opendatahub/model-serving-api-54d89bd5fc-t4v2b" Jul 08 19:10:02.443687 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:02.443687 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23b0aacd-a6a4-49a0-8881-77974f6465b1-tls-certs\") pod \"model-serving-api-54d89bd5fc-t4v2b\" (UID: \"23b0aacd-a6a4-49a0-8881-77974f6465b1\") " pod="opendatahub/model-serving-api-54d89bd5fc-t4v2b" Jul 08 19:10:02.446629 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:02.446605 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23b0aacd-a6a4-49a0-8881-77974f6465b1-tls-certs\") pod \"model-serving-api-54d89bd5fc-t4v2b\" (UID: \"23b0aacd-a6a4-49a0-8881-77974f6465b1\") " pod="opendatahub/model-serving-api-54d89bd5fc-t4v2b" Jul 08 19:10:02.465476 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:02.465415 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbkg9\" (UniqueName: \"kubernetes.io/projected/23b0aacd-a6a4-49a0-8881-77974f6465b1-kube-api-access-rbkg9\") pod \"model-serving-api-54d89bd5fc-t4v2b\" (UID: \"23b0aacd-a6a4-49a0-8881-77974f6465b1\") " pod="opendatahub/model-serving-api-54d89bd5fc-t4v2b" Jul 08 19:10:02.502711 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:02.502676 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-qzg9x" event={"ID":"4e80abce-1fd7-43ca-942a-d23acb4b1f87","Type":"ContainerStarted","Data":"984a746788f7efbe6fe87df4d55998bca98d66433971e59c05c8a2ce204a88c6"} Jul 08 19:10:02.503844 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:02.503818 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-685cdb786f-4pcjp" event={"ID":"02ef5a7c-9326-42c8-9933-c2e53a151ff2","Type":"ContainerStarted","Data":"013724619aefac41c3612ebba142f20f1b13d99311c35c03cfa4af9b97a83cd9"} Jul 08 19:10:02.518114 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:02.518096 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-t4v2b" Jul 08 19:10:02.649541 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:02.649507 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-lxxp5"] Jul 08 19:10:02.652752 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:02.652728 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lxxp5" Jul 08 19:10:02.653732 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:02.653707 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-t4v2b"] Jul 08 19:10:02.655278 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:02.655252 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 08 19:10:02.655278 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:02.655261 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 08 19:10:02.655515 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:02.655484 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-5lfgg\"" Jul 08 19:10:02.655654 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:02.655639 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 08 19:10:02.655769 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:02.655755 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 08 19:10:02.656486 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:10:02.656463 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23b0aacd_a6a4_49a0_8881_77974f6465b1.slice/crio-5d8c44055d97c95f0250ea43d626089d4769157580789f8c016d6b02aae7b86b WatchSource:0}: Error finding container 5d8c44055d97c95f0250ea43d626089d4769157580789f8c016d6b02aae7b86b: Status 404 returned error can't find the container with id 5d8c44055d97c95f0250ea43d626089d4769157580789f8c016d6b02aae7b86b Jul 08 19:10:02.670549 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:02.670521 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-lxxp5"] Jul 08 19:10:02.746013 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:02.745929 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/729b8f3d-4260-4e07-8d14-a6d2c1fef820-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-lxxp5\" (UID: \"729b8f3d-4260-4e07-8d14-a6d2c1fef820\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lxxp5" Jul 08 19:10:02.746013 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:02.745972 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/729b8f3d-4260-4e07-8d14-a6d2c1fef820-istio-token\") pod \"istiod-openshift-gateway-7574459656-lxxp5\" (UID: \"729b8f3d-4260-4e07-8d14-a6d2c1fef820\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lxxp5" Jul 08 19:10:02.746232 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:02.746013 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/729b8f3d-4260-4e07-8d14-a6d2c1fef820-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-lxxp5\" (UID: \"729b8f3d-4260-4e07-8d14-a6d2c1fef820\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lxxp5" Jul 08 19:10:02.746232 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:02.746043 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/729b8f3d-4260-4e07-8d14-a6d2c1fef820-cacerts\") pod \"istiod-openshift-gateway-7574459656-lxxp5\" (UID: \"729b8f3d-4260-4e07-8d14-a6d2c1fef820\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lxxp5" Jul 08 19:10:02.746232 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:02.746071 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qv6xt\" (UniqueName: \"kubernetes.io/projected/729b8f3d-4260-4e07-8d14-a6d2c1fef820-kube-api-access-qv6xt\") pod \"istiod-openshift-gateway-7574459656-lxxp5\" (UID: \"729b8f3d-4260-4e07-8d14-a6d2c1fef820\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lxxp5" Jul 08 19:10:02.746232 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:02.746124 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/729b8f3d-4260-4e07-8d14-a6d2c1fef820-local-certs\") pod \"istiod-openshift-gateway-7574459656-lxxp5\" (UID: \"729b8f3d-4260-4e07-8d14-a6d2c1fef820\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lxxp5" Jul 08 19:10:02.746232 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:02.746157 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/729b8f3d-4260-4e07-8d14-a6d2c1fef820-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-lxxp5\" (UID: \"729b8f3d-4260-4e07-8d14-a6d2c1fef820\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lxxp5" Jul 08 19:10:02.847413 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:02.847379 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/729b8f3d-4260-4e07-8d14-a6d2c1fef820-istio-token\") pod \"istiod-openshift-gateway-7574459656-lxxp5\" (UID: \"729b8f3d-4260-4e07-8d14-a6d2c1fef820\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lxxp5" Jul 08 19:10:02.847413 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:02.847413 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/729b8f3d-4260-4e07-8d14-a6d2c1fef820-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-lxxp5\" (UID: \"729b8f3d-4260-4e07-8d14-a6d2c1fef820\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lxxp5" Jul 08 19:10:02.847667 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:02.847451 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/729b8f3d-4260-4e07-8d14-a6d2c1fef820-cacerts\") pod \"istiod-openshift-gateway-7574459656-lxxp5\" (UID: \"729b8f3d-4260-4e07-8d14-a6d2c1fef820\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lxxp5" Jul 08 19:10:02.847667 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:02.847477 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qv6xt\" (UniqueName: \"kubernetes.io/projected/729b8f3d-4260-4e07-8d14-a6d2c1fef820-kube-api-access-qv6xt\") pod \"istiod-openshift-gateway-7574459656-lxxp5\" (UID: \"729b8f3d-4260-4e07-8d14-a6d2c1fef820\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lxxp5" Jul 08 19:10:02.847667 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:02.847520 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/729b8f3d-4260-4e07-8d14-a6d2c1fef820-local-certs\") pod \"istiod-openshift-gateway-7574459656-lxxp5\" (UID: \"729b8f3d-4260-4e07-8d14-a6d2c1fef820\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lxxp5" Jul 08 19:10:02.847667 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:02.847552 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/729b8f3d-4260-4e07-8d14-a6d2c1fef820-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-lxxp5\" (UID: \"729b8f3d-4260-4e07-8d14-a6d2c1fef820\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lxxp5" Jul 08 19:10:02.847667 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:02.847614 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/729b8f3d-4260-4e07-8d14-a6d2c1fef820-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-lxxp5\" (UID: \"729b8f3d-4260-4e07-8d14-a6d2c1fef820\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lxxp5" Jul 08 19:10:02.848603 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:02.848572 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/729b8f3d-4260-4e07-8d14-a6d2c1fef820-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-lxxp5\" (UID: \"729b8f3d-4260-4e07-8d14-a6d2c1fef820\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lxxp5" Jul 08 19:10:02.850233 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:02.850210 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/729b8f3d-4260-4e07-8d14-a6d2c1fef820-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-lxxp5\" (UID: \"729b8f3d-4260-4e07-8d14-a6d2c1fef820\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lxxp5" Jul 08 19:10:02.850336 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:02.850273 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/729b8f3d-4260-4e07-8d14-a6d2c1fef820-cacerts\") pod \"istiod-openshift-gateway-7574459656-lxxp5\" (UID: \"729b8f3d-4260-4e07-8d14-a6d2c1fef820\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lxxp5" Jul 08 19:10:02.850536 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:02.850514 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/729b8f3d-4260-4e07-8d14-a6d2c1fef820-local-certs\") pod \"istiod-openshift-gateway-7574459656-lxxp5\" (UID: \"729b8f3d-4260-4e07-8d14-a6d2c1fef820\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lxxp5" Jul 08 19:10:02.850599 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:02.850523 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/729b8f3d-4260-4e07-8d14-a6d2c1fef820-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-lxxp5\" (UID: \"729b8f3d-4260-4e07-8d14-a6d2c1fef820\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lxxp5" Jul 08 19:10:02.859186 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:02.859160 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qv6xt\" (UniqueName: \"kubernetes.io/projected/729b8f3d-4260-4e07-8d14-a6d2c1fef820-kube-api-access-qv6xt\") pod \"istiod-openshift-gateway-7574459656-lxxp5\" (UID: \"729b8f3d-4260-4e07-8d14-a6d2c1fef820\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lxxp5" Jul 08 19:10:02.859342 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:02.859320 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/729b8f3d-4260-4e07-8d14-a6d2c1fef820-istio-token\") pod \"istiod-openshift-gateway-7574459656-lxxp5\" (UID: \"729b8f3d-4260-4e07-8d14-a6d2c1fef820\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-lxxp5" Jul 08 19:10:02.963189 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:02.963156 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lxxp5" Jul 08 19:10:03.122387 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:03.122312 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-lxxp5"] Jul 08 19:10:03.127041 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:10:03.127003 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod729b8f3d_4260_4e07_8d14_a6d2c1fef820.slice/crio-30c81fc5bb7d6ca0add605dc62dd25d05ef827204f10df474471f238761ee548 WatchSource:0}: Error finding container 30c81fc5bb7d6ca0add605dc62dd25d05ef827204f10df474471f238761ee548: Status 404 returned error can't find the container with id 30c81fc5bb7d6ca0add605dc62dd25d05ef827204f10df474471f238761ee548 Jul 08 19:10:03.510788 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:03.510742 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-t4v2b" event={"ID":"23b0aacd-a6a4-49a0-8881-77974f6465b1","Type":"ContainerStarted","Data":"5d8c44055d97c95f0250ea43d626089d4769157580789f8c016d6b02aae7b86b"} Jul 08 19:10:03.513467 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:03.513411 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lxxp5" event={"ID":"729b8f3d-4260-4e07-8d14-a6d2c1fef820","Type":"ContainerStarted","Data":"30c81fc5bb7d6ca0add605dc62dd25d05ef827204f10df474471f238761ee548"} Jul 08 19:10:07.672288 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:07.672250 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:10:07.672596 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:07.672333 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:10:08.535328 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:08.535283 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-qzg9x" event={"ID":"4e80abce-1fd7-43ca-942a-d23acb4b1f87","Type":"ContainerStarted","Data":"05e742a8ed5c653746ed4491beacdb517e1c7ad229f169a8f11c4f1684b88d2e"} Jul 08 19:10:08.535552 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:08.535397 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-qzg9x" Jul 08 19:10:08.536806 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:08.536783 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-685cdb786f-4pcjp" event={"ID":"02ef5a7c-9326-42c8-9933-c2e53a151ff2","Type":"ContainerStarted","Data":"88bdb5d1ae0501331477874b14ce1924b0a37081500e16298a089d55bf881e14"} Jul 08 19:10:08.538273 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:08.538253 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-t4v2b" event={"ID":"23b0aacd-a6a4-49a0-8881-77974f6465b1","Type":"ContainerStarted","Data":"75681c100b555bdca6fb7ec0672254be7c586349d2c82e349059b5ff19c07de2"} Jul 08 19:10:08.538381 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:08.538362 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-t4v2b" Jul 08 19:10:08.539649 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:08.539628 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lxxp5" event={"ID":"729b8f3d-4260-4e07-8d14-a6d2c1fef820","Type":"ContainerStarted","Data":"c96e415f59b43ee66b8433bbfbfa6a22da79de0081ded36d74f09b0f79c85c52"} Jul 08 19:10:08.539727 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:08.539676 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lxxp5" Jul 08 19:10:08.541488 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:08.541468 2583 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-lxxp5 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 08 19:10:08.541572 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:08.541512 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lxxp5" podUID="729b8f3d-4260-4e07-8d14-a6d2c1fef820" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 19:10:08.563619 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:08.563575 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-qzg9x" podStartSLOduration=2.178016149 podStartE2EDuration="7.563562459s" totalImagesPullingTime="5.38554631s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:10:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:10:08.562836487 +0000 UTC m=+373.841924107" watchObservedRunningTime="2026-07-08 19:10:08.563562459 +0000 UTC m=+373.842650079" Jul 08 19:10:08.588379 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:08.588338 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-685cdb786f-4pcjp" podStartSLOduration=1.604977859 podStartE2EDuration="7.588323914s" totalImagesPullingTime="5.983346055s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:10:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:10:08.587758303 +0000 UTC m=+373.866845927" watchObservedRunningTime="2026-07-08 19:10:08.588323914 +0000 UTC m=+373.867411533" Jul 08 19:10:08.673568 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:08.673507 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-t4v2b" podStartSLOduration=1.668057445 podStartE2EDuration="6.673486214s" totalImagesPullingTime="5.005428769s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:10:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:10:08.672312829 +0000 UTC m=+373.951400451" watchObservedRunningTime="2026-07-08 19:10:08.673486214 +0000 UTC m=+373.952573835" Jul 08 19:10:08.735662 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:08.735589 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lxxp5" podStartSLOduration=2.193431736 podStartE2EDuration="6.735567379s" totalImagesPullingTime="4.542135643s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:10:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:10:08.735087917 +0000 UTC m=+374.014175530" watchObservedRunningTime="2026-07-08 19:10:08.735567379 +0000 UTC m=+374.014654994" Jul 08 19:10:09.344356 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:09.344322 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44g28w"] Jul 08 19:10:09.348104 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:09.348083 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44g28w" Jul 08 19:10:09.351921 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:09.351899 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-wdczs\"" Jul 08 19:10:09.364398 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:09.364370 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44g28w"] Jul 08 19:10:09.501743 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:09.501708 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44g28w\" (UID: \"cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44g28w" Jul 08 19:10:09.501743 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:09.501745 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44g28w\" (UID: \"cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44g28w" Jul 08 19:10:09.501964 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:09.501773 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44g28w\" (UID: \"cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44g28w" Jul 08 19:10:09.501964 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:09.501880 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44g28w\" (UID: \"cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44g28w" Jul 08 19:10:09.502072 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:09.501986 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44g28w\" (UID: \"cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44g28w" Jul 08 19:10:09.502072 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:09.502027 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44g28w\" (UID: \"cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44g28w" Jul 08 19:10:09.502153 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:09.502081 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44g28w\" (UID: \"cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44g28w" Jul 08 19:10:09.502153 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:09.502099 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44g28w\" (UID: \"cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44g28w" Jul 08 19:10:09.502153 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:09.502122 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fgpm\" (UniqueName: \"kubernetes.io/projected/cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a-kube-api-access-8fgpm\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44g28w\" (UID: \"cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44g28w" Jul 08 19:10:09.543759 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:09.543738 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lxxp5" Jul 08 19:10:09.603789 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:09.603037 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44g28w\" (UID: \"cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44g28w" Jul 08 19:10:09.603789 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:09.603100 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44g28w\" (UID: \"cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44g28w" Jul 08 19:10:09.603789 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:09.603129 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44g28w\" (UID: \"cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44g28w" Jul 08 19:10:09.603789 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:09.603173 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44g28w\" (UID: \"cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44g28w" Jul 08 19:10:09.603789 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:09.603202 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44g28w\" (UID: \"cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44g28w" Jul 08 19:10:09.603789 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:09.603238 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fgpm\" (UniqueName: \"kubernetes.io/projected/cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a-kube-api-access-8fgpm\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44g28w\" (UID: \"cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44g28w" Jul 08 19:10:09.603789 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:09.603281 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44g28w\" (UID: \"cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44g28w" Jul 08 19:10:09.603789 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:09.603308 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44g28w\" (UID: \"cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44g28w" Jul 08 19:10:09.603789 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:09.603348 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44g28w\" (UID: \"cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44g28w" Jul 08 19:10:09.604355 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:09.603863 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44g28w\" (UID: \"cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44g28w" Jul 08 19:10:09.604355 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:09.603942 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44g28w\" (UID: \"cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44g28w" Jul 08 19:10:09.604998 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:09.604973 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44g28w\" (UID: \"cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44g28w" Jul 08 19:10:09.605366 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:09.605346 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44g28w\" (UID: \"cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44g28w" Jul 08 19:10:09.608086 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:09.608057 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44g28w\" (UID: \"cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44g28w" Jul 08 19:10:09.608672 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:09.608629 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44g28w\" (UID: \"cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44g28w" Jul 08 19:10:09.609766 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:09.609737 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44g28w\" (UID: \"cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44g28w" Jul 08 19:10:09.620803 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:09.620779 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44g28w\" (UID: \"cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44g28w" Jul 08 19:10:09.621343 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:09.621311 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fgpm\" (UniqueName: \"kubernetes.io/projected/cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a-kube-api-access-8fgpm\") pod \"data-science-gateway-data-science-gateway-class-6576859cf44g28w\" (UID: \"cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44g28w" Jul 08 19:10:09.659476 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:09.659422 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44g28w" Jul 08 19:10:09.800991 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:09.800965 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44g28w"] Jul 08 19:10:09.802410 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:10:09.802383 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbd46bf8_62f9_4d3a_96e5_ef0544a2d91a.slice/crio-d477d1d3761dfbbd0df0d451c539c4d86ad7231145ea8d9c699a04654ebb343c WatchSource:0}: Error finding container d477d1d3761dfbbd0df0d451c539c4d86ad7231145ea8d9c699a04654ebb343c: Status 404 returned error can't find the container with id d477d1d3761dfbbd0df0d451c539c4d86ad7231145ea8d9c699a04654ebb343c Jul 08 19:10:10.548794 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:10.548758 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44g28w" event={"ID":"cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a","Type":"ContainerStarted","Data":"d477d1d3761dfbbd0df0d451c539c4d86ad7231145ea8d9c699a04654ebb343c"} Jul 08 19:10:12.402668 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:12.402635 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:10:12.402935 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:12.402702 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:10:12.402935 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:12.402730 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:10:12.561933 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:12.561884 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44g28w" event={"ID":"cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a","Type":"ContainerStarted","Data":"7e4c11f91c327592fa6a209b0f05da98f2c808370513655f1e50420506004fff"} Jul 08 19:10:12.602939 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:12.602631 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44g28w" podStartSLOduration=1.004668891 podStartE2EDuration="3.602607462s" totalImagesPullingTime="2.597938571s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:10:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:10:12.602019161 +0000 UTC m=+377.881106852" watchObservedRunningTime="2026-07-08 19:10:12.602607462 +0000 UTC m=+377.881695083" Jul 08 19:10:12.663823 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:12.663793 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44g28w" Jul 08 19:10:12.673337 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:12.673307 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-xx469"] Jul 08 19:10:12.676952 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:12.676934 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xx469" Jul 08 19:10:12.681676 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:12.681507 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 08 19:10:12.681676 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:12.681565 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-t2jc2\"" Jul 08 19:10:12.695036 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:12.695011 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-xx469"] Jul 08 19:10:12.724933 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:12.724898 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-sjflv"] Jul 08 19:10:12.728123 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:12.728101 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sjflv" Jul 08 19:10:12.744979 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:12.744960 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 08 19:10:12.745163 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:12.745146 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-ppcgk\"" Jul 08 19:10:12.790338 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:12.790306 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-sjflv"] Jul 08 19:10:12.832307 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:12.832273 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqltk\" (UniqueName: \"kubernetes.io/projected/5d6c1088-bfef-4914-8c2e-d444040c89aa-kube-api-access-wqltk\") pod \"llmisvc-controller-manager-777c6bd7d5-sjflv\" (UID: \"5d6c1088-bfef-4914-8c2e-d444040c89aa\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sjflv" Jul 08 19:10:12.832485 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:12.832324 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5d6c1088-bfef-4914-8c2e-d444040c89aa-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-sjflv\" (UID: \"5d6c1088-bfef-4914-8c2e-d444040c89aa\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sjflv" Jul 08 19:10:12.832485 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:12.832363 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7ac398a1-e561-4a15-9217-4ae93da132f9-cert\") pod \"kserve-controller-manager-d6f4b65b8-xx469\" (UID: \"7ac398a1-e561-4a15-9217-4ae93da132f9\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xx469" Jul 08 19:10:12.832485 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:12.832456 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-299s7\" (UniqueName: \"kubernetes.io/projected/7ac398a1-e561-4a15-9217-4ae93da132f9-kube-api-access-299s7\") pod \"kserve-controller-manager-d6f4b65b8-xx469\" (UID: \"7ac398a1-e561-4a15-9217-4ae93da132f9\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xx469" Jul 08 19:10:12.933083 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:12.932974 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-299s7\" (UniqueName: \"kubernetes.io/projected/7ac398a1-e561-4a15-9217-4ae93da132f9-kube-api-access-299s7\") pod \"kserve-controller-manager-d6f4b65b8-xx469\" (UID: \"7ac398a1-e561-4a15-9217-4ae93da132f9\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xx469" Jul 08 19:10:12.933083 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:12.933014 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqltk\" (UniqueName: \"kubernetes.io/projected/5d6c1088-bfef-4914-8c2e-d444040c89aa-kube-api-access-wqltk\") pod \"llmisvc-controller-manager-777c6bd7d5-sjflv\" (UID: \"5d6c1088-bfef-4914-8c2e-d444040c89aa\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sjflv" Jul 08 19:10:12.933083 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:12.933050 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5d6c1088-bfef-4914-8c2e-d444040c89aa-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-sjflv\" (UID: \"5d6c1088-bfef-4914-8c2e-d444040c89aa\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sjflv" Jul 08 19:10:12.933083 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:12.933084 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7ac398a1-e561-4a15-9217-4ae93da132f9-cert\") pod \"kserve-controller-manager-d6f4b65b8-xx469\" (UID: \"7ac398a1-e561-4a15-9217-4ae93da132f9\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xx469" Jul 08 19:10:12.935808 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:12.935784 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5d6c1088-bfef-4914-8c2e-d444040c89aa-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-sjflv\" (UID: \"5d6c1088-bfef-4914-8c2e-d444040c89aa\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sjflv" Jul 08 19:10:12.935930 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:12.935907 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7ac398a1-e561-4a15-9217-4ae93da132f9-cert\") pod \"kserve-controller-manager-d6f4b65b8-xx469\" (UID: \"7ac398a1-e561-4a15-9217-4ae93da132f9\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xx469" Jul 08 19:10:12.967575 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:12.967521 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqltk\" (UniqueName: \"kubernetes.io/projected/5d6c1088-bfef-4914-8c2e-d444040c89aa-kube-api-access-wqltk\") pod \"llmisvc-controller-manager-777c6bd7d5-sjflv\" (UID: \"5d6c1088-bfef-4914-8c2e-d444040c89aa\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sjflv" Jul 08 19:10:12.968746 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:12.968722 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-299s7\" (UniqueName: \"kubernetes.io/projected/7ac398a1-e561-4a15-9217-4ae93da132f9-kube-api-access-299s7\") pod \"kserve-controller-manager-d6f4b65b8-xx469\" (UID: \"7ac398a1-e561-4a15-9217-4ae93da132f9\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xx469" Jul 08 19:10:12.988728 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:12.988689 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xx469" Jul 08 19:10:13.049759 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:13.040779 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sjflv" Jul 08 19:10:13.176468 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:13.176417 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-lxxp5"] Jul 08 19:10:13.176773 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:13.176742 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lxxp5" podUID="729b8f3d-4260-4e07-8d14-a6d2c1fef820" containerName="discovery" containerID="cri-o://c96e415f59b43ee66b8433bbfbfa6a22da79de0081ded36d74f09b0f79c85c52" gracePeriod=30 Jul 08 19:10:13.186060 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:13.186034 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-xx469"] Jul 08 19:10:13.188055 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:10:13.188026 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ac398a1_e561_4a15_9217_4ae93da132f9.slice/crio-43da4ffc49bb290581deae3f736711c98df7f67e1bdcb63bef08a167ee8fcd09 WatchSource:0}: Error finding container 43da4ffc49bb290581deae3f736711c98df7f67e1bdcb63bef08a167ee8fcd09: Status 404 returned error can't find the container with id 43da4ffc49bb290581deae3f736711c98df7f67e1bdcb63bef08a167ee8fcd09 Jul 08 19:10:13.253145 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:13.253008 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-sjflv"] Jul 08 19:10:13.273509 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:10:13.273482 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod5d6c1088_bfef_4914_8c2e_d444040c89aa.slice/crio-b3359f5cf08d80b3006ed1a71689da9d58b67dd89e1732035c0acb822dcab394 WatchSource:0}: Error finding container b3359f5cf08d80b3006ed1a71689da9d58b67dd89e1732035c0acb822dcab394: Status 404 returned error can't find the container with id b3359f5cf08d80b3006ed1a71689da9d58b67dd89e1732035c0acb822dcab394 Jul 08 19:10:13.415982 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:13.415958 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lxxp5" Jul 08 19:10:13.538929 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:13.538845 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/729b8f3d-4260-4e07-8d14-a6d2c1fef820-istio-token\") pod \"729b8f3d-4260-4e07-8d14-a6d2c1fef820\" (UID: \"729b8f3d-4260-4e07-8d14-a6d2c1fef820\") " Jul 08 19:10:13.538929 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:13.538903 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/729b8f3d-4260-4e07-8d14-a6d2c1fef820-istio-kubeconfig\") pod \"729b8f3d-4260-4e07-8d14-a6d2c1fef820\" (UID: \"729b8f3d-4260-4e07-8d14-a6d2c1fef820\") " Jul 08 19:10:13.539143 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:13.538943 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qv6xt\" (UniqueName: \"kubernetes.io/projected/729b8f3d-4260-4e07-8d14-a6d2c1fef820-kube-api-access-qv6xt\") pod \"729b8f3d-4260-4e07-8d14-a6d2c1fef820\" (UID: \"729b8f3d-4260-4e07-8d14-a6d2c1fef820\") " Jul 08 19:10:13.539143 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:13.538970 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/729b8f3d-4260-4e07-8d14-a6d2c1fef820-local-certs\") pod \"729b8f3d-4260-4e07-8d14-a6d2c1fef820\" (UID: \"729b8f3d-4260-4e07-8d14-a6d2c1fef820\") " Jul 08 19:10:13.539143 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:13.538990 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/729b8f3d-4260-4e07-8d14-a6d2c1fef820-cacerts\") pod \"729b8f3d-4260-4e07-8d14-a6d2c1fef820\" (UID: \"729b8f3d-4260-4e07-8d14-a6d2c1fef820\") " Jul 08 19:10:13.539143 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:13.539006 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/729b8f3d-4260-4e07-8d14-a6d2c1fef820-istio-csr-ca-configmap\") pod \"729b8f3d-4260-4e07-8d14-a6d2c1fef820\" (UID: \"729b8f3d-4260-4e07-8d14-a6d2c1fef820\") " Jul 08 19:10:13.539143 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:13.539026 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/729b8f3d-4260-4e07-8d14-a6d2c1fef820-istio-csr-dns-cert\") pod \"729b8f3d-4260-4e07-8d14-a6d2c1fef820\" (UID: \"729b8f3d-4260-4e07-8d14-a6d2c1fef820\") " Jul 08 19:10:13.539592 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:13.539535 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/729b8f3d-4260-4e07-8d14-a6d2c1fef820-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "729b8f3d-4260-4e07-8d14-a6d2c1fef820" (UID: "729b8f3d-4260-4e07-8d14-a6d2c1fef820"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:10:13.541506 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:13.541463 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/729b8f3d-4260-4e07-8d14-a6d2c1fef820-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "729b8f3d-4260-4e07-8d14-a6d2c1fef820" (UID: "729b8f3d-4260-4e07-8d14-a6d2c1fef820"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:10:13.541506 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:13.541492 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/729b8f3d-4260-4e07-8d14-a6d2c1fef820-cacerts" (OuterVolumeSpecName: "cacerts") pod "729b8f3d-4260-4e07-8d14-a6d2c1fef820" (UID: "729b8f3d-4260-4e07-8d14-a6d2c1fef820"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:10:13.541748 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:13.541723 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/729b8f3d-4260-4e07-8d14-a6d2c1fef820-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "729b8f3d-4260-4e07-8d14-a6d2c1fef820" (UID: "729b8f3d-4260-4e07-8d14-a6d2c1fef820"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:10:13.541839 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:13.541815 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/729b8f3d-4260-4e07-8d14-a6d2c1fef820-istio-token" (OuterVolumeSpecName: "istio-token") pod "729b8f3d-4260-4e07-8d14-a6d2c1fef820" (UID: "729b8f3d-4260-4e07-8d14-a6d2c1fef820"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:10:13.541881 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:13.541847 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/729b8f3d-4260-4e07-8d14-a6d2c1fef820-local-certs" (OuterVolumeSpecName: "local-certs") pod "729b8f3d-4260-4e07-8d14-a6d2c1fef820" (UID: "729b8f3d-4260-4e07-8d14-a6d2c1fef820"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:10:13.541914 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:13.541889 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/729b8f3d-4260-4e07-8d14-a6d2c1fef820-kube-api-access-qv6xt" (OuterVolumeSpecName: "kube-api-access-qv6xt") pod "729b8f3d-4260-4e07-8d14-a6d2c1fef820" (UID: "729b8f3d-4260-4e07-8d14-a6d2c1fef820"). InnerVolumeSpecName "kube-api-access-qv6xt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:10:13.567259 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:13.567226 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sjflv" event={"ID":"5d6c1088-bfef-4914-8c2e-d444040c89aa","Type":"ContainerStarted","Data":"b3359f5cf08d80b3006ed1a71689da9d58b67dd89e1732035c0acb822dcab394"} Jul 08 19:10:13.568477 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:13.568455 2583 generic.go:356] "Generic (PLEG): container finished" podID="729b8f3d-4260-4e07-8d14-a6d2c1fef820" containerID="c96e415f59b43ee66b8433bbfbfa6a22da79de0081ded36d74f09b0f79c85c52" exitCode=0 Jul 08 19:10:13.568588 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:13.568491 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lxxp5" Jul 08 19:10:13.568588 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:13.568534 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lxxp5" event={"ID":"729b8f3d-4260-4e07-8d14-a6d2c1fef820","Type":"ContainerDied","Data":"c96e415f59b43ee66b8433bbfbfa6a22da79de0081ded36d74f09b0f79c85c52"} Jul 08 19:10:13.568588 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:13.568580 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-lxxp5" event={"ID":"729b8f3d-4260-4e07-8d14-a6d2c1fef820","Type":"ContainerDied","Data":"30c81fc5bb7d6ca0add605dc62dd25d05ef827204f10df474471f238761ee548"} Jul 08 19:10:13.568754 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:13.568603 2583 scope.go:117] "RemoveContainer" containerID="c96e415f59b43ee66b8433bbfbfa6a22da79de0081ded36d74f09b0f79c85c52" Jul 08 19:10:13.569834 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:13.569804 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xx469" event={"ID":"7ac398a1-e561-4a15-9217-4ae93da132f9","Type":"ContainerStarted","Data":"43da4ffc49bb290581deae3f736711c98df7f67e1bdcb63bef08a167ee8fcd09"} Jul 08 19:10:13.577636 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:13.577620 2583 scope.go:117] "RemoveContainer" containerID="c96e415f59b43ee66b8433bbfbfa6a22da79de0081ded36d74f09b0f79c85c52" Jul 08 19:10:13.577894 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:10:13.577873 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c96e415f59b43ee66b8433bbfbfa6a22da79de0081ded36d74f09b0f79c85c52\": container with ID starting with c96e415f59b43ee66b8433bbfbfa6a22da79de0081ded36d74f09b0f79c85c52 not found: ID does not exist" containerID="c96e415f59b43ee66b8433bbfbfa6a22da79de0081ded36d74f09b0f79c85c52" Jul 08 19:10:13.577937 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:13.577902 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c96e415f59b43ee66b8433bbfbfa6a22da79de0081ded36d74f09b0f79c85c52"} err="failed to get container status \"c96e415f59b43ee66b8433bbfbfa6a22da79de0081ded36d74f09b0f79c85c52\": rpc error: code = NotFound desc = could not find container \"c96e415f59b43ee66b8433bbfbfa6a22da79de0081ded36d74f09b0f79c85c52\": container with ID starting with c96e415f59b43ee66b8433bbfbfa6a22da79de0081ded36d74f09b0f79c85c52 not found: ID does not exist" Jul 08 19:10:13.605112 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:13.605088 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-lxxp5"] Jul 08 19:10:13.617010 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:13.616985 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-lxxp5"] Jul 08 19:10:13.640595 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:13.640570 2583 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/729b8f3d-4260-4e07-8d14-a6d2c1fef820-istio-kubeconfig\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:10:13.640595 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:13.640593 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qv6xt\" (UniqueName: \"kubernetes.io/projected/729b8f3d-4260-4e07-8d14-a6d2c1fef820-kube-api-access-qv6xt\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:10:13.640762 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:13.640605 2583 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/729b8f3d-4260-4e07-8d14-a6d2c1fef820-local-certs\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:10:13.640762 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:13.640616 2583 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/729b8f3d-4260-4e07-8d14-a6d2c1fef820-cacerts\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:10:13.640762 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:13.640625 2583 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/729b8f3d-4260-4e07-8d14-a6d2c1fef820-istio-csr-ca-configmap\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:10:13.640762 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:13.640633 2583 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/729b8f3d-4260-4e07-8d14-a6d2c1fef820-istio-csr-dns-cert\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:10:13.640762 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:13.640641 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/729b8f3d-4260-4e07-8d14-a6d2c1fef820-istio-token\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:10:13.665700 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:13.665677 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44g28w" Jul 08 19:10:14.575788 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:14.575760 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44g28w" Jul 08 19:10:14.576812 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:14.576793 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf44g28w" Jul 08 19:10:15.175132 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:15.175094 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="729b8f3d-4260-4e07-8d14-a6d2c1fef820" path="/var/lib/kubelet/pods/729b8f3d-4260-4e07-8d14-a6d2c1fef820/volumes" Jul 08 19:10:17.592739 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:17.592644 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xx469" event={"ID":"7ac398a1-e561-4a15-9217-4ae93da132f9","Type":"ContainerStarted","Data":"ac03406f60437aecd86393b1d33094552f468c023ed6ce7ecd62e6c47cf730b4"} Jul 08 19:10:17.593158 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:17.592753 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xx469" Jul 08 19:10:17.594242 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:17.594219 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sjflv" event={"ID":"5d6c1088-bfef-4914-8c2e-d444040c89aa","Type":"ContainerStarted","Data":"fe00828ab6fdd93a548a15680edf181fd2ef61cfb1adaf90962d93de11ebd5e3"} Jul 08 19:10:17.594368 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:17.594357 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sjflv" Jul 08 19:10:17.642924 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:17.642878 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xx469" podStartSLOduration=1.647108901 podStartE2EDuration="5.642859628s" totalImagesPullingTime="3.995750727s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:10:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:10:17.641972901 +0000 UTC m=+382.921060521" watchObservedRunningTime="2026-07-08 19:10:17.642859628 +0000 UTC m=+382.921947249" Jul 08 19:10:17.665177 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:17.665130 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sjflv" podStartSLOduration=1.762963347 podStartE2EDuration="5.665115969s" totalImagesPullingTime="3.902152622s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:10:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:10:17.663489493 +0000 UTC m=+382.942577117" watchObservedRunningTime="2026-07-08 19:10:17.665115969 +0000 UTC m=+382.944203590" Jul 08 19:10:19.545145 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:19.545114 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-qzg9x" Jul 08 19:10:19.547299 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:19.547279 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-t4v2b" Jul 08 19:10:48.600058 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:48.600025 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-sjflv" Jul 08 19:10:48.603106 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:10:48.603083 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xx469" Jul 08 19:11:02.182892 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:02.182859 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4lw7p"] Jul 08 19:11:02.183360 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:02.183292 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="729b8f3d-4260-4e07-8d14-a6d2c1fef820" containerName="discovery" Jul 08 19:11:02.183360 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:02.183311 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="729b8f3d-4260-4e07-8d14-a6d2c1fef820" containerName="discovery" Jul 08 19:11:02.183509 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:02.183407 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="729b8f3d-4260-4e07-8d14-a6d2c1fef820" containerName="discovery" Jul 08 19:11:02.193184 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:02.193155 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-4lw7p" Jul 08 19:11:02.196465 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:02.196426 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-mvdxb\"" Jul 08 19:11:02.197491 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:02.197467 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 08 19:11:02.197491 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:02.197483 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 08 19:11:02.203298 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:02.203276 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4lw7p"] Jul 08 19:11:02.364893 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:02.364854 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5j8fg\" (UniqueName: \"kubernetes.io/projected/e1f3c964-0b63-4bab-8fdf-068e8807a797-kube-api-access-5j8fg\") pod \"kuadrant-operator-catalog-4lw7p\" (UID: \"e1f3c964-0b63-4bab-8fdf-068e8807a797\") " pod="kuadrant-system/kuadrant-operator-catalog-4lw7p" Jul 08 19:11:02.465485 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:02.465359 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5j8fg\" (UniqueName: \"kubernetes.io/projected/e1f3c964-0b63-4bab-8fdf-068e8807a797-kube-api-access-5j8fg\") pod \"kuadrant-operator-catalog-4lw7p\" (UID: \"e1f3c964-0b63-4bab-8fdf-068e8807a797\") " pod="kuadrant-system/kuadrant-operator-catalog-4lw7p" Jul 08 19:11:02.475017 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:02.474984 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5j8fg\" (UniqueName: \"kubernetes.io/projected/e1f3c964-0b63-4bab-8fdf-068e8807a797-kube-api-access-5j8fg\") pod \"kuadrant-operator-catalog-4lw7p\" (UID: \"e1f3c964-0b63-4bab-8fdf-068e8807a797\") " pod="kuadrant-system/kuadrant-operator-catalog-4lw7p" Jul 08 19:11:02.493713 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:02.493687 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4lw7p"] Jul 08 19:11:02.493946 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:02.493933 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-4lw7p" Jul 08 19:11:02.630466 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:02.630416 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4lw7p"] Jul 08 19:11:02.633227 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:11:02.633201 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1f3c964_0b63_4bab_8fdf_068e8807a797.slice/crio-caeea3d9f36c851084b6932196df4edbcd6b55a68a17c4218a5bd44811fc3e72 WatchSource:0}: Error finding container caeea3d9f36c851084b6932196df4edbcd6b55a68a17c4218a5bd44811fc3e72: Status 404 returned error can't find the container with id caeea3d9f36c851084b6932196df4edbcd6b55a68a17c4218a5bd44811fc3e72 Jul 08 19:11:02.738219 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:02.738139 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xwj24"] Jul 08 19:11:02.742553 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:02.742537 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-xwj24" Jul 08 19:11:02.772664 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:02.772631 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-4lw7p" event={"ID":"e1f3c964-0b63-4bab-8fdf-068e8807a797","Type":"ContainerStarted","Data":"caeea3d9f36c851084b6932196df4edbcd6b55a68a17c4218a5bd44811fc3e72"} Jul 08 19:11:02.777381 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:02.777360 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xwj24"] Jul 08 19:11:02.867976 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:02.867938 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xrbp\" (UniqueName: \"kubernetes.io/projected/761c0231-6742-402c-9c83-a0d45f2d8fff-kube-api-access-4xrbp\") pod \"kuadrant-operator-catalog-xwj24\" (UID: \"761c0231-6742-402c-9c83-a0d45f2d8fff\") " pod="kuadrant-system/kuadrant-operator-catalog-xwj24" Jul 08 19:11:02.968960 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:02.968926 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xrbp\" (UniqueName: \"kubernetes.io/projected/761c0231-6742-402c-9c83-a0d45f2d8fff-kube-api-access-4xrbp\") pod \"kuadrant-operator-catalog-xwj24\" (UID: \"761c0231-6742-402c-9c83-a0d45f2d8fff\") " pod="kuadrant-system/kuadrant-operator-catalog-xwj24" Jul 08 19:11:02.978048 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:02.978023 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xrbp\" (UniqueName: \"kubernetes.io/projected/761c0231-6742-402c-9c83-a0d45f2d8fff-kube-api-access-4xrbp\") pod \"kuadrant-operator-catalog-xwj24\" (UID: \"761c0231-6742-402c-9c83-a0d45f2d8fff\") " pod="kuadrant-system/kuadrant-operator-catalog-xwj24" Jul 08 19:11:03.051623 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:03.051540 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-xwj24" Jul 08 19:11:03.191536 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:03.191453 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-xwj24"] Jul 08 19:11:03.232726 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:11:03.232697 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod761c0231_6742_402c_9c83_a0d45f2d8fff.slice/crio-0cbf06767ebec9474a9cf5259452ec0aaa6965648531f9284213d1a653e9a54c WatchSource:0}: Error finding container 0cbf06767ebec9474a9cf5259452ec0aaa6965648531f9284213d1a653e9a54c: Status 404 returned error can't find the container with id 0cbf06767ebec9474a9cf5259452ec0aaa6965648531f9284213d1a653e9a54c Jul 08 19:11:03.779644 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:03.779604 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-xwj24" event={"ID":"761c0231-6742-402c-9c83-a0d45f2d8fff","Type":"ContainerStarted","Data":"0cbf06767ebec9474a9cf5259452ec0aaa6965648531f9284213d1a653e9a54c"} Jul 08 19:11:05.789820 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:05.789773 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-xwj24" event={"ID":"761c0231-6742-402c-9c83-a0d45f2d8fff","Type":"ContainerStarted","Data":"3b83af55caf41599e497b98a1f2855b771bda01966cd6175212caa5f7e2b2000"} Jul 08 19:11:05.791185 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:05.791160 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-4lw7p" event={"ID":"e1f3c964-0b63-4bab-8fdf-068e8807a797","Type":"ContainerStarted","Data":"22a9494eda28957262598b68d68d05a21e519fff04985eb5708284849fb395c0"} Jul 08 19:11:05.791305 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:05.791265 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-4lw7p" podUID="e1f3c964-0b63-4bab-8fdf-068e8807a797" containerName="registry-server" containerID="cri-o://22a9494eda28957262598b68d68d05a21e519fff04985eb5708284849fb395c0" gracePeriod=2 Jul 08 19:11:05.807974 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:05.807926 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-xwj24" podStartSLOduration=2.009000673 podStartE2EDuration="3.807911834s" totalImagesPullingTime="1.798911161s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:11:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:11:05.805812855 +0000 UTC m=+431.084900476" watchObservedRunningTime="2026-07-08 19:11:05.807911834 +0000 UTC m=+431.086999446" Jul 08 19:11:05.822240 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:05.822198 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-4lw7p" podStartSLOduration=1.4270842209999999 podStartE2EDuration="3.822184163s" totalImagesPullingTime="2.395099942s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:11:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:11:05.821018939 +0000 UTC m=+431.100106559" watchObservedRunningTime="2026-07-08 19:11:05.822184163 +0000 UTC m=+431.101271783" Jul 08 19:11:06.039022 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:06.038995 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-4lw7p" Jul 08 19:11:06.195686 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:06.195658 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5j8fg\" (UniqueName: \"kubernetes.io/projected/e1f3c964-0b63-4bab-8fdf-068e8807a797-kube-api-access-5j8fg\") pod \"e1f3c964-0b63-4bab-8fdf-068e8807a797\" (UID: \"e1f3c964-0b63-4bab-8fdf-068e8807a797\") " Jul 08 19:11:06.197954 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:06.197931 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1f3c964-0b63-4bab-8fdf-068e8807a797-kube-api-access-5j8fg" (OuterVolumeSpecName: "kube-api-access-5j8fg") pod "e1f3c964-0b63-4bab-8fdf-068e8807a797" (UID: "e1f3c964-0b63-4bab-8fdf-068e8807a797"). InnerVolumeSpecName "kube-api-access-5j8fg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:11:06.296734 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:06.296699 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5j8fg\" (UniqueName: \"kubernetes.io/projected/e1f3c964-0b63-4bab-8fdf-068e8807a797-kube-api-access-5j8fg\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:11:06.796545 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:06.796503 2583 generic.go:356] "Generic (PLEG): container finished" podID="e1f3c964-0b63-4bab-8fdf-068e8807a797" containerID="22a9494eda28957262598b68d68d05a21e519fff04985eb5708284849fb395c0" exitCode=0 Jul 08 19:11:06.797011 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:06.796553 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-4lw7p" Jul 08 19:11:06.797011 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:06.796600 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-4lw7p" event={"ID":"e1f3c964-0b63-4bab-8fdf-068e8807a797","Type":"ContainerDied","Data":"22a9494eda28957262598b68d68d05a21e519fff04985eb5708284849fb395c0"} Jul 08 19:11:06.797011 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:06.796660 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-4lw7p" event={"ID":"e1f3c964-0b63-4bab-8fdf-068e8807a797","Type":"ContainerDied","Data":"caeea3d9f36c851084b6932196df4edbcd6b55a68a17c4218a5bd44811fc3e72"} Jul 08 19:11:06.797011 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:06.796696 2583 scope.go:117] "RemoveContainer" containerID="22a9494eda28957262598b68d68d05a21e519fff04985eb5708284849fb395c0" Jul 08 19:11:06.805806 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:06.805785 2583 scope.go:117] "RemoveContainer" containerID="22a9494eda28957262598b68d68d05a21e519fff04985eb5708284849fb395c0" Jul 08 19:11:06.806065 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:11:06.806049 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22a9494eda28957262598b68d68d05a21e519fff04985eb5708284849fb395c0\": container with ID starting with 22a9494eda28957262598b68d68d05a21e519fff04985eb5708284849fb395c0 not found: ID does not exist" containerID="22a9494eda28957262598b68d68d05a21e519fff04985eb5708284849fb395c0" Jul 08 19:11:06.806105 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:06.806072 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22a9494eda28957262598b68d68d05a21e519fff04985eb5708284849fb395c0"} err="failed to get container status \"22a9494eda28957262598b68d68d05a21e519fff04985eb5708284849fb395c0\": rpc error: code = NotFound desc = could not find container \"22a9494eda28957262598b68d68d05a21e519fff04985eb5708284849fb395c0\": container with ID starting with 22a9494eda28957262598b68d68d05a21e519fff04985eb5708284849fb395c0 not found: ID does not exist" Jul 08 19:11:06.818296 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:06.818272 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4lw7p"] Jul 08 19:11:06.822486 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:06.822466 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4lw7p"] Jul 08 19:11:07.174149 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:07.174116 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1f3c964-0b63-4bab-8fdf-068e8807a797" path="/var/lib/kubelet/pods/e1f3c964-0b63-4bab-8fdf-068e8807a797/volumes" Jul 08 19:11:13.051760 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:13.051726 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-xwj24" Jul 08 19:11:13.052177 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:13.051958 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-xwj24" Jul 08 19:11:13.075008 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:13.074980 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-xwj24" Jul 08 19:11:13.851860 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:13.851833 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-xwj24" Jul 08 19:11:35.013248 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:35.013175 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-lk5nn"] Jul 08 19:11:35.013712 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:35.013482 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1f3c964-0b63-4bab-8fdf-068e8807a797" containerName="registry-server" Jul 08 19:11:35.013712 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:35.013493 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1f3c964-0b63-4bab-8fdf-068e8807a797" containerName="registry-server" Jul 08 19:11:35.013712 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:35.013552 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1f3c964-0b63-4bab-8fdf-068e8807a797" containerName="registry-server" Jul 08 19:11:35.016392 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:35.016376 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lk5nn" Jul 08 19:11:35.018987 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:35.018968 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-n5w5k\"" Jul 08 19:11:35.059224 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:35.059198 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-lk5nn"] Jul 08 19:11:35.123492 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:35.123461 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m86qk\" (UniqueName: \"kubernetes.io/projected/e1bee84b-23f5-434f-b00b-82d65e2035e5-kube-api-access-m86qk\") pod \"authorino-operator-75cc8d5cb7-lk5nn\" (UID: \"e1bee84b-23f5-434f-b00b-82d65e2035e5\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-lk5nn" Jul 08 19:11:35.224234 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:35.224192 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m86qk\" (UniqueName: \"kubernetes.io/projected/e1bee84b-23f5-434f-b00b-82d65e2035e5-kube-api-access-m86qk\") pod \"authorino-operator-75cc8d5cb7-lk5nn\" (UID: \"e1bee84b-23f5-434f-b00b-82d65e2035e5\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-lk5nn" Jul 08 19:11:35.242254 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:35.242215 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m86qk\" (UniqueName: \"kubernetes.io/projected/e1bee84b-23f5-434f-b00b-82d65e2035e5-kube-api-access-m86qk\") pod \"authorino-operator-75cc8d5cb7-lk5nn\" (UID: \"e1bee84b-23f5-434f-b00b-82d65e2035e5\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-lk5nn" Jul 08 19:11:35.325982 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:35.325897 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lk5nn" Jul 08 19:11:35.465849 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:35.465825 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-lk5nn"] Jul 08 19:11:35.467642 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:11:35.467614 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1bee84b_23f5_434f_b00b_82d65e2035e5.slice/crio-d3871ef971d1d979cdb5c86f894d752bd2a3bb3c548df9b98fed0b633fa23b7f WatchSource:0}: Error finding container d3871ef971d1d979cdb5c86f894d752bd2a3bb3c548df9b98fed0b633fa23b7f: Status 404 returned error can't find the container with id d3871ef971d1d979cdb5c86f894d752bd2a3bb3c548df9b98fed0b633fa23b7f Jul 08 19:11:35.918414 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:35.918380 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lk5nn" event={"ID":"e1bee84b-23f5-434f-b00b-82d65e2035e5","Type":"ContainerStarted","Data":"d3871ef971d1d979cdb5c86f894d752bd2a3bb3c548df9b98fed0b633fa23b7f"} Jul 08 19:11:37.859154 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:37.859122 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n7bhk"] Jul 08 19:11:37.862347 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:37.862330 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n7bhk" Jul 08 19:11:37.865144 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:37.865118 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-slkcc\"" Jul 08 19:11:37.874163 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:37.874142 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n7bhk"] Jul 08 19:11:37.928032 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:37.927996 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lk5nn" event={"ID":"e1bee84b-23f5-434f-b00b-82d65e2035e5","Type":"ContainerStarted","Data":"61df54edd60750edd2c4cfa001aa0f300e5c23d6f5293a5398fbf5c9e4703e9d"} Jul 08 19:11:37.928206 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:37.928095 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lk5nn" Jul 08 19:11:37.945167 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:37.945138 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vl6d\" (UniqueName: \"kubernetes.io/projected/6f02ef90-7094-4a80-8c02-8fd0ba9288da-kube-api-access-4vl6d\") pod \"limitador-operator-controller-manager-859d7b78bd-n7bhk\" (UID: \"6f02ef90-7094-4a80-8c02-8fd0ba9288da\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n7bhk" Jul 08 19:11:37.957031 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:37.956951 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lk5nn" podStartSLOduration=1.727185295 podStartE2EDuration="3.956934309s" totalImagesPullingTime="2.229749014s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:11:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:11:37.956548905 +0000 UTC m=+463.235636525" watchObservedRunningTime="2026-07-08 19:11:37.956934309 +0000 UTC m=+463.236021928" Jul 08 19:11:38.046571 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:38.046538 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vl6d\" (UniqueName: \"kubernetes.io/projected/6f02ef90-7094-4a80-8c02-8fd0ba9288da-kube-api-access-4vl6d\") pod \"limitador-operator-controller-manager-859d7b78bd-n7bhk\" (UID: \"6f02ef90-7094-4a80-8c02-8fd0ba9288da\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n7bhk" Jul 08 19:11:38.055497 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:38.055468 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vl6d\" (UniqueName: \"kubernetes.io/projected/6f02ef90-7094-4a80-8c02-8fd0ba9288da-kube-api-access-4vl6d\") pod \"limitador-operator-controller-manager-859d7b78bd-n7bhk\" (UID: \"6f02ef90-7094-4a80-8c02-8fd0ba9288da\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n7bhk" Jul 08 19:11:38.172290 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:38.172241 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n7bhk" Jul 08 19:11:38.321258 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:38.321179 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n7bhk"] Jul 08 19:11:38.323482 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:11:38.323452 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f02ef90_7094_4a80_8c02_8fd0ba9288da.slice/crio-bd79f7badc1be6b7a9a30e8766dab787828e06967d5f42d2ad367c314050b6f1 WatchSource:0}: Error finding container bd79f7badc1be6b7a9a30e8766dab787828e06967d5f42d2ad367c314050b6f1: Status 404 returned error can't find the container with id bd79f7badc1be6b7a9a30e8766dab787828e06967d5f42d2ad367c314050b6f1 Jul 08 19:11:38.934858 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:38.934820 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n7bhk" event={"ID":"6f02ef90-7094-4a80-8c02-8fd0ba9288da","Type":"ContainerStarted","Data":"bd79f7badc1be6b7a9a30e8766dab787828e06967d5f42d2ad367c314050b6f1"} Jul 08 19:11:39.941511 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:39.941476 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n7bhk" event={"ID":"6f02ef90-7094-4a80-8c02-8fd0ba9288da","Type":"ContainerStarted","Data":"942b12f4046c3aa163040f2d5a676f8b18882a63da9b550bae08ef9780bec55f"} Jul 08 19:11:39.941848 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:39.941609 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n7bhk" Jul 08 19:11:39.968893 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:39.968840 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n7bhk" podStartSLOduration=1.466543903 podStartE2EDuration="2.96882311s" totalImagesPullingTime="1.502279207s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:11:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:11:39.966886092 +0000 UTC m=+465.245973712" watchObservedRunningTime="2026-07-08 19:11:39.96882311 +0000 UTC m=+465.247910729" Jul 08 19:11:41.857974 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:41.857938 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hjvj9"] Jul 08 19:11:41.861293 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:41.861278 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hjvj9" Jul 08 19:11:41.865667 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:41.865646 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-gc28r\"" Jul 08 19:11:41.901181 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:41.901152 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hjvj9"] Jul 08 19:11:41.979115 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:41.979077 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cf1e7797-55a1-4411-b7e0-d38a734b78d3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hjvj9\" (UID: \"cf1e7797-55a1-4411-b7e0-d38a734b78d3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hjvj9" Jul 08 19:11:41.979312 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:41.979129 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hq87r\" (UniqueName: \"kubernetes.io/projected/cf1e7797-55a1-4411-b7e0-d38a734b78d3-kube-api-access-hq87r\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hjvj9\" (UID: \"cf1e7797-55a1-4411-b7e0-d38a734b78d3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hjvj9" Jul 08 19:11:42.080622 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:42.080589 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cf1e7797-55a1-4411-b7e0-d38a734b78d3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hjvj9\" (UID: \"cf1e7797-55a1-4411-b7e0-d38a734b78d3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hjvj9" Jul 08 19:11:42.080777 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:42.080639 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hq87r\" (UniqueName: \"kubernetes.io/projected/cf1e7797-55a1-4411-b7e0-d38a734b78d3-kube-api-access-hq87r\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hjvj9\" (UID: \"cf1e7797-55a1-4411-b7e0-d38a734b78d3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hjvj9" Jul 08 19:11:42.081034 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:42.081014 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cf1e7797-55a1-4411-b7e0-d38a734b78d3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hjvj9\" (UID: \"cf1e7797-55a1-4411-b7e0-d38a734b78d3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hjvj9" Jul 08 19:11:42.111785 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:42.111716 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hq87r\" (UniqueName: \"kubernetes.io/projected/cf1e7797-55a1-4411-b7e0-d38a734b78d3-kube-api-access-hq87r\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hjvj9\" (UID: \"cf1e7797-55a1-4411-b7e0-d38a734b78d3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hjvj9" Jul 08 19:11:42.170819 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:42.170783 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hjvj9" Jul 08 19:11:42.323007 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:42.322985 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hjvj9"] Jul 08 19:11:42.324644 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:11:42.324618 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf1e7797_55a1_4411_b7e0_d38a734b78d3.slice/crio-1a8876934f5d8de65559c17aab91a64a5e662431743d3a02b7e8311e4bfb7e74 WatchSource:0}: Error finding container 1a8876934f5d8de65559c17aab91a64a5e662431743d3a02b7e8311e4bfb7e74: Status 404 returned error can't find the container with id 1a8876934f5d8de65559c17aab91a64a5e662431743d3a02b7e8311e4bfb7e74 Jul 08 19:11:42.955233 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:42.955203 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hjvj9" event={"ID":"cf1e7797-55a1-4411-b7e0-d38a734b78d3","Type":"ContainerStarted","Data":"1a8876934f5d8de65559c17aab91a64a5e662431743d3a02b7e8311e4bfb7e74"} Jul 08 19:11:47.978399 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:47.978359 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hjvj9" event={"ID":"cf1e7797-55a1-4411-b7e0-d38a734b78d3","Type":"ContainerStarted","Data":"3fb6d7c958dc640198c1fe3036c9f4535594bccbd0f9f4cba49c6ebe8bada766"} Jul 08 19:11:47.978858 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:47.978596 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hjvj9" Jul 08 19:11:48.000913 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:48.000856 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hjvj9" podStartSLOduration=1.876378635 podStartE2EDuration="7.000839248s" totalImagesPullingTime="5.124460613s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:11:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:11:47.998854819 +0000 UTC m=+473.277942439" watchObservedRunningTime="2026-07-08 19:11:48.000839248 +0000 UTC m=+473.279926867" Jul 08 19:11:48.937274 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:48.937238 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-lk5nn" Jul 08 19:11:50.948711 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:50.948678 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n7bhk" Jul 08 19:11:58.988753 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:11:58.988721 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hjvj9" Jul 08 19:12:01.237556 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:01.237525 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hjvj9"] Jul 08 19:12:01.237953 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:01.237699 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hjvj9" podUID="cf1e7797-55a1-4411-b7e0-d38a734b78d3" containerName="manager" containerID="cri-o://3fb6d7c958dc640198c1fe3036c9f4535594bccbd0f9f4cba49c6ebe8bada766" gracePeriod=2 Jul 08 19:12:01.272102 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:01.272068 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hjvj9"] Jul 08 19:12:01.296167 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:12:01.296125 2583 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"cf1e7797-55a1-4411-b7e0-d38a734b78d3\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-hjvj9\": pods \"kuadrant-operator-controller-manager-8684c9ff49-hjvj9\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hjvj9" Jul 08 19:12:01.310523 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:01.310495 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rk4xw"] Jul 08 19:12:01.310882 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:01.310860 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf1e7797-55a1-4411-b7e0-d38a734b78d3" containerName="manager" Jul 08 19:12:01.310882 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:01.310881 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf1e7797-55a1-4411-b7e0-d38a734b78d3" containerName="manager" Jul 08 19:12:01.311064 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:01.310954 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf1e7797-55a1-4411-b7e0-d38a734b78d3" containerName="manager" Jul 08 19:12:01.314100 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:01.314078 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rk4xw" Jul 08 19:12:01.317587 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:12:01.317553 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-hjvj9\" is forbidden: User \"system:node:ip-10-0-137-244.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-244.ec2.internal' and this object" podUID="cf1e7797-55a1-4411-b7e0-d38a734b78d3" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hjvj9" Jul 08 19:12:01.375281 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:01.375248 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rk4xw"] Jul 08 19:12:01.380185 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:01.380160 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n7bhk"] Jul 08 19:12:01.380362 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:01.380342 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n7bhk" podUID="6f02ef90-7094-4a80-8c02-8fd0ba9288da" containerName="manager" containerID="cri-o://942b12f4046c3aa163040f2d5a676f8b18882a63da9b550bae08ef9780bec55f" gracePeriod=2 Jul 08 19:12:01.406979 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:01.406947 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n7bhk"] Jul 08 19:12:01.426935 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:12:01.426905 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-hjvj9\" is forbidden: User \"system:node:ip-10-0-137-244.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-244.ec2.internal' and this object" podUID="cf1e7797-55a1-4411-b7e0-d38a734b78d3" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hjvj9" Jul 08 19:12:01.429081 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:12:01.429051 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-n7bhk\" is forbidden: User \"system:node:ip-10-0-137-244.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-244.ec2.internal' and this object" podUID="6f02ef90-7094-4a80-8c02-8fd0ba9288da" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n7bhk" Jul 08 19:12:01.445539 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:01.445507 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a3dd03ee-6f21-44eb-97b1-2925af3e288a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rk4xw\" (UID: \"a3dd03ee-6f21-44eb-97b1-2925af3e288a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rk4xw" Jul 08 19:12:01.445666 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:01.445626 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbzwk\" (UniqueName: \"kubernetes.io/projected/a3dd03ee-6f21-44eb-97b1-2925af3e288a-kube-api-access-fbzwk\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rk4xw\" (UID: \"a3dd03ee-6f21-44eb-97b1-2925af3e288a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rk4xw" Jul 08 19:12:01.502181 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:01.502157 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hjvj9" Jul 08 19:12:01.505754 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:01.505730 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tn9qn"] Jul 08 19:12:01.506073 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:01.506059 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f02ef90-7094-4a80-8c02-8fd0ba9288da" containerName="manager" Jul 08 19:12:01.506116 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:01.506075 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f02ef90-7094-4a80-8c02-8fd0ba9288da" containerName="manager" Jul 08 19:12:01.506152 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:01.506140 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f02ef90-7094-4a80-8c02-8fd0ba9288da" containerName="manager" Jul 08 19:12:01.508937 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:01.508918 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tn9qn" Jul 08 19:12:01.517864 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:12:01.517829 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-hjvj9\" is forbidden: User \"system:node:ip-10-0-137-244.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-244.ec2.internal' and this object" podUID="cf1e7797-55a1-4411-b7e0-d38a734b78d3" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hjvj9" Jul 08 19:12:01.521171 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:12:01.521144 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-n7bhk\" is forbidden: User \"system:node:ip-10-0-137-244.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-244.ec2.internal' and this object" podUID="6f02ef90-7094-4a80-8c02-8fd0ba9288da" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n7bhk" Jul 08 19:12:01.530280 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:12:01.530236 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-n7bhk\" is forbidden: User \"system:node:ip-10-0-137-244.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-244.ec2.internal' and this object" podUID="6f02ef90-7094-4a80-8c02-8fd0ba9288da" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n7bhk" Jul 08 19:12:01.544602 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:01.544578 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tn9qn"] Jul 08 19:12:01.546230 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:01.546201 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a3dd03ee-6f21-44eb-97b1-2925af3e288a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rk4xw\" (UID: \"a3dd03ee-6f21-44eb-97b1-2925af3e288a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rk4xw" Jul 08 19:12:01.546333 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:01.546280 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbzwk\" (UniqueName: \"kubernetes.io/projected/a3dd03ee-6f21-44eb-97b1-2925af3e288a-kube-api-access-fbzwk\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rk4xw\" (UID: \"a3dd03ee-6f21-44eb-97b1-2925af3e288a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rk4xw" Jul 08 19:12:01.546689 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:01.546665 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a3dd03ee-6f21-44eb-97b1-2925af3e288a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rk4xw\" (UID: \"a3dd03ee-6f21-44eb-97b1-2925af3e288a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rk4xw" Jul 08 19:12:01.605731 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:01.605697 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbzwk\" (UniqueName: \"kubernetes.io/projected/a3dd03ee-6f21-44eb-97b1-2925af3e288a-kube-api-access-fbzwk\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rk4xw\" (UID: \"a3dd03ee-6f21-44eb-97b1-2925af3e288a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rk4xw" Jul 08 19:12:01.609094 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:12:01.609062 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-hjvj9\" is forbidden: User \"system:node:ip-10-0-137-244.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-244.ec2.internal' and this object" podUID="cf1e7797-55a1-4411-b7e0-d38a734b78d3" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hjvj9" Jul 08 19:12:01.626261 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:01.626240 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rk4xw" Jul 08 19:12:01.646855 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:01.646832 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hq87r\" (UniqueName: \"kubernetes.io/projected/cf1e7797-55a1-4411-b7e0-d38a734b78d3-kube-api-access-hq87r\") pod \"cf1e7797-55a1-4411-b7e0-d38a734b78d3\" (UID: \"cf1e7797-55a1-4411-b7e0-d38a734b78d3\") " Jul 08 19:12:01.646979 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:01.646871 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cf1e7797-55a1-4411-b7e0-d38a734b78d3-extensions-socket-volume\") pod \"cf1e7797-55a1-4411-b7e0-d38a734b78d3\" (UID: \"cf1e7797-55a1-4411-b7e0-d38a734b78d3\") " Jul 08 19:12:01.647071 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:01.647052 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4x7db\" (UniqueName: \"kubernetes.io/projected/80f18c7b-fa4a-4389-a961-f92e015fd1f6-kube-api-access-4x7db\") pod \"limitador-operator-controller-manager-859d7b78bd-tn9qn\" (UID: \"80f18c7b-fa4a-4389-a961-f92e015fd1f6\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tn9qn" Jul 08 19:12:01.647351 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:01.647313 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf1e7797-55a1-4411-b7e0-d38a734b78d3-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "cf1e7797-55a1-4411-b7e0-d38a734b78d3" (UID: "cf1e7797-55a1-4411-b7e0-d38a734b78d3"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:12:01.647898 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:01.647881 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n7bhk" Jul 08 19:12:01.649154 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:01.649124 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf1e7797-55a1-4411-b7e0-d38a734b78d3-kube-api-access-hq87r" (OuterVolumeSpecName: "kube-api-access-hq87r") pod "cf1e7797-55a1-4411-b7e0-d38a734b78d3" (UID: "cf1e7797-55a1-4411-b7e0-d38a734b78d3"). InnerVolumeSpecName "kube-api-access-hq87r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:12:01.654655 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:12:01.654628 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-hjvj9\" is forbidden: User \"system:node:ip-10-0-137-244.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-244.ec2.internal' and this object" podUID="cf1e7797-55a1-4411-b7e0-d38a734b78d3" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hjvj9" Jul 08 19:12:01.659648 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:12:01.659620 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-n7bhk\" is forbidden: User \"system:node:ip-10-0-137-244.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-244.ec2.internal' and this object" podUID="6f02ef90-7094-4a80-8c02-8fd0ba9288da" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n7bhk" Jul 08 19:12:01.747707 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:01.747629 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4vl6d\" (UniqueName: \"kubernetes.io/projected/6f02ef90-7094-4a80-8c02-8fd0ba9288da-kube-api-access-4vl6d\") pod \"6f02ef90-7094-4a80-8c02-8fd0ba9288da\" (UID: \"6f02ef90-7094-4a80-8c02-8fd0ba9288da\") " Jul 08 19:12:01.747879 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:01.747862 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4x7db\" (UniqueName: \"kubernetes.io/projected/80f18c7b-fa4a-4389-a961-f92e015fd1f6-kube-api-access-4x7db\") pod \"limitador-operator-controller-manager-859d7b78bd-tn9qn\" (UID: \"80f18c7b-fa4a-4389-a961-f92e015fd1f6\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tn9qn" Jul 08 19:12:01.747961 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:01.747941 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hq87r\" (UniqueName: \"kubernetes.io/projected/cf1e7797-55a1-4411-b7e0-d38a734b78d3-kube-api-access-hq87r\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:12:01.748007 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:01.747958 2583 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cf1e7797-55a1-4411-b7e0-d38a734b78d3-extensions-socket-volume\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:12:01.749688 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:01.749667 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f02ef90-7094-4a80-8c02-8fd0ba9288da-kube-api-access-4vl6d" (OuterVolumeSpecName: "kube-api-access-4vl6d") pod "6f02ef90-7094-4a80-8c02-8fd0ba9288da" (UID: "6f02ef90-7094-4a80-8c02-8fd0ba9288da"). InnerVolumeSpecName "kube-api-access-4vl6d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:12:01.762142 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:01.762108 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4x7db\" (UniqueName: \"kubernetes.io/projected/80f18c7b-fa4a-4389-a961-f92e015fd1f6-kube-api-access-4x7db\") pod \"limitador-operator-controller-manager-859d7b78bd-tn9qn\" (UID: \"80f18c7b-fa4a-4389-a961-f92e015fd1f6\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tn9qn" Jul 08 19:12:01.821413 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:01.821325 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rk4xw"] Jul 08 19:12:01.821413 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:01.821381 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tn9qn" Jul 08 19:12:01.824296 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:12:01.824265 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3dd03ee_6f21_44eb_97b1_2925af3e288a.slice/crio-62ff0717874023e5f41fe99432151cae9d665f5f6f6c344b13568e5c9eb791e3 WatchSource:0}: Error finding container 62ff0717874023e5f41fe99432151cae9d665f5f6f6c344b13568e5c9eb791e3: Status 404 returned error can't find the container with id 62ff0717874023e5f41fe99432151cae9d665f5f6f6c344b13568e5c9eb791e3 Jul 08 19:12:01.849003 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:01.848983 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4vl6d\" (UniqueName: \"kubernetes.io/projected/6f02ef90-7094-4a80-8c02-8fd0ba9288da-kube-api-access-4vl6d\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:12:02.011963 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:02.011938 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tn9qn"] Jul 08 19:12:02.014367 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:12:02.014347 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80f18c7b_fa4a_4389_a961_f92e015fd1f6.slice/crio-dc57c3cbf9f398ce7dd259056a51e50342bf6c32488fb7e036a345926f26e725 WatchSource:0}: Error finding container dc57c3cbf9f398ce7dd259056a51e50342bf6c32488fb7e036a345926f26e725: Status 404 returned error can't find the container with id dc57c3cbf9f398ce7dd259056a51e50342bf6c32488fb7e036a345926f26e725 Jul 08 19:12:02.043578 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:02.043553 2583 generic.go:356] "Generic (PLEG): container finished" podID="cf1e7797-55a1-4411-b7e0-d38a734b78d3" containerID="3fb6d7c958dc640198c1fe3036c9f4535594bccbd0f9f4cba49c6ebe8bada766" exitCode=0 Jul 08 19:12:02.043699 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:02.043605 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hjvj9" Jul 08 19:12:02.043699 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:02.043640 2583 scope.go:117] "RemoveContainer" containerID="3fb6d7c958dc640198c1fe3036c9f4535594bccbd0f9f4cba49c6ebe8bada766" Jul 08 19:12:02.044946 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:02.044924 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tn9qn" event={"ID":"80f18c7b-fa4a-4389-a961-f92e015fd1f6","Type":"ContainerStarted","Data":"dc57c3cbf9f398ce7dd259056a51e50342bf6c32488fb7e036a345926f26e725"} Jul 08 19:12:02.046525 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:12:02.046497 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-hjvj9\" is forbidden: User \"system:node:ip-10-0-137-244.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-244.ec2.internal' and this object" podUID="cf1e7797-55a1-4411-b7e0-d38a734b78d3" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hjvj9" Jul 08 19:12:02.046853 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:02.046820 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rk4xw" event={"ID":"a3dd03ee-6f21-44eb-97b1-2925af3e288a","Type":"ContainerStarted","Data":"09fcae12952384eef228b3f87c84c95421fa585871b13c6a18feabe7facb7989"} Jul 08 19:12:02.046960 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:02.046876 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rk4xw" event={"ID":"a3dd03ee-6f21-44eb-97b1-2925af3e288a","Type":"ContainerStarted","Data":"62ff0717874023e5f41fe99432151cae9d665f5f6f6c344b13568e5c9eb791e3"} Jul 08 19:12:02.046960 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:02.046912 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rk4xw" Jul 08 19:12:02.048820 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:02.048797 2583 generic.go:356] "Generic (PLEG): container finished" podID="6f02ef90-7094-4a80-8c02-8fd0ba9288da" containerID="942b12f4046c3aa163040f2d5a676f8b18882a63da9b550bae08ef9780bec55f" exitCode=0 Jul 08 19:12:02.048923 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:02.048839 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n7bhk" Jul 08 19:12:02.049031 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:12:02.049009 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-n7bhk\" is forbidden: User \"system:node:ip-10-0-137-244.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-244.ec2.internal' and this object" podUID="6f02ef90-7094-4a80-8c02-8fd0ba9288da" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n7bhk" Jul 08 19:12:02.051074 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:12:02.051047 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-hjvj9\" is forbidden: User \"system:node:ip-10-0-137-244.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-244.ec2.internal' and this object" podUID="cf1e7797-55a1-4411-b7e0-d38a734b78d3" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hjvj9" Jul 08 19:12:02.058050 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:02.058007 2583 scope.go:117] "RemoveContainer" containerID="3fb6d7c958dc640198c1fe3036c9f4535594bccbd0f9f4cba49c6ebe8bada766" Jul 08 19:12:02.058391 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:12:02.058365 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3fb6d7c958dc640198c1fe3036c9f4535594bccbd0f9f4cba49c6ebe8bada766\": container with ID starting with 3fb6d7c958dc640198c1fe3036c9f4535594bccbd0f9f4cba49c6ebe8bada766 not found: ID does not exist" containerID="3fb6d7c958dc640198c1fe3036c9f4535594bccbd0f9f4cba49c6ebe8bada766" Jul 08 19:12:02.058496 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:02.058402 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fb6d7c958dc640198c1fe3036c9f4535594bccbd0f9f4cba49c6ebe8bada766"} err="failed to get container status \"3fb6d7c958dc640198c1fe3036c9f4535594bccbd0f9f4cba49c6ebe8bada766\": rpc error: code = NotFound desc = could not find container \"3fb6d7c958dc640198c1fe3036c9f4535594bccbd0f9f4cba49c6ebe8bada766\": container with ID starting with 3fb6d7c958dc640198c1fe3036c9f4535594bccbd0f9f4cba49c6ebe8bada766 not found: ID does not exist" Jul 08 19:12:02.058496 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:02.058454 2583 scope.go:117] "RemoveContainer" containerID="942b12f4046c3aa163040f2d5a676f8b18882a63da9b550bae08ef9780bec55f" Jul 08 19:12:02.077225 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:02.077209 2583 scope.go:117] "RemoveContainer" containerID="942b12f4046c3aa163040f2d5a676f8b18882a63da9b550bae08ef9780bec55f" Jul 08 19:12:02.077521 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:12:02.077498 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"942b12f4046c3aa163040f2d5a676f8b18882a63da9b550bae08ef9780bec55f\": container with ID starting with 942b12f4046c3aa163040f2d5a676f8b18882a63da9b550bae08ef9780bec55f not found: ID does not exist" containerID="942b12f4046c3aa163040f2d5a676f8b18882a63da9b550bae08ef9780bec55f" Jul 08 19:12:02.077599 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:02.077531 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"942b12f4046c3aa163040f2d5a676f8b18882a63da9b550bae08ef9780bec55f"} err="failed to get container status \"942b12f4046c3aa163040f2d5a676f8b18882a63da9b550bae08ef9780bec55f\": rpc error: code = NotFound desc = could not find container \"942b12f4046c3aa163040f2d5a676f8b18882a63da9b550bae08ef9780bec55f\": container with ID starting with 942b12f4046c3aa163040f2d5a676f8b18882a63da9b550bae08ef9780bec55f not found: ID does not exist" Jul 08 19:12:02.128950 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:02.123954 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rk4xw" podStartSLOduration=1.123924515 podStartE2EDuration="1.123924515s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:12:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:12:02.123014951 +0000 UTC m=+487.402102572" watchObservedRunningTime="2026-07-08 19:12:02.123924515 +0000 UTC m=+487.403012136" Jul 08 19:12:02.128950 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:12:02.126593 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-n7bhk\" is forbidden: User \"system:node:ip-10-0-137-244.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-244.ec2.internal' and this object" podUID="6f02ef90-7094-4a80-8c02-8fd0ba9288da" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n7bhk" Jul 08 19:12:02.136838 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:12:02.136805 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-hjvj9\" is forbidden: User \"system:node:ip-10-0-137-244.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-244.ec2.internal' and this object" podUID="cf1e7797-55a1-4411-b7e0-d38a734b78d3" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hjvj9" Jul 08 19:12:02.145911 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:12:02.145872 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-n7bhk\" is forbidden: User \"system:node:ip-10-0-137-244.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-244.ec2.internal' and this object" podUID="6f02ef90-7094-4a80-8c02-8fd0ba9288da" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n7bhk" Jul 08 19:12:03.056124 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:03.056085 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tn9qn" event={"ID":"80f18c7b-fa4a-4389-a961-f92e015fd1f6","Type":"ContainerStarted","Data":"57118278515e19cec9b29002188cc2970414fbec137e04402436f419a65fb1e8"} Jul 08 19:12:03.094552 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:03.094501 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tn9qn" podStartSLOduration=2.094486148 podStartE2EDuration="2.094486148s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:12:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:12:03.093704603 +0000 UTC m=+488.372792223" watchObservedRunningTime="2026-07-08 19:12:03.094486148 +0000 UTC m=+488.373573769" Jul 08 19:12:03.096266 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:12:03.096237 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-hjvj9\" is forbidden: User \"system:node:ip-10-0-137-244.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-244.ec2.internal' and this object" podUID="cf1e7797-55a1-4411-b7e0-d38a734b78d3" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hjvj9" Jul 08 19:12:03.098579 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:12:03.098550 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-n7bhk\" is forbidden: User \"system:node:ip-10-0-137-244.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-244.ec2.internal' and this object" podUID="6f02ef90-7094-4a80-8c02-8fd0ba9288da" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-n7bhk" Jul 08 19:12:03.173738 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:03.173709 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f02ef90-7094-4a80-8c02-8fd0ba9288da" path="/var/lib/kubelet/pods/6f02ef90-7094-4a80-8c02-8fd0ba9288da/volumes" Jul 08 19:12:03.174040 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:03.174027 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf1e7797-55a1-4411-b7e0-d38a734b78d3" path="/var/lib/kubelet/pods/cf1e7797-55a1-4411-b7e0-d38a734b78d3/volumes" Jul 08 19:12:04.060170 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:04.060136 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tn9qn" Jul 08 19:12:06.352617 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:06.352584 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4ntqt"] Jul 08 19:12:06.356774 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:06.356758 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4ntqt" Jul 08 19:12:06.383628 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:06.383600 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4ntqt"] Jul 08 19:12:06.489609 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:06.489574 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/34dd9e84-4ff2-4c4d-b485-1c4a8e81dd4d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-4ntqt\" (UID: \"34dd9e84-4ff2-4c4d-b485-1c4a8e81dd4d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4ntqt" Jul 08 19:12:06.489609 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:06.489615 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kb2xj\" (UniqueName: \"kubernetes.io/projected/34dd9e84-4ff2-4c4d-b485-1c4a8e81dd4d-kube-api-access-kb2xj\") pod \"kuadrant-operator-controller-manager-86878c769f-4ntqt\" (UID: \"34dd9e84-4ff2-4c4d-b485-1c4a8e81dd4d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4ntqt" Jul 08 19:12:06.590917 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:06.590877 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/34dd9e84-4ff2-4c4d-b485-1c4a8e81dd4d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-4ntqt\" (UID: \"34dd9e84-4ff2-4c4d-b485-1c4a8e81dd4d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4ntqt" Jul 08 19:12:06.590917 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:06.590915 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kb2xj\" (UniqueName: \"kubernetes.io/projected/34dd9e84-4ff2-4c4d-b485-1c4a8e81dd4d-kube-api-access-kb2xj\") pod \"kuadrant-operator-controller-manager-86878c769f-4ntqt\" (UID: \"34dd9e84-4ff2-4c4d-b485-1c4a8e81dd4d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4ntqt" Jul 08 19:12:06.591292 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:06.591271 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/34dd9e84-4ff2-4c4d-b485-1c4a8e81dd4d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-4ntqt\" (UID: \"34dd9e84-4ff2-4c4d-b485-1c4a8e81dd4d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4ntqt" Jul 08 19:12:06.604794 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:06.604719 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kb2xj\" (UniqueName: \"kubernetes.io/projected/34dd9e84-4ff2-4c4d-b485-1c4a8e81dd4d-kube-api-access-kb2xj\") pod \"kuadrant-operator-controller-manager-86878c769f-4ntqt\" (UID: \"34dd9e84-4ff2-4c4d-b485-1c4a8e81dd4d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4ntqt" Jul 08 19:12:06.666682 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:06.666650 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4ntqt" Jul 08 19:12:06.938687 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:06.938649 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4ntqt"] Jul 08 19:12:06.951178 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:12:06.951145 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34dd9e84_4ff2_4c4d_b485_1c4a8e81dd4d.slice/crio-26eb824f15f9e9f57847089af4526cce9531795e4734e6ad471849fc36ede638 WatchSource:0}: Error finding container 26eb824f15f9e9f57847089af4526cce9531795e4734e6ad471849fc36ede638: Status 404 returned error can't find the container with id 26eb824f15f9e9f57847089af4526cce9531795e4734e6ad471849fc36ede638 Jul 08 19:12:07.073919 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:07.073883 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4ntqt" event={"ID":"34dd9e84-4ff2-4c4d-b485-1c4a8e81dd4d","Type":"ContainerStarted","Data":"138d9721a512c84d5c9728348270f73a368a5c1bb936719eac60715e0aebec8b"} Jul 08 19:12:07.074077 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:07.073943 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4ntqt" event={"ID":"34dd9e84-4ff2-4c4d-b485-1c4a8e81dd4d","Type":"ContainerStarted","Data":"26eb824f15f9e9f57847089af4526cce9531795e4734e6ad471849fc36ede638"} Jul 08 19:12:07.074077 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:07.073972 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4ntqt" Jul 08 19:12:13.058582 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:13.058553 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rk4xw" Jul 08 19:12:13.102963 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:13.102913 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4ntqt" podStartSLOduration=7.102896339 podStartE2EDuration="7.102896339s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:12:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:12:07.179391179 +0000 UTC m=+492.458478781" watchObservedRunningTime="2026-07-08 19:12:13.102896339 +0000 UTC m=+498.381983959" Jul 08 19:12:15.066765 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:15.066731 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-tn9qn" Jul 08 19:12:18.081038 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:18.081001 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4ntqt" Jul 08 19:12:18.183481 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:18.183445 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rk4xw"] Jul 08 19:12:18.183666 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:18.183632 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rk4xw" podUID="a3dd03ee-6f21-44eb-97b1-2925af3e288a" containerName="manager" containerID="cri-o://09fcae12952384eef228b3f87c84c95421fa585871b13c6a18feabe7facb7989" gracePeriod=10 Jul 08 19:12:18.433031 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:18.433007 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rk4xw" Jul 08 19:12:18.490101 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:18.490074 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a3dd03ee-6f21-44eb-97b1-2925af3e288a-extensions-socket-volume\") pod \"a3dd03ee-6f21-44eb-97b1-2925af3e288a\" (UID: \"a3dd03ee-6f21-44eb-97b1-2925af3e288a\") " Jul 08 19:12:18.490246 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:18.490107 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fbzwk\" (UniqueName: \"kubernetes.io/projected/a3dd03ee-6f21-44eb-97b1-2925af3e288a-kube-api-access-fbzwk\") pod \"a3dd03ee-6f21-44eb-97b1-2925af3e288a\" (UID: \"a3dd03ee-6f21-44eb-97b1-2925af3e288a\") " Jul 08 19:12:18.490559 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:18.490537 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3dd03ee-6f21-44eb-97b1-2925af3e288a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a3dd03ee-6f21-44eb-97b1-2925af3e288a" (UID: "a3dd03ee-6f21-44eb-97b1-2925af3e288a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:12:18.492488 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:18.492454 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3dd03ee-6f21-44eb-97b1-2925af3e288a-kube-api-access-fbzwk" (OuterVolumeSpecName: "kube-api-access-fbzwk") pod "a3dd03ee-6f21-44eb-97b1-2925af3e288a" (UID: "a3dd03ee-6f21-44eb-97b1-2925af3e288a"). InnerVolumeSpecName "kube-api-access-fbzwk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:12:18.590678 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:18.590644 2583 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a3dd03ee-6f21-44eb-97b1-2925af3e288a-extensions-socket-volume\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:12:18.590678 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:18.590674 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fbzwk\" (UniqueName: \"kubernetes.io/projected/a3dd03ee-6f21-44eb-97b1-2925af3e288a-kube-api-access-fbzwk\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:12:19.131045 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:19.131003 2583 generic.go:356] "Generic (PLEG): container finished" podID="a3dd03ee-6f21-44eb-97b1-2925af3e288a" containerID="09fcae12952384eef228b3f87c84c95421fa585871b13c6a18feabe7facb7989" exitCode=0 Jul 08 19:12:19.131544 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:19.131053 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rk4xw" Jul 08 19:12:19.131544 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:19.131087 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rk4xw" event={"ID":"a3dd03ee-6f21-44eb-97b1-2925af3e288a","Type":"ContainerDied","Data":"09fcae12952384eef228b3f87c84c95421fa585871b13c6a18feabe7facb7989"} Jul 08 19:12:19.131544 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:19.131169 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rk4xw" event={"ID":"a3dd03ee-6f21-44eb-97b1-2925af3e288a","Type":"ContainerDied","Data":"62ff0717874023e5f41fe99432151cae9d665f5f6f6c344b13568e5c9eb791e3"} Jul 08 19:12:19.131544 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:19.131193 2583 scope.go:117] "RemoveContainer" containerID="09fcae12952384eef228b3f87c84c95421fa585871b13c6a18feabe7facb7989" Jul 08 19:12:19.140186 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:19.140171 2583 scope.go:117] "RemoveContainer" containerID="09fcae12952384eef228b3f87c84c95421fa585871b13c6a18feabe7facb7989" Jul 08 19:12:19.140479 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:12:19.140462 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09fcae12952384eef228b3f87c84c95421fa585871b13c6a18feabe7facb7989\": container with ID starting with 09fcae12952384eef228b3f87c84c95421fa585871b13c6a18feabe7facb7989 not found: ID does not exist" containerID="09fcae12952384eef228b3f87c84c95421fa585871b13c6a18feabe7facb7989" Jul 08 19:12:19.140532 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:19.140489 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09fcae12952384eef228b3f87c84c95421fa585871b13c6a18feabe7facb7989"} err="failed to get container status \"09fcae12952384eef228b3f87c84c95421fa585871b13c6a18feabe7facb7989\": rpc error: code = NotFound desc = could not find container \"09fcae12952384eef228b3f87c84c95421fa585871b13c6a18feabe7facb7989\": container with ID starting with 09fcae12952384eef228b3f87c84c95421fa585871b13c6a18feabe7facb7989 not found: ID does not exist" Jul 08 19:12:19.175229 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:19.175204 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rk4xw"] Jul 08 19:12:19.181523 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:19.181504 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rk4xw"] Jul 08 19:12:21.173870 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:21.173827 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3dd03ee-6f21-44eb-97b1-2925af3e288a" path="/var/lib/kubelet/pods/a3dd03ee-6f21-44eb-97b1-2925af3e288a/volumes" Jul 08 19:12:34.820978 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:34.820944 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7w2g7"] Jul 08 19:12:34.821402 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:34.821300 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3dd03ee-6f21-44eb-97b1-2925af3e288a" containerName="manager" Jul 08 19:12:34.821402 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:34.821311 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3dd03ee-6f21-44eb-97b1-2925af3e288a" containerName="manager" Jul 08 19:12:34.821402 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:34.821370 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3dd03ee-6f21-44eb-97b1-2925af3e288a" containerName="manager" Jul 08 19:12:34.824764 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:34.824741 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7w2g7" Jul 08 19:12:34.830354 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:34.830325 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-xr27c\"" Jul 08 19:12:34.845510 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:34.845477 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7w2g7"] Jul 08 19:12:34.947691 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:34.947649 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a885949-6c60-41c0-a333-7e9cdb926d69-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-7w2g7\" (UID: \"5a885949-6c60-41c0-a333-7e9cdb926d69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7w2g7" Jul 08 19:12:34.947691 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:34.947696 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a885949-6c60-41c0-a333-7e9cdb926d69-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-7w2g7\" (UID: \"5a885949-6c60-41c0-a333-7e9cdb926d69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7w2g7" Jul 08 19:12:34.947955 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:34.947725 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a885949-6c60-41c0-a333-7e9cdb926d69-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-7w2g7\" (UID: \"5a885949-6c60-41c0-a333-7e9cdb926d69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7w2g7" Jul 08 19:12:34.947955 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:34.947798 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a885949-6c60-41c0-a333-7e9cdb926d69-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-7w2g7\" (UID: \"5a885949-6c60-41c0-a333-7e9cdb926d69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7w2g7" Jul 08 19:12:34.947955 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:34.947837 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a885949-6c60-41c0-a333-7e9cdb926d69-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-7w2g7\" (UID: \"5a885949-6c60-41c0-a333-7e9cdb926d69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7w2g7" Jul 08 19:12:34.947955 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:34.947869 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6l89\" (UniqueName: \"kubernetes.io/projected/5a885949-6c60-41c0-a333-7e9cdb926d69-kube-api-access-k6l89\") pod \"maas-default-gateway-openshift-default-56654dcf7-7w2g7\" (UID: \"5a885949-6c60-41c0-a333-7e9cdb926d69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7w2g7" Jul 08 19:12:34.947955 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:34.947898 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a885949-6c60-41c0-a333-7e9cdb926d69-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-7w2g7\" (UID: \"5a885949-6c60-41c0-a333-7e9cdb926d69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7w2g7" Jul 08 19:12:34.948127 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:34.947993 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a885949-6c60-41c0-a333-7e9cdb926d69-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-7w2g7\" (UID: \"5a885949-6c60-41c0-a333-7e9cdb926d69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7w2g7" Jul 08 19:12:34.948127 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:34.948018 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a885949-6c60-41c0-a333-7e9cdb926d69-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-7w2g7\" (UID: \"5a885949-6c60-41c0-a333-7e9cdb926d69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7w2g7" Jul 08 19:12:35.048722 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:35.048686 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a885949-6c60-41c0-a333-7e9cdb926d69-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-7w2g7\" (UID: \"5a885949-6c60-41c0-a333-7e9cdb926d69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7w2g7" Jul 08 19:12:35.048722 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:35.048735 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a885949-6c60-41c0-a333-7e9cdb926d69-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-7w2g7\" (UID: \"5a885949-6c60-41c0-a333-7e9cdb926d69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7w2g7" Jul 08 19:12:35.048941 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:35.048798 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a885949-6c60-41c0-a333-7e9cdb926d69-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-7w2g7\" (UID: \"5a885949-6c60-41c0-a333-7e9cdb926d69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7w2g7" Jul 08 19:12:35.048941 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:35.048828 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a885949-6c60-41c0-a333-7e9cdb926d69-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-7w2g7\" (UID: \"5a885949-6c60-41c0-a333-7e9cdb926d69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7w2g7" Jul 08 19:12:35.048941 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:35.048872 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a885949-6c60-41c0-a333-7e9cdb926d69-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-7w2g7\" (UID: \"5a885949-6c60-41c0-a333-7e9cdb926d69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7w2g7" Jul 08 19:12:35.048941 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:35.048906 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a885949-6c60-41c0-a333-7e9cdb926d69-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-7w2g7\" (UID: \"5a885949-6c60-41c0-a333-7e9cdb926d69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7w2g7" Jul 08 19:12:35.048941 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:35.048933 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a885949-6c60-41c0-a333-7e9cdb926d69-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-7w2g7\" (UID: \"5a885949-6c60-41c0-a333-7e9cdb926d69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7w2g7" Jul 08 19:12:35.049130 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:35.048965 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6l89\" (UniqueName: \"kubernetes.io/projected/5a885949-6c60-41c0-a333-7e9cdb926d69-kube-api-access-k6l89\") pod \"maas-default-gateway-openshift-default-56654dcf7-7w2g7\" (UID: \"5a885949-6c60-41c0-a333-7e9cdb926d69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7w2g7" Jul 08 19:12:35.049130 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:35.048996 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a885949-6c60-41c0-a333-7e9cdb926d69-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-7w2g7\" (UID: \"5a885949-6c60-41c0-a333-7e9cdb926d69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7w2g7" Jul 08 19:12:35.049227 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:35.049139 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a885949-6c60-41c0-a333-7e9cdb926d69-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-7w2g7\" (UID: \"5a885949-6c60-41c0-a333-7e9cdb926d69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7w2g7" Jul 08 19:12:35.049477 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:35.049398 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a885949-6c60-41c0-a333-7e9cdb926d69-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-7w2g7\" (UID: \"5a885949-6c60-41c0-a333-7e9cdb926d69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7w2g7" Jul 08 19:12:35.049477 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:35.049415 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a885949-6c60-41c0-a333-7e9cdb926d69-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-7w2g7\" (UID: \"5a885949-6c60-41c0-a333-7e9cdb926d69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7w2g7" Jul 08 19:12:35.050037 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:35.050013 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a885949-6c60-41c0-a333-7e9cdb926d69-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-7w2g7\" (UID: \"5a885949-6c60-41c0-a333-7e9cdb926d69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7w2g7" Jul 08 19:12:35.050130 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:35.050075 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a885949-6c60-41c0-a333-7e9cdb926d69-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-7w2g7\" (UID: \"5a885949-6c60-41c0-a333-7e9cdb926d69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7w2g7" Jul 08 19:12:35.051384 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:35.051360 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a885949-6c60-41c0-a333-7e9cdb926d69-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-7w2g7\" (UID: \"5a885949-6c60-41c0-a333-7e9cdb926d69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7w2g7" Jul 08 19:12:35.052059 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:35.052040 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a885949-6c60-41c0-a333-7e9cdb926d69-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-7w2g7\" (UID: \"5a885949-6c60-41c0-a333-7e9cdb926d69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7w2g7" Jul 08 19:12:35.060482 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:35.060456 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a885949-6c60-41c0-a333-7e9cdb926d69-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-7w2g7\" (UID: \"5a885949-6c60-41c0-a333-7e9cdb926d69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7w2g7" Jul 08 19:12:35.060803 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:35.060781 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6l89\" (UniqueName: \"kubernetes.io/projected/5a885949-6c60-41c0-a333-7e9cdb926d69-kube-api-access-k6l89\") pod \"maas-default-gateway-openshift-default-56654dcf7-7w2g7\" (UID: \"5a885949-6c60-41c0-a333-7e9cdb926d69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7w2g7" Jul 08 19:12:35.134544 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:35.134461 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7w2g7" Jul 08 19:12:35.294829 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:35.294804 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7w2g7"] Jul 08 19:12:35.296325 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:12:35.296302 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a885949_6c60_41c0_a333_7e9cdb926d69.slice/crio-c8028a5eac01d74b62821cd702189900c799175cafe870ae4d9ba00f3bb34127 WatchSource:0}: Error finding container c8028a5eac01d74b62821cd702189900c799175cafe870ae4d9ba00f3bb34127: Status 404 returned error can't find the container with id c8028a5eac01d74b62821cd702189900c799175cafe870ae4d9ba00f3bb34127 Jul 08 19:12:35.298459 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:35.298399 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:12:35.298541 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:35.298496 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:12:35.298541 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:35.298528 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:12:36.207894 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:36.207852 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7w2g7" event={"ID":"5a885949-6c60-41c0-a333-7e9cdb926d69","Type":"ContainerStarted","Data":"02370e5398809ab8f6fd046320663d0d15e43fc51e9bc538b39fe7124b6efa77"} Jul 08 19:12:36.208331 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:36.207952 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7w2g7" event={"ID":"5a885949-6c60-41c0-a333-7e9cdb926d69","Type":"ContainerStarted","Data":"c8028a5eac01d74b62821cd702189900c799175cafe870ae4d9ba00f3bb34127"} Jul 08 19:12:36.230417 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:36.230372 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7w2g7" podStartSLOduration=2.230358214 podStartE2EDuration="2.230358214s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:12:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:12:36.22916477 +0000 UTC m=+521.508252391" watchObservedRunningTime="2026-07-08 19:12:36.230358214 +0000 UTC m=+521.509445834" Jul 08 19:12:37.135490 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:37.135453 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7w2g7" Jul 08 19:12:37.140295 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:37.140269 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7w2g7" Jul 08 19:12:37.212031 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:37.211995 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7w2g7" Jul 08 19:12:37.212900 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:37.212880 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7w2g7" Jul 08 19:12:38.168289 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:38.168258 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-vrxbt"] Jul 08 19:12:38.171606 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:38.171585 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-vrxbt" Jul 08 19:12:38.174151 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:38.174131 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-g597r\"" Jul 08 19:12:38.177680 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:38.177659 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-vrxbt"] Jul 08 19:12:38.276565 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:38.276526 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzb5m\" (UniqueName: \"kubernetes.io/projected/00ad183b-67ca-44cf-8741-1aa4a0c8e8da-kube-api-access-nzb5m\") pod \"authorino-58cd955675-vrxbt\" (UID: \"00ad183b-67ca-44cf-8741-1aa4a0c8e8da\") " pod="kuadrant-system/authorino-58cd955675-vrxbt" Jul 08 19:12:38.377962 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:38.377925 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzb5m\" (UniqueName: \"kubernetes.io/projected/00ad183b-67ca-44cf-8741-1aa4a0c8e8da-kube-api-access-nzb5m\") pod \"authorino-58cd955675-vrxbt\" (UID: \"00ad183b-67ca-44cf-8741-1aa4a0c8e8da\") " pod="kuadrant-system/authorino-58cd955675-vrxbt" Jul 08 19:12:38.386949 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:38.386910 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzb5m\" (UniqueName: \"kubernetes.io/projected/00ad183b-67ca-44cf-8741-1aa4a0c8e8da-kube-api-access-nzb5m\") pod \"authorino-58cd955675-vrxbt\" (UID: \"00ad183b-67ca-44cf-8741-1aa4a0c8e8da\") " pod="kuadrant-system/authorino-58cd955675-vrxbt" Jul 08 19:12:38.398065 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:38.398039 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-bhb6v"] Jul 08 19:12:38.401073 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:38.401056 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-bhb6v" Jul 08 19:12:38.409132 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:38.409107 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-bhb6v"] Jul 08 19:12:38.478518 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:38.478399 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hq55s\" (UniqueName: \"kubernetes.io/projected/ea0c3d78-9bac-45d2-832c-80a2623886fd-kube-api-access-hq55s\") pod \"authorino-75d556bd77-bhb6v\" (UID: \"ea0c3d78-9bac-45d2-832c-80a2623886fd\") " pod="kuadrant-system/authorino-75d556bd77-bhb6v" Jul 08 19:12:38.481305 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:38.481284 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-vrxbt" Jul 08 19:12:38.579600 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:38.579567 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hq55s\" (UniqueName: \"kubernetes.io/projected/ea0c3d78-9bac-45d2-832c-80a2623886fd-kube-api-access-hq55s\") pod \"authorino-75d556bd77-bhb6v\" (UID: \"ea0c3d78-9bac-45d2-832c-80a2623886fd\") " pod="kuadrant-system/authorino-75d556bd77-bhb6v" Jul 08 19:12:38.588959 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:38.588927 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hq55s\" (UniqueName: \"kubernetes.io/projected/ea0c3d78-9bac-45d2-832c-80a2623886fd-kube-api-access-hq55s\") pod \"authorino-75d556bd77-bhb6v\" (UID: \"ea0c3d78-9bac-45d2-832c-80a2623886fd\") " pod="kuadrant-system/authorino-75d556bd77-bhb6v" Jul 08 19:12:38.615526 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:38.615395 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-vrxbt"] Jul 08 19:12:38.618010 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:12:38.617983 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod00ad183b_67ca_44cf_8741_1aa4a0c8e8da.slice/crio-2a5ef2a95b5fa15dae10d4421e8be840a44f1a214cf976bcc2c68deb61b900e9 WatchSource:0}: Error finding container 2a5ef2a95b5fa15dae10d4421e8be840a44f1a214cf976bcc2c68deb61b900e9: Status 404 returned error can't find the container with id 2a5ef2a95b5fa15dae10d4421e8be840a44f1a214cf976bcc2c68deb61b900e9 Jul 08 19:12:38.712003 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:38.711972 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-bhb6v" Jul 08 19:12:38.844772 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:38.844750 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-bhb6v"] Jul 08 19:12:38.846909 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:12:38.846882 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podea0c3d78_9bac_45d2_832c_80a2623886fd.slice/crio-b45c76b68464ad3885b7b85d9c6a595b24b8588dab8f2c60b3b78edca490765f WatchSource:0}: Error finding container b45c76b68464ad3885b7b85d9c6a595b24b8588dab8f2c60b3b78edca490765f: Status 404 returned error can't find the container with id b45c76b68464ad3885b7b85d9c6a595b24b8588dab8f2c60b3b78edca490765f Jul 08 19:12:39.227668 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:39.227619 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-bhb6v" event={"ID":"ea0c3d78-9bac-45d2-832c-80a2623886fd","Type":"ContainerStarted","Data":"b45c76b68464ad3885b7b85d9c6a595b24b8588dab8f2c60b3b78edca490765f"} Jul 08 19:12:39.229057 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:39.229023 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-vrxbt" event={"ID":"00ad183b-67ca-44cf-8741-1aa4a0c8e8da","Type":"ContainerStarted","Data":"2a5ef2a95b5fa15dae10d4421e8be840a44f1a214cf976bcc2c68deb61b900e9"} Jul 08 19:12:42.254235 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:42.253944 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-bhb6v" event={"ID":"ea0c3d78-9bac-45d2-832c-80a2623886fd","Type":"ContainerStarted","Data":"87527f37c2d64781878a67e9ef15595d8d6600ad39d33167d31575ff3b7f8b6f"} Jul 08 19:12:42.255778 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:42.255748 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-vrxbt" event={"ID":"00ad183b-67ca-44cf-8741-1aa4a0c8e8da","Type":"ContainerStarted","Data":"9b611044c852b554353f569e923f12cc21b184d07503fe2e730d8703c2ac92e3"} Jul 08 19:12:42.272654 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:42.272602 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-bhb6v" podStartSLOduration=0.939640155 podStartE2EDuration="4.272584902s" totalImagesPullingTime="3.332944747s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:12:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:12:42.270650185 +0000 UTC m=+527.549737812" watchObservedRunningTime="2026-07-08 19:12:42.272584902 +0000 UTC m=+527.551672522" Jul 08 19:12:42.289066 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:42.289014 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-vrxbt" podStartSLOduration=0.738138454 podStartE2EDuration="4.288994636s" totalImagesPullingTime="3.550856182s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:12:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:12:42.288018993 +0000 UTC m=+527.567106614" watchObservedRunningTime="2026-07-08 19:12:42.288994636 +0000 UTC m=+527.568082248" Jul 08 19:12:42.316040 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:42.315709 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-vrxbt"] Jul 08 19:12:44.264941 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:44.264903 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-vrxbt" podUID="00ad183b-67ca-44cf-8741-1aa4a0c8e8da" containerName="authorino" containerID="cri-o://9b611044c852b554353f569e923f12cc21b184d07503fe2e730d8703c2ac92e3" gracePeriod=30 Jul 08 19:12:44.517471 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:44.517388 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-vrxbt" Jul 08 19:12:44.642940 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:44.642910 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nzb5m\" (UniqueName: \"kubernetes.io/projected/00ad183b-67ca-44cf-8741-1aa4a0c8e8da-kube-api-access-nzb5m\") pod \"00ad183b-67ca-44cf-8741-1aa4a0c8e8da\" (UID: \"00ad183b-67ca-44cf-8741-1aa4a0c8e8da\") " Jul 08 19:12:44.645061 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:44.645027 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00ad183b-67ca-44cf-8741-1aa4a0c8e8da-kube-api-access-nzb5m" (OuterVolumeSpecName: "kube-api-access-nzb5m") pod "00ad183b-67ca-44cf-8741-1aa4a0c8e8da" (UID: "00ad183b-67ca-44cf-8741-1aa4a0c8e8da"). InnerVolumeSpecName "kube-api-access-nzb5m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:12:44.743824 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:44.743776 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nzb5m\" (UniqueName: \"kubernetes.io/projected/00ad183b-67ca-44cf-8741-1aa4a0c8e8da-kube-api-access-nzb5m\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:12:45.270020 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:45.269983 2583 generic.go:356] "Generic (PLEG): container finished" podID="00ad183b-67ca-44cf-8741-1aa4a0c8e8da" containerID="9b611044c852b554353f569e923f12cc21b184d07503fe2e730d8703c2ac92e3" exitCode=0 Jul 08 19:12:45.270427 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:45.270044 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-vrxbt" Jul 08 19:12:45.270427 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:45.270071 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-vrxbt" event={"ID":"00ad183b-67ca-44cf-8741-1aa4a0c8e8da","Type":"ContainerDied","Data":"9b611044c852b554353f569e923f12cc21b184d07503fe2e730d8703c2ac92e3"} Jul 08 19:12:45.270427 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:45.270119 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-vrxbt" event={"ID":"00ad183b-67ca-44cf-8741-1aa4a0c8e8da","Type":"ContainerDied","Data":"2a5ef2a95b5fa15dae10d4421e8be840a44f1a214cf976bcc2c68deb61b900e9"} Jul 08 19:12:45.270427 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:45.270143 2583 scope.go:117] "RemoveContainer" containerID="9b611044c852b554353f569e923f12cc21b184d07503fe2e730d8703c2ac92e3" Jul 08 19:12:45.278309 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:45.278272 2583 scope.go:117] "RemoveContainer" containerID="9b611044c852b554353f569e923f12cc21b184d07503fe2e730d8703c2ac92e3" Jul 08 19:12:45.278628 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:12:45.278609 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b611044c852b554353f569e923f12cc21b184d07503fe2e730d8703c2ac92e3\": container with ID starting with 9b611044c852b554353f569e923f12cc21b184d07503fe2e730d8703c2ac92e3 not found: ID does not exist" containerID="9b611044c852b554353f569e923f12cc21b184d07503fe2e730d8703c2ac92e3" Jul 08 19:12:45.278689 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:45.278637 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b611044c852b554353f569e923f12cc21b184d07503fe2e730d8703c2ac92e3"} err="failed to get container status \"9b611044c852b554353f569e923f12cc21b184d07503fe2e730d8703c2ac92e3\": rpc error: code = NotFound desc = could not find container \"9b611044c852b554353f569e923f12cc21b184d07503fe2e730d8703c2ac92e3\": container with ID starting with 9b611044c852b554353f569e923f12cc21b184d07503fe2e730d8703c2ac92e3 not found: ID does not exist" Jul 08 19:12:45.287346 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:45.287324 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-vrxbt"] Jul 08 19:12:45.293033 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:45.293013 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-vrxbt"] Jul 08 19:12:47.174600 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:12:47.174569 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00ad183b-67ca-44cf-8741-1aa4a0c8e8da" path="/var/lib/kubelet/pods/00ad183b-67ca-44cf-8741-1aa4a0c8e8da/volumes" Jul 08 19:13:11.454066 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:11.454028 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-dqwrx"] Jul 08 19:13:11.454475 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:11.454349 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00ad183b-67ca-44cf-8741-1aa4a0c8e8da" containerName="authorino" Jul 08 19:13:11.454475 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:11.454361 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="00ad183b-67ca-44cf-8741-1aa4a0c8e8da" containerName="authorino" Jul 08 19:13:11.454475 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:11.454444 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="00ad183b-67ca-44cf-8741-1aa4a0c8e8da" containerName="authorino" Jul 08 19:13:11.459747 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:11.459724 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-dqwrx" Jul 08 19:13:11.466780 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:11.466748 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-dqwrx"] Jul 08 19:13:11.562521 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:11.562482 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wk74p\" (UniqueName: \"kubernetes.io/projected/4f868966-5792-4727-a361-31660df084e0-kube-api-access-wk74p\") pod \"authorino-5d4b67c446-dqwrx\" (UID: \"4f868966-5792-4727-a361-31660df084e0\") " pod="kuadrant-system/authorino-5d4b67c446-dqwrx" Jul 08 19:13:11.663785 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:11.663727 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wk74p\" (UniqueName: \"kubernetes.io/projected/4f868966-5792-4727-a361-31660df084e0-kube-api-access-wk74p\") pod \"authorino-5d4b67c446-dqwrx\" (UID: \"4f868966-5792-4727-a361-31660df084e0\") " pod="kuadrant-system/authorino-5d4b67c446-dqwrx" Jul 08 19:13:11.675549 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:11.675520 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wk74p\" (UniqueName: \"kubernetes.io/projected/4f868966-5792-4727-a361-31660df084e0-kube-api-access-wk74p\") pod \"authorino-5d4b67c446-dqwrx\" (UID: \"4f868966-5792-4727-a361-31660df084e0\") " pod="kuadrant-system/authorino-5d4b67c446-dqwrx" Jul 08 19:13:11.698274 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:11.698243 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-dqwrx"] Jul 08 19:13:11.698524 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:11.698511 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-dqwrx" Jul 08 19:13:11.740981 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:11.740945 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-dfcbdbcd4-jrhbv"] Jul 08 19:13:11.744716 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:11.744691 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-dfcbdbcd4-jrhbv" Jul 08 19:13:11.754033 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:11.753968 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-dfcbdbcd4-jrhbv"] Jul 08 19:13:11.865527 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:11.865493 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwvc2\" (UniqueName: \"kubernetes.io/projected/598dcdd5-3bc5-4d9b-9a45-9fc17823d335-kube-api-access-jwvc2\") pod \"authorino-dfcbdbcd4-jrhbv\" (UID: \"598dcdd5-3bc5-4d9b-9a45-9fc17823d335\") " pod="kuadrant-system/authorino-dfcbdbcd4-jrhbv" Jul 08 19:13:11.871118 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:11.871096 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-dqwrx"] Jul 08 19:13:11.873840 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:13:11.873809 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4f868966_5792_4727_a361_31660df084e0.slice/crio-f9cf10636c47df0b4cf3784ff568175df5caf3a8c8c3bf9f75a22f9fda126199 WatchSource:0}: Error finding container f9cf10636c47df0b4cf3784ff568175df5caf3a8c8c3bf9f75a22f9fda126199: Status 404 returned error can't find the container with id f9cf10636c47df0b4cf3784ff568175df5caf3a8c8c3bf9f75a22f9fda126199 Jul 08 19:13:11.966057 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:11.965953 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwvc2\" (UniqueName: \"kubernetes.io/projected/598dcdd5-3bc5-4d9b-9a45-9fc17823d335-kube-api-access-jwvc2\") pod \"authorino-dfcbdbcd4-jrhbv\" (UID: \"598dcdd5-3bc5-4d9b-9a45-9fc17823d335\") " pod="kuadrant-system/authorino-dfcbdbcd4-jrhbv" Jul 08 19:13:11.977158 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:11.977134 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwvc2\" (UniqueName: \"kubernetes.io/projected/598dcdd5-3bc5-4d9b-9a45-9fc17823d335-kube-api-access-jwvc2\") pod \"authorino-dfcbdbcd4-jrhbv\" (UID: \"598dcdd5-3bc5-4d9b-9a45-9fc17823d335\") " pod="kuadrant-system/authorino-dfcbdbcd4-jrhbv" Jul 08 19:13:12.055770 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:12.055740 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-dfcbdbcd4-jrhbv" Jul 08 19:13:12.071276 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:12.071246 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-dfcbdbcd4-jrhbv"] Jul 08 19:13:12.112327 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:12.112291 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-99879867b-m977s"] Jul 08 19:13:12.139680 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:12.139641 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-99879867b-m977s"] Jul 08 19:13:12.139803 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:12.139736 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-99879867b-m977s" Jul 08 19:13:12.142781 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:12.142757 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 08 19:13:12.207790 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:12.207764 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-dfcbdbcd4-jrhbv"] Jul 08 19:13:12.209992 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:13:12.209967 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod598dcdd5_3bc5_4d9b_9a45_9fc17823d335.slice/crio-756baba239bb3f5f766a453d1664d539d03946e1f4d18bd8abc784bbed9fa0ec WatchSource:0}: Error finding container 756baba239bb3f5f766a453d1664d539d03946e1f4d18bd8abc784bbed9fa0ec: Status 404 returned error can't find the container with id 756baba239bb3f5f766a453d1664d539d03946e1f4d18bd8abc784bbed9fa0ec Jul 08 19:13:12.269860 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:12.269783 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7w4c\" (UniqueName: \"kubernetes.io/projected/abb9a538-578d-4699-a837-d522d2967ef6-kube-api-access-t7w4c\") pod \"authorino-99879867b-m977s\" (UID: \"abb9a538-578d-4699-a837-d522d2967ef6\") " pod="kuadrant-system/authorino-99879867b-m977s" Jul 08 19:13:12.269860 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:12.269835 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/abb9a538-578d-4699-a837-d522d2967ef6-tls-cert\") pod \"authorino-99879867b-m977s\" (UID: \"abb9a538-578d-4699-a837-d522d2967ef6\") " pod="kuadrant-system/authorino-99879867b-m977s" Jul 08 19:13:12.370758 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:12.370723 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/abb9a538-578d-4699-a837-d522d2967ef6-tls-cert\") pod \"authorino-99879867b-m977s\" (UID: \"abb9a538-578d-4699-a837-d522d2967ef6\") " pod="kuadrant-system/authorino-99879867b-m977s" Jul 08 19:13:12.370903 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:12.370808 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7w4c\" (UniqueName: \"kubernetes.io/projected/abb9a538-578d-4699-a837-d522d2967ef6-kube-api-access-t7w4c\") pod \"authorino-99879867b-m977s\" (UID: \"abb9a538-578d-4699-a837-d522d2967ef6\") " pod="kuadrant-system/authorino-99879867b-m977s" Jul 08 19:13:12.373262 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:12.373234 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/abb9a538-578d-4699-a837-d522d2967ef6-tls-cert\") pod \"authorino-99879867b-m977s\" (UID: \"abb9a538-578d-4699-a837-d522d2967ef6\") " pod="kuadrant-system/authorino-99879867b-m977s" Jul 08 19:13:12.379567 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:12.379547 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7w4c\" (UniqueName: \"kubernetes.io/projected/abb9a538-578d-4699-a837-d522d2967ef6-kube-api-access-t7w4c\") pod \"authorino-99879867b-m977s\" (UID: \"abb9a538-578d-4699-a837-d522d2967ef6\") " pod="kuadrant-system/authorino-99879867b-m977s" Jul 08 19:13:12.388374 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:12.388346 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-dqwrx" event={"ID":"4f868966-5792-4727-a361-31660df084e0","Type":"ContainerStarted","Data":"f9cf10636c47df0b4cf3784ff568175df5caf3a8c8c3bf9f75a22f9fda126199"} Jul 08 19:13:12.389455 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:12.389409 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-dfcbdbcd4-jrhbv" event={"ID":"598dcdd5-3bc5-4d9b-9a45-9fc17823d335","Type":"ContainerStarted","Data":"756baba239bb3f5f766a453d1664d539d03946e1f4d18bd8abc784bbed9fa0ec"} Jul 08 19:13:12.449977 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:12.449954 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-99879867b-m977s" Jul 08 19:13:12.638352 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:12.638318 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-99879867b-m977s"] Jul 08 19:13:12.641709 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:13:12.641681 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podabb9a538_578d_4699_a837_d522d2967ef6.slice/crio-5645c168d4ec3331bfb86669937aa8607c64116bdbcb489ab2feca822ebec366 WatchSource:0}: Error finding container 5645c168d4ec3331bfb86669937aa8607c64116bdbcb489ab2feca822ebec366: Status 404 returned error can't find the container with id 5645c168d4ec3331bfb86669937aa8607c64116bdbcb489ab2feca822ebec366 Jul 08 19:13:13.394710 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:13.394620 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-dqwrx" event={"ID":"4f868966-5792-4727-a361-31660df084e0","Type":"ContainerStarted","Data":"b7dba51c88abb6ac55d318aecc4366438f1fab137a2e5cfe350e9bd1e8a0faee"} Jul 08 19:13:13.394710 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:13.394673 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-dqwrx" podUID="4f868966-5792-4727-a361-31660df084e0" containerName="authorino" containerID="cri-o://b7dba51c88abb6ac55d318aecc4366438f1fab137a2e5cfe350e9bd1e8a0faee" gracePeriod=30 Jul 08 19:13:13.396385 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:13.396355 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-dfcbdbcd4-jrhbv" event={"ID":"598dcdd5-3bc5-4d9b-9a45-9fc17823d335","Type":"ContainerStarted","Data":"d0393a3305c2919a3581c1571bbc73d715c48a0a4d84c0ba3a8443a47fcc4471"} Jul 08 19:13:13.396385 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:13.396372 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-dfcbdbcd4-jrhbv" podUID="598dcdd5-3bc5-4d9b-9a45-9fc17823d335" containerName="authorino" containerID="cri-o://d0393a3305c2919a3581c1571bbc73d715c48a0a4d84c0ba3a8443a47fcc4471" gracePeriod=30 Jul 08 19:13:13.398232 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:13.398206 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-99879867b-m977s" event={"ID":"abb9a538-578d-4699-a837-d522d2967ef6","Type":"ContainerStarted","Data":"36f4941dab9b372b224e9b6603f2d4a63fce19668133fe365b881d36bf50a918"} Jul 08 19:13:13.398344 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:13.398243 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-99879867b-m977s" event={"ID":"abb9a538-578d-4699-a837-d522d2967ef6","Type":"ContainerStarted","Data":"5645c168d4ec3331bfb86669937aa8607c64116bdbcb489ab2feca822ebec366"} Jul 08 19:13:13.411652 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:13.411609 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-dqwrx" podStartSLOduration=1.862349112 podStartE2EDuration="2.411593466s" totalImagesPullingTime="549.244354ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:13:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:13:13.410636603 +0000 UTC m=+558.689724221" watchObservedRunningTime="2026-07-08 19:13:13.411593466 +0000 UTC m=+558.690681085" Jul 08 19:13:13.427483 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:13.427414 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-99879867b-m977s" podStartSLOduration=0.982938485 podStartE2EDuration="1.427398534s" totalImagesPullingTime="444.460049ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:13:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:13:13.426456032 +0000 UTC m=+558.705543643" watchObservedRunningTime="2026-07-08 19:13:13.427398534 +0000 UTC m=+558.706486154" Jul 08 19:13:13.444823 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:13.444765 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-dfcbdbcd4-jrhbv" podStartSLOduration=1.991282993 podStartE2EDuration="2.444740228s" totalImagesPullingTime="453.457235ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:13:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:13:13.442334841 +0000 UTC m=+558.721422470" watchObservedRunningTime="2026-07-08 19:13:13.444740228 +0000 UTC m=+558.723827850" Jul 08 19:13:13.461210 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:13.461182 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-bhb6v"] Jul 08 19:13:13.461372 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:13.461353 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-bhb6v" podUID="ea0c3d78-9bac-45d2-832c-80a2623886fd" containerName="authorino" containerID="cri-o://87527f37c2d64781878a67e9ef15595d8d6600ad39d33167d31575ff3b7f8b6f" gracePeriod=30 Jul 08 19:13:13.758135 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:13.758107 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-dfcbdbcd4-jrhbv" Jul 08 19:13:13.760765 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:13.760744 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-dqwrx" Jul 08 19:13:13.764176 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:13.764156 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-bhb6v" Jul 08 19:13:13.883809 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:13.883771 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hq55s\" (UniqueName: \"kubernetes.io/projected/ea0c3d78-9bac-45d2-832c-80a2623886fd-kube-api-access-hq55s\") pod \"ea0c3d78-9bac-45d2-832c-80a2623886fd\" (UID: \"ea0c3d78-9bac-45d2-832c-80a2623886fd\") " Jul 08 19:13:13.883809 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:13.883812 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jwvc2\" (UniqueName: \"kubernetes.io/projected/598dcdd5-3bc5-4d9b-9a45-9fc17823d335-kube-api-access-jwvc2\") pod \"598dcdd5-3bc5-4d9b-9a45-9fc17823d335\" (UID: \"598dcdd5-3bc5-4d9b-9a45-9fc17823d335\") " Jul 08 19:13:13.884028 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:13.883838 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wk74p\" (UniqueName: \"kubernetes.io/projected/4f868966-5792-4727-a361-31660df084e0-kube-api-access-wk74p\") pod \"4f868966-5792-4727-a361-31660df084e0\" (UID: \"4f868966-5792-4727-a361-31660df084e0\") " Jul 08 19:13:13.886089 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:13.886064 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea0c3d78-9bac-45d2-832c-80a2623886fd-kube-api-access-hq55s" (OuterVolumeSpecName: "kube-api-access-hq55s") pod "ea0c3d78-9bac-45d2-832c-80a2623886fd" (UID: "ea0c3d78-9bac-45d2-832c-80a2623886fd"). InnerVolumeSpecName "kube-api-access-hq55s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:13:13.886194 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:13.886088 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/598dcdd5-3bc5-4d9b-9a45-9fc17823d335-kube-api-access-jwvc2" (OuterVolumeSpecName: "kube-api-access-jwvc2") pod "598dcdd5-3bc5-4d9b-9a45-9fc17823d335" (UID: "598dcdd5-3bc5-4d9b-9a45-9fc17823d335"). InnerVolumeSpecName "kube-api-access-jwvc2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:13:13.886194 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:13.886102 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f868966-5792-4727-a361-31660df084e0-kube-api-access-wk74p" (OuterVolumeSpecName: "kube-api-access-wk74p") pod "4f868966-5792-4727-a361-31660df084e0" (UID: "4f868966-5792-4727-a361-31660df084e0"). InnerVolumeSpecName "kube-api-access-wk74p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:13:13.984934 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:13.984847 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hq55s\" (UniqueName: \"kubernetes.io/projected/ea0c3d78-9bac-45d2-832c-80a2623886fd-kube-api-access-hq55s\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:13:13.984934 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:13.984876 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jwvc2\" (UniqueName: \"kubernetes.io/projected/598dcdd5-3bc5-4d9b-9a45-9fc17823d335-kube-api-access-jwvc2\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:13:13.984934 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:13.984886 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wk74p\" (UniqueName: \"kubernetes.io/projected/4f868966-5792-4727-a361-31660df084e0-kube-api-access-wk74p\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:13:14.404362 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:14.404324 2583 generic.go:356] "Generic (PLEG): container finished" podID="598dcdd5-3bc5-4d9b-9a45-9fc17823d335" containerID="d0393a3305c2919a3581c1571bbc73d715c48a0a4d84c0ba3a8443a47fcc4471" exitCode=0 Jul 08 19:13:14.404559 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:14.404364 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-dfcbdbcd4-jrhbv" event={"ID":"598dcdd5-3bc5-4d9b-9a45-9fc17823d335","Type":"ContainerDied","Data":"d0393a3305c2919a3581c1571bbc73d715c48a0a4d84c0ba3a8443a47fcc4471"} Jul 08 19:13:14.404559 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:14.404378 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-dfcbdbcd4-jrhbv" Jul 08 19:13:14.404559 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:14.404410 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-dfcbdbcd4-jrhbv" event={"ID":"598dcdd5-3bc5-4d9b-9a45-9fc17823d335","Type":"ContainerDied","Data":"756baba239bb3f5f766a453d1664d539d03946e1f4d18bd8abc784bbed9fa0ec"} Jul 08 19:13:14.404559 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:14.404460 2583 scope.go:117] "RemoveContainer" containerID="d0393a3305c2919a3581c1571bbc73d715c48a0a4d84c0ba3a8443a47fcc4471" Jul 08 19:13:14.405868 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:14.405849 2583 generic.go:356] "Generic (PLEG): container finished" podID="ea0c3d78-9bac-45d2-832c-80a2623886fd" containerID="87527f37c2d64781878a67e9ef15595d8d6600ad39d33167d31575ff3b7f8b6f" exitCode=0 Jul 08 19:13:14.405979 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:14.405895 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-bhb6v" Jul 08 19:13:14.405979 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:14.405935 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-bhb6v" event={"ID":"ea0c3d78-9bac-45d2-832c-80a2623886fd","Type":"ContainerDied","Data":"87527f37c2d64781878a67e9ef15595d8d6600ad39d33167d31575ff3b7f8b6f"} Jul 08 19:13:14.405979 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:14.405968 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-bhb6v" event={"ID":"ea0c3d78-9bac-45d2-832c-80a2623886fd","Type":"ContainerDied","Data":"b45c76b68464ad3885b7b85d9c6a595b24b8588dab8f2c60b3b78edca490765f"} Jul 08 19:13:14.407475 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:14.407450 2583 generic.go:356] "Generic (PLEG): container finished" podID="4f868966-5792-4727-a361-31660df084e0" containerID="b7dba51c88abb6ac55d318aecc4366438f1fab137a2e5cfe350e9bd1e8a0faee" exitCode=0 Jul 08 19:13:14.407595 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:14.407490 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-dqwrx" Jul 08 19:13:14.407595 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:14.407504 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-dqwrx" event={"ID":"4f868966-5792-4727-a361-31660df084e0","Type":"ContainerDied","Data":"b7dba51c88abb6ac55d318aecc4366438f1fab137a2e5cfe350e9bd1e8a0faee"} Jul 08 19:13:14.407595 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:14.407546 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-dqwrx" event={"ID":"4f868966-5792-4727-a361-31660df084e0","Type":"ContainerDied","Data":"f9cf10636c47df0b4cf3784ff568175df5caf3a8c8c3bf9f75a22f9fda126199"} Jul 08 19:13:14.413401 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:14.413381 2583 scope.go:117] "RemoveContainer" containerID="d0393a3305c2919a3581c1571bbc73d715c48a0a4d84c0ba3a8443a47fcc4471" Jul 08 19:13:14.413690 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:13:14.413673 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0393a3305c2919a3581c1571bbc73d715c48a0a4d84c0ba3a8443a47fcc4471\": container with ID starting with d0393a3305c2919a3581c1571bbc73d715c48a0a4d84c0ba3a8443a47fcc4471 not found: ID does not exist" containerID="d0393a3305c2919a3581c1571bbc73d715c48a0a4d84c0ba3a8443a47fcc4471" Jul 08 19:13:14.413778 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:14.413696 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0393a3305c2919a3581c1571bbc73d715c48a0a4d84c0ba3a8443a47fcc4471"} err="failed to get container status \"d0393a3305c2919a3581c1571bbc73d715c48a0a4d84c0ba3a8443a47fcc4471\": rpc error: code = NotFound desc = could not find container \"d0393a3305c2919a3581c1571bbc73d715c48a0a4d84c0ba3a8443a47fcc4471\": container with ID starting with d0393a3305c2919a3581c1571bbc73d715c48a0a4d84c0ba3a8443a47fcc4471 not found: ID does not exist" Jul 08 19:13:14.413778 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:14.413715 2583 scope.go:117] "RemoveContainer" containerID="87527f37c2d64781878a67e9ef15595d8d6600ad39d33167d31575ff3b7f8b6f" Jul 08 19:13:14.422336 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:14.422320 2583 scope.go:117] "RemoveContainer" containerID="87527f37c2d64781878a67e9ef15595d8d6600ad39d33167d31575ff3b7f8b6f" Jul 08 19:13:14.422637 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:13:14.422614 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87527f37c2d64781878a67e9ef15595d8d6600ad39d33167d31575ff3b7f8b6f\": container with ID starting with 87527f37c2d64781878a67e9ef15595d8d6600ad39d33167d31575ff3b7f8b6f not found: ID does not exist" containerID="87527f37c2d64781878a67e9ef15595d8d6600ad39d33167d31575ff3b7f8b6f" Jul 08 19:13:14.422723 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:14.422642 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87527f37c2d64781878a67e9ef15595d8d6600ad39d33167d31575ff3b7f8b6f"} err="failed to get container status \"87527f37c2d64781878a67e9ef15595d8d6600ad39d33167d31575ff3b7f8b6f\": rpc error: code = NotFound desc = could not find container \"87527f37c2d64781878a67e9ef15595d8d6600ad39d33167d31575ff3b7f8b6f\": container with ID starting with 87527f37c2d64781878a67e9ef15595d8d6600ad39d33167d31575ff3b7f8b6f not found: ID does not exist" Jul 08 19:13:14.422723 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:14.422658 2583 scope.go:117] "RemoveContainer" containerID="b7dba51c88abb6ac55d318aecc4366438f1fab137a2e5cfe350e9bd1e8a0faee" Jul 08 19:13:14.431956 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:14.431936 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-dfcbdbcd4-jrhbv"] Jul 08 19:13:14.435685 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:14.435663 2583 scope.go:117] "RemoveContainer" containerID="b7dba51c88abb6ac55d318aecc4366438f1fab137a2e5cfe350e9bd1e8a0faee" Jul 08 19:13:14.435754 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:14.435717 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-dfcbdbcd4-jrhbv"] Jul 08 19:13:14.435970 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:13:14.435951 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7dba51c88abb6ac55d318aecc4366438f1fab137a2e5cfe350e9bd1e8a0faee\": container with ID starting with b7dba51c88abb6ac55d318aecc4366438f1fab137a2e5cfe350e9bd1e8a0faee not found: ID does not exist" containerID="b7dba51c88abb6ac55d318aecc4366438f1fab137a2e5cfe350e9bd1e8a0faee" Jul 08 19:13:14.436016 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:14.435989 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7dba51c88abb6ac55d318aecc4366438f1fab137a2e5cfe350e9bd1e8a0faee"} err="failed to get container status \"b7dba51c88abb6ac55d318aecc4366438f1fab137a2e5cfe350e9bd1e8a0faee\": rpc error: code = NotFound desc = could not find container \"b7dba51c88abb6ac55d318aecc4366438f1fab137a2e5cfe350e9bd1e8a0faee\": container with ID starting with b7dba51c88abb6ac55d318aecc4366438f1fab137a2e5cfe350e9bd1e8a0faee not found: ID does not exist" Jul 08 19:13:14.449109 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:14.449086 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-dqwrx"] Jul 08 19:13:14.453213 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:14.453190 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-dqwrx"] Jul 08 19:13:14.464652 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:14.464625 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-bhb6v"] Jul 08 19:13:14.469240 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:14.469218 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-bhb6v"] Jul 08 19:13:15.174651 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:15.174611 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f868966-5792-4727-a361-31660df084e0" path="/var/lib/kubelet/pods/4f868966-5792-4727-a361-31660df084e0/volumes" Jul 08 19:13:15.175217 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:15.175069 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="598dcdd5-3bc5-4d9b-9a45-9fc17823d335" path="/var/lib/kubelet/pods/598dcdd5-3bc5-4d9b-9a45-9fc17823d335/volumes" Jul 08 19:13:15.175469 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:13:15.175428 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea0c3d78-9bac-45d2-832c-80a2623886fd" path="/var/lib/kubelet/pods/ea0c3d78-9bac-45d2-832c-80a2623886fd/volumes" Jul 08 19:14:02.628917 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:02.628885 2583 generic.go:356] "Generic (PLEG): container finished" podID="5a885949-6c60-41c0-a333-7e9cdb926d69" containerID="02370e5398809ab8f6fd046320663d0d15e43fc51e9bc538b39fe7124b6efa77" exitCode=0 Jul 08 19:14:02.629475 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:02.628968 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7w2g7" event={"ID":"5a885949-6c60-41c0-a333-7e9cdb926d69","Type":"ContainerDied","Data":"02370e5398809ab8f6fd046320663d0d15e43fc51e9bc538b39fe7124b6efa77"} Jul 08 19:14:02.629475 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:02.629465 2583 scope.go:117] "RemoveContainer" containerID="02370e5398809ab8f6fd046320663d0d15e43fc51e9bc538b39fe7124b6efa77" Jul 08 19:14:02.630550 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:02.630516 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:14:02.630628 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:02.630606 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:14:02.630671 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:02.630653 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:14:03.635589 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:03.635553 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7w2g7" event={"ID":"5a885949-6c60-41c0-a333-7e9cdb926d69","Type":"ContainerStarted","Data":"dee2557a8867921311e552e2312a08321be7a0ac31cd12c2463a77b8dcfc134e"} Jul 08 19:14:04.135538 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:04.135494 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7w2g7" Jul 08 19:14:04.136685 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:04.136656 2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-7w2g7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body= Jul 08 19:14:04.136803 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:04.136721 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7w2g7" podUID="5a885949-6c60-41c0-a333-7e9cdb926d69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" Jul 08 19:14:05.135562 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:05.135526 2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-7w2g7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body= Jul 08 19:14:05.135982 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:05.135587 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7w2g7" podUID="5a885949-6c60-41c0-a333-7e9cdb926d69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" Jul 08 19:14:06.135209 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:06.135175 2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-7w2g7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body= Jul 08 19:14:06.135380 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:06.135233 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7w2g7" podUID="5a885949-6c60-41c0-a333-7e9cdb926d69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" Jul 08 19:14:07.135286 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:07.135246 2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-7w2g7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body= Jul 08 19:14:07.135722 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:07.135305 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7w2g7" podUID="5a885949-6c60-41c0-a333-7e9cdb926d69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" Jul 08 19:14:07.212182 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:07.212148 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7w2g7" Jul 08 19:14:08.138601 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:08.138567 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7w2g7" Jul 08 19:14:08.139381 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:08.139362 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-7w2g7" Jul 08 19:14:38.808645 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:38.808608 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-bnhkc"] Jul 08 19:14:38.809136 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:38.809120 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea0c3d78-9bac-45d2-832c-80a2623886fd" containerName="authorino" Jul 08 19:14:38.809178 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:38.809140 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea0c3d78-9bac-45d2-832c-80a2623886fd" containerName="authorino" Jul 08 19:14:38.809178 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:38.809166 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="598dcdd5-3bc5-4d9b-9a45-9fc17823d335" containerName="authorino" Jul 08 19:14:38.809178 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:38.809175 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="598dcdd5-3bc5-4d9b-9a45-9fc17823d335" containerName="authorino" Jul 08 19:14:38.809268 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:38.809187 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f868966-5792-4727-a361-31660df084e0" containerName="authorino" Jul 08 19:14:38.809268 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:38.809195 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f868966-5792-4727-a361-31660df084e0" containerName="authorino" Jul 08 19:14:38.809333 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:38.809281 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea0c3d78-9bac-45d2-832c-80a2623886fd" containerName="authorino" Jul 08 19:14:38.809333 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:38.809294 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f868966-5792-4727-a361-31660df084e0" containerName="authorino" Jul 08 19:14:38.809333 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:38.809304 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="598dcdd5-3bc5-4d9b-9a45-9fc17823d335" containerName="authorino" Jul 08 19:14:38.812741 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:38.812723 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bnhkc" Jul 08 19:14:38.815626 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:38.815606 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 08 19:14:38.815769 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:38.815610 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-7lzf5\"" Jul 08 19:14:38.816744 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:38.816723 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 08 19:14:38.816869 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:38.816852 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 08 19:14:38.821419 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:38.821397 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-bnhkc"] Jul 08 19:14:39.000968 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:39.000930 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a031dfc-2a95-4ee0-91df-e2cb55cc167e-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bnhkc\" (UID: \"5a031dfc-2a95-4ee0-91df-e2cb55cc167e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bnhkc" Jul 08 19:14:39.001141 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:39.000977 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a031dfc-2a95-4ee0-91df-e2cb55cc167e-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bnhkc\" (UID: \"5a031dfc-2a95-4ee0-91df-e2cb55cc167e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bnhkc" Jul 08 19:14:39.001141 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:39.001009 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a031dfc-2a95-4ee0-91df-e2cb55cc167e-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bnhkc\" (UID: \"5a031dfc-2a95-4ee0-91df-e2cb55cc167e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bnhkc" Jul 08 19:14:39.001141 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:39.001122 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a031dfc-2a95-4ee0-91df-e2cb55cc167e-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bnhkc\" (UID: \"5a031dfc-2a95-4ee0-91df-e2cb55cc167e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bnhkc" Jul 08 19:14:39.001263 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:39.001198 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sm2ck\" (UniqueName: \"kubernetes.io/projected/5a031dfc-2a95-4ee0-91df-e2cb55cc167e-kube-api-access-sm2ck\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bnhkc\" (UID: \"5a031dfc-2a95-4ee0-91df-e2cb55cc167e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bnhkc" Jul 08 19:14:39.001263 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:39.001235 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a031dfc-2a95-4ee0-91df-e2cb55cc167e-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bnhkc\" (UID: \"5a031dfc-2a95-4ee0-91df-e2cb55cc167e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bnhkc" Jul 08 19:14:39.101882 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:39.101796 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sm2ck\" (UniqueName: \"kubernetes.io/projected/5a031dfc-2a95-4ee0-91df-e2cb55cc167e-kube-api-access-sm2ck\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bnhkc\" (UID: \"5a031dfc-2a95-4ee0-91df-e2cb55cc167e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bnhkc" Jul 08 19:14:39.101882 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:39.101836 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a031dfc-2a95-4ee0-91df-e2cb55cc167e-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bnhkc\" (UID: \"5a031dfc-2a95-4ee0-91df-e2cb55cc167e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bnhkc" Jul 08 19:14:39.101882 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:39.101858 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a031dfc-2a95-4ee0-91df-e2cb55cc167e-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bnhkc\" (UID: \"5a031dfc-2a95-4ee0-91df-e2cb55cc167e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bnhkc" Jul 08 19:14:39.101882 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:39.101884 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a031dfc-2a95-4ee0-91df-e2cb55cc167e-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bnhkc\" (UID: \"5a031dfc-2a95-4ee0-91df-e2cb55cc167e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bnhkc" Jul 08 19:14:39.102184 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:39.101913 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a031dfc-2a95-4ee0-91df-e2cb55cc167e-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bnhkc\" (UID: \"5a031dfc-2a95-4ee0-91df-e2cb55cc167e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bnhkc" Jul 08 19:14:39.102184 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:39.101968 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a031dfc-2a95-4ee0-91df-e2cb55cc167e-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bnhkc\" (UID: \"5a031dfc-2a95-4ee0-91df-e2cb55cc167e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bnhkc" Jul 08 19:14:39.102284 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:39.102257 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a031dfc-2a95-4ee0-91df-e2cb55cc167e-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bnhkc\" (UID: \"5a031dfc-2a95-4ee0-91df-e2cb55cc167e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bnhkc" Jul 08 19:14:39.102336 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:39.102294 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a031dfc-2a95-4ee0-91df-e2cb55cc167e-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bnhkc\" (UID: \"5a031dfc-2a95-4ee0-91df-e2cb55cc167e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bnhkc" Jul 08 19:14:39.102428 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:39.102404 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a031dfc-2a95-4ee0-91df-e2cb55cc167e-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bnhkc\" (UID: \"5a031dfc-2a95-4ee0-91df-e2cb55cc167e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bnhkc" Jul 08 19:14:39.104303 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:39.104283 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a031dfc-2a95-4ee0-91df-e2cb55cc167e-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bnhkc\" (UID: \"5a031dfc-2a95-4ee0-91df-e2cb55cc167e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bnhkc" Jul 08 19:14:39.104646 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:39.104626 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a031dfc-2a95-4ee0-91df-e2cb55cc167e-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bnhkc\" (UID: \"5a031dfc-2a95-4ee0-91df-e2cb55cc167e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bnhkc" Jul 08 19:14:39.110092 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:39.110072 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sm2ck\" (UniqueName: \"kubernetes.io/projected/5a031dfc-2a95-4ee0-91df-e2cb55cc167e-kube-api-access-sm2ck\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bnhkc\" (UID: \"5a031dfc-2a95-4ee0-91df-e2cb55cc167e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bnhkc" Jul 08 19:14:39.124141 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:39.124119 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bnhkc" Jul 08 19:14:39.281699 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:39.281673 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-bnhkc"] Jul 08 19:14:39.284374 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:14:39.284347 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a031dfc_2a95_4ee0_91df_e2cb55cc167e.slice/crio-1f103818f2e2d3523f2e89bfaebfd8accb16a5d8a192f8a51b33dfa96978a076 WatchSource:0}: Error finding container 1f103818f2e2d3523f2e89bfaebfd8accb16a5d8a192f8a51b33dfa96978a076: Status 404 returned error can't find the container with id 1f103818f2e2d3523f2e89bfaebfd8accb16a5d8a192f8a51b33dfa96978a076 Jul 08 19:14:39.799500 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:39.799462 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bnhkc" event={"ID":"5a031dfc-2a95-4ee0-91df-e2cb55cc167e","Type":"ContainerStarted","Data":"1f103818f2e2d3523f2e89bfaebfd8accb16a5d8a192f8a51b33dfa96978a076"} Jul 08 19:14:40.806841 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:40.806750 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bnhkc" event={"ID":"5a031dfc-2a95-4ee0-91df-e2cb55cc167e","Type":"ContainerStarted","Data":"b977837ed3cbfa1afee9dcbfa8ae74713513f0da4eeadba84077eaeb6fb46cc4"} Jul 08 19:14:40.828617 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:40.828570 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bnhkc" podStartSLOduration=1.703689952 podStartE2EDuration="2.82855317s" totalImagesPullingTime="1.124863218s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:14:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:14:40.826519466 +0000 UTC m=+646.105607087" watchObservedRunningTime="2026-07-08 19:14:40.82855317 +0000 UTC m=+646.107640789" Jul 08 19:14:49.124410 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:49.124375 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bnhkc" Jul 08 19:14:49.124818 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:49.124480 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bnhkc" Jul 08 19:14:49.137211 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:49.137183 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bnhkc" Jul 08 19:14:49.857609 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:49.857581 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bnhkc" Jul 08 19:14:56.435328 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:56.435293 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-blpg4"] Jul 08 19:14:56.438795 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:56.438771 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-blpg4" Jul 08 19:14:56.441734 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:56.441710 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 08 19:14:56.451308 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:56.451282 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-blpg4"] Jul 08 19:14:56.549368 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:56.549333 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50133b47-8d89-4405-b67e-a951717e8710-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-blpg4\" (UID: \"50133b47-8d89-4405-b67e-a951717e8710\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-blpg4" Jul 08 19:14:56.549368 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:56.549381 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50133b47-8d89-4405-b67e-a951717e8710-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-blpg4\" (UID: \"50133b47-8d89-4405-b67e-a951717e8710\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-blpg4" Jul 08 19:14:56.549621 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:56.549466 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfq5k\" (UniqueName: \"kubernetes.io/projected/50133b47-8d89-4405-b67e-a951717e8710-kube-api-access-qfq5k\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-blpg4\" (UID: \"50133b47-8d89-4405-b67e-a951717e8710\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-blpg4" Jul 08 19:14:56.549621 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:56.549554 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50133b47-8d89-4405-b67e-a951717e8710-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-blpg4\" (UID: \"50133b47-8d89-4405-b67e-a951717e8710\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-blpg4" Jul 08 19:14:56.549621 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:56.549585 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50133b47-8d89-4405-b67e-a951717e8710-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-blpg4\" (UID: \"50133b47-8d89-4405-b67e-a951717e8710\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-blpg4" Jul 08 19:14:56.549735 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:56.549625 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50133b47-8d89-4405-b67e-a951717e8710-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-blpg4\" (UID: \"50133b47-8d89-4405-b67e-a951717e8710\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-blpg4" Jul 08 19:14:56.650907 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:56.650862 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50133b47-8d89-4405-b67e-a951717e8710-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-blpg4\" (UID: \"50133b47-8d89-4405-b67e-a951717e8710\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-blpg4" Jul 08 19:14:56.651086 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:56.650919 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50133b47-8d89-4405-b67e-a951717e8710-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-blpg4\" (UID: \"50133b47-8d89-4405-b67e-a951717e8710\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-blpg4" Jul 08 19:14:56.651086 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:56.650956 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfq5k\" (UniqueName: \"kubernetes.io/projected/50133b47-8d89-4405-b67e-a951717e8710-kube-api-access-qfq5k\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-blpg4\" (UID: \"50133b47-8d89-4405-b67e-a951717e8710\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-blpg4" Jul 08 19:14:56.651086 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:56.650992 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50133b47-8d89-4405-b67e-a951717e8710-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-blpg4\" (UID: \"50133b47-8d89-4405-b67e-a951717e8710\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-blpg4" Jul 08 19:14:56.651086 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:56.651027 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50133b47-8d89-4405-b67e-a951717e8710-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-blpg4\" (UID: \"50133b47-8d89-4405-b67e-a951717e8710\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-blpg4" Jul 08 19:14:56.651344 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:56.651313 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50133b47-8d89-4405-b67e-a951717e8710-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-blpg4\" (UID: \"50133b47-8d89-4405-b67e-a951717e8710\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-blpg4" Jul 08 19:14:56.651411 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:56.651329 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50133b47-8d89-4405-b67e-a951717e8710-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-blpg4\" (UID: \"50133b47-8d89-4405-b67e-a951717e8710\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-blpg4" Jul 08 19:14:56.651411 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:56.651375 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50133b47-8d89-4405-b67e-a951717e8710-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-blpg4\" (UID: \"50133b47-8d89-4405-b67e-a951717e8710\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-blpg4" Jul 08 19:14:56.651551 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:56.651420 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50133b47-8d89-4405-b67e-a951717e8710-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-blpg4\" (UID: \"50133b47-8d89-4405-b67e-a951717e8710\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-blpg4" Jul 08 19:14:56.653640 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:56.653611 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50133b47-8d89-4405-b67e-a951717e8710-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-blpg4\" (UID: \"50133b47-8d89-4405-b67e-a951717e8710\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-blpg4" Jul 08 19:14:56.653793 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:56.653775 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50133b47-8d89-4405-b67e-a951717e8710-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-blpg4\" (UID: \"50133b47-8d89-4405-b67e-a951717e8710\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-blpg4" Jul 08 19:14:56.663406 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:56.663383 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfq5k\" (UniqueName: \"kubernetes.io/projected/50133b47-8d89-4405-b67e-a951717e8710-kube-api-access-qfq5k\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-blpg4\" (UID: \"50133b47-8d89-4405-b67e-a951717e8710\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-blpg4" Jul 08 19:14:56.750153 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:56.750054 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-blpg4" Jul 08 19:14:56.887126 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:56.887103 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-blpg4"] Jul 08 19:14:56.888837 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:14:56.888807 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50133b47_8d89_4405_b67e_a951717e8710.slice/crio-8a0f44f98b27beb10eff4f94faff902221717b8eb1f31ed731c5fcf9a6703d6d WatchSource:0}: Error finding container 8a0f44f98b27beb10eff4f94faff902221717b8eb1f31ed731c5fcf9a6703d6d: Status 404 returned error can't find the container with id 8a0f44f98b27beb10eff4f94faff902221717b8eb1f31ed731c5fcf9a6703d6d Jul 08 19:14:56.890157 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:56.890142 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 19:14:57.886942 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:57.886903 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-blpg4" event={"ID":"50133b47-8d89-4405-b67e-a951717e8710","Type":"ContainerStarted","Data":"a9e6ea2c1800cf6a6b7a3349cf5831d02a99a307e33007a956d8061ecf6ce2cc"} Jul 08 19:14:57.887360 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:57.886952 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-blpg4" event={"ID":"50133b47-8d89-4405-b67e-a951717e8710","Type":"ContainerStarted","Data":"8a0f44f98b27beb10eff4f94faff902221717b8eb1f31ed731c5fcf9a6703d6d"} Jul 08 19:14:57.921655 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:14:57.921603 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-blpg4" podStartSLOduration=1.7315907670000001 podStartE2EDuration="1.921587885s" totalImagesPullingTime="189.997118ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:14:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:14:57.92033596 +0000 UTC m=+663.199423580" watchObservedRunningTime="2026-07-08 19:14:57.921587885 +0000 UTC m=+663.200675505" Jul 08 19:15:06.751484 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:15:06.751450 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-blpg4" Jul 08 19:15:06.751965 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:15:06.751511 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-blpg4" Jul 08 19:15:06.769398 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:15:06.769367 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-blpg4" Jul 08 19:15:06.936392 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:15:06.936366 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-blpg4" Jul 08 19:16:09.517284 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:16:09.517249 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-664b8dd44f-ffm98"] Jul 08 19:16:09.520799 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:16:09.520776 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-664b8dd44f-ffm98" Jul 08 19:16:09.531740 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:16:09.531716 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-664b8dd44f-ffm98"] Jul 08 19:16:09.625536 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:16:09.625497 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tnfx\" (UniqueName: \"kubernetes.io/projected/65eee363-b056-4e91-8a78-dc9d141ad46c-kube-api-access-5tnfx\") pod \"authorino-664b8dd44f-ffm98\" (UID: \"65eee363-b056-4e91-8a78-dc9d141ad46c\") " pod="kuadrant-system/authorino-664b8dd44f-ffm98" Jul 08 19:16:09.625721 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:16:09.625567 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/65eee363-b056-4e91-8a78-dc9d141ad46c-tls-cert\") pod \"authorino-664b8dd44f-ffm98\" (UID: \"65eee363-b056-4e91-8a78-dc9d141ad46c\") " pod="kuadrant-system/authorino-664b8dd44f-ffm98" Jul 08 19:16:09.726730 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:16:09.726669 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tnfx\" (UniqueName: \"kubernetes.io/projected/65eee363-b056-4e91-8a78-dc9d141ad46c-kube-api-access-5tnfx\") pod \"authorino-664b8dd44f-ffm98\" (UID: \"65eee363-b056-4e91-8a78-dc9d141ad46c\") " pod="kuadrant-system/authorino-664b8dd44f-ffm98" Jul 08 19:16:09.726911 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:16:09.726784 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/65eee363-b056-4e91-8a78-dc9d141ad46c-tls-cert\") pod \"authorino-664b8dd44f-ffm98\" (UID: \"65eee363-b056-4e91-8a78-dc9d141ad46c\") " pod="kuadrant-system/authorino-664b8dd44f-ffm98" Jul 08 19:16:09.729410 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:16:09.729376 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/65eee363-b056-4e91-8a78-dc9d141ad46c-tls-cert\") pod \"authorino-664b8dd44f-ffm98\" (UID: \"65eee363-b056-4e91-8a78-dc9d141ad46c\") " pod="kuadrant-system/authorino-664b8dd44f-ffm98" Jul 08 19:16:09.737984 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:16:09.737954 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tnfx\" (UniqueName: \"kubernetes.io/projected/65eee363-b056-4e91-8a78-dc9d141ad46c-kube-api-access-5tnfx\") pod \"authorino-664b8dd44f-ffm98\" (UID: \"65eee363-b056-4e91-8a78-dc9d141ad46c\") " pod="kuadrant-system/authorino-664b8dd44f-ffm98" Jul 08 19:16:09.829773 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:16:09.829688 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-664b8dd44f-ffm98" Jul 08 19:16:09.974335 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:16:09.974316 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-664b8dd44f-ffm98"] Jul 08 19:16:09.976852 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:16:09.976820 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod65eee363_b056_4e91_8a78_dc9d141ad46c.slice/crio-65259deb610756a81b96179365a3e5586fbc48bce175173f932d91bdddb44bc3 WatchSource:0}: Error finding container 65259deb610756a81b96179365a3e5586fbc48bce175173f932d91bdddb44bc3: Status 404 returned error can't find the container with id 65259deb610756a81b96179365a3e5586fbc48bce175173f932d91bdddb44bc3 Jul 08 19:16:10.205887 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:16:10.205856 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-664b8dd44f-ffm98" event={"ID":"65eee363-b056-4e91-8a78-dc9d141ad46c","Type":"ContainerStarted","Data":"65259deb610756a81b96179365a3e5586fbc48bce175173f932d91bdddb44bc3"} Jul 08 19:16:11.213927 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:16:11.213884 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-664b8dd44f-ffm98" event={"ID":"65eee363-b056-4e91-8a78-dc9d141ad46c","Type":"ContainerStarted","Data":"ea2b8b65bc44a43af39319f335c3b0a856572f6bab964417d4b6c4061d5335ff"} Jul 08 19:16:11.255217 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:16:11.255146 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-664b8dd44f-ffm98" podStartSLOduration=1.799438555 podStartE2EDuration="2.255122933s" totalImagesPullingTime="455.684378ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:16:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:16:11.253616856 +0000 UTC m=+736.532704467" watchObservedRunningTime="2026-07-08 19:16:11.255122933 +0000 UTC m=+736.534210555" Jul 08 19:16:11.309084 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:16:11.309033 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-99879867b-m977s"] Jul 08 19:16:11.309272 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:16:11.309247 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-99879867b-m977s" podUID="abb9a538-578d-4699-a837-d522d2967ef6" containerName="authorino" containerID="cri-o://36f4941dab9b372b224e9b6603f2d4a63fce19668133fe365b881d36bf50a918" gracePeriod=30 Jul 08 19:16:11.575722 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:16:11.575695 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-99879867b-m977s" Jul 08 19:16:11.643080 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:16:11.643044 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t7w4c\" (UniqueName: \"kubernetes.io/projected/abb9a538-578d-4699-a837-d522d2967ef6-kube-api-access-t7w4c\") pod \"abb9a538-578d-4699-a837-d522d2967ef6\" (UID: \"abb9a538-578d-4699-a837-d522d2967ef6\") " Jul 08 19:16:11.643235 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:16:11.643092 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/abb9a538-578d-4699-a837-d522d2967ef6-tls-cert\") pod \"abb9a538-578d-4699-a837-d522d2967ef6\" (UID: \"abb9a538-578d-4699-a837-d522d2967ef6\") " Jul 08 19:16:11.645336 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:16:11.645298 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abb9a538-578d-4699-a837-d522d2967ef6-kube-api-access-t7w4c" (OuterVolumeSpecName: "kube-api-access-t7w4c") pod "abb9a538-578d-4699-a837-d522d2967ef6" (UID: "abb9a538-578d-4699-a837-d522d2967ef6"). InnerVolumeSpecName "kube-api-access-t7w4c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:16:11.654255 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:16:11.654227 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/abb9a538-578d-4699-a837-d522d2967ef6-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "abb9a538-578d-4699-a837-d522d2967ef6" (UID: "abb9a538-578d-4699-a837-d522d2967ef6"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:16:11.743721 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:16:11.743685 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t7w4c\" (UniqueName: \"kubernetes.io/projected/abb9a538-578d-4699-a837-d522d2967ef6-kube-api-access-t7w4c\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:16:11.743721 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:16:11.743715 2583 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/abb9a538-578d-4699-a837-d522d2967ef6-tls-cert\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:16:12.219751 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:16:12.219716 2583 generic.go:356] "Generic (PLEG): container finished" podID="abb9a538-578d-4699-a837-d522d2967ef6" containerID="36f4941dab9b372b224e9b6603f2d4a63fce19668133fe365b881d36bf50a918" exitCode=0 Jul 08 19:16:12.220224 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:16:12.219790 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-99879867b-m977s" Jul 08 19:16:12.220224 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:16:12.219815 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-99879867b-m977s" event={"ID":"abb9a538-578d-4699-a837-d522d2967ef6","Type":"ContainerDied","Data":"36f4941dab9b372b224e9b6603f2d4a63fce19668133fe365b881d36bf50a918"} Jul 08 19:16:12.220224 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:16:12.219871 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-99879867b-m977s" event={"ID":"abb9a538-578d-4699-a837-d522d2967ef6","Type":"ContainerDied","Data":"5645c168d4ec3331bfb86669937aa8607c64116bdbcb489ab2feca822ebec366"} Jul 08 19:16:12.220224 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:16:12.219903 2583 scope.go:117] "RemoveContainer" containerID="36f4941dab9b372b224e9b6603f2d4a63fce19668133fe365b881d36bf50a918" Jul 08 19:16:12.228530 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:16:12.228511 2583 scope.go:117] "RemoveContainer" containerID="36f4941dab9b372b224e9b6603f2d4a63fce19668133fe365b881d36bf50a918" Jul 08 19:16:12.228836 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:16:12.228816 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36f4941dab9b372b224e9b6603f2d4a63fce19668133fe365b881d36bf50a918\": container with ID starting with 36f4941dab9b372b224e9b6603f2d4a63fce19668133fe365b881d36bf50a918 not found: ID does not exist" containerID="36f4941dab9b372b224e9b6603f2d4a63fce19668133fe365b881d36bf50a918" Jul 08 19:16:12.228894 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:16:12.228845 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36f4941dab9b372b224e9b6603f2d4a63fce19668133fe365b881d36bf50a918"} err="failed to get container status \"36f4941dab9b372b224e9b6603f2d4a63fce19668133fe365b881d36bf50a918\": rpc error: code = NotFound desc = could not find container \"36f4941dab9b372b224e9b6603f2d4a63fce19668133fe365b881d36bf50a918\": container with ID starting with 36f4941dab9b372b224e9b6603f2d4a63fce19668133fe365b881d36bf50a918 not found: ID does not exist" Jul 08 19:16:12.245713 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:16:12.245671 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-99879867b-m977s"] Jul 08 19:16:12.249596 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:16:12.249518 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-99879867b-m977s"] Jul 08 19:16:13.174853 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:16:13.174816 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="abb9a538-578d-4699-a837-d522d2967ef6" path="/var/lib/kubelet/pods/abb9a538-578d-4699-a837-d522d2967ef6/volumes" Jul 08 19:22:16.054106 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:22:16.054064 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4ntqt"] Jul 08 19:22:16.054677 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:22:16.054286 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4ntqt" podUID="34dd9e84-4ff2-4c4d-b485-1c4a8e81dd4d" containerName="manager" containerID="cri-o://138d9721a512c84d5c9728348270f73a368a5c1bb936719eac60715e0aebec8b" gracePeriod=10 Jul 08 19:22:18.709830 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:22:18.709804 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4ntqt" Jul 08 19:22:18.820901 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:22:18.820812 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/34dd9e84-4ff2-4c4d-b485-1c4a8e81dd4d-extensions-socket-volume\") pod \"34dd9e84-4ff2-4c4d-b485-1c4a8e81dd4d\" (UID: \"34dd9e84-4ff2-4c4d-b485-1c4a8e81dd4d\") " Jul 08 19:22:18.820901 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:22:18.820894 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kb2xj\" (UniqueName: \"kubernetes.io/projected/34dd9e84-4ff2-4c4d-b485-1c4a8e81dd4d-kube-api-access-kb2xj\") pod \"34dd9e84-4ff2-4c4d-b485-1c4a8e81dd4d\" (UID: \"34dd9e84-4ff2-4c4d-b485-1c4a8e81dd4d\") " Jul 08 19:22:18.821166 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:22:18.821144 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34dd9e84-4ff2-4c4d-b485-1c4a8e81dd4d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "34dd9e84-4ff2-4c4d-b485-1c4a8e81dd4d" (UID: "34dd9e84-4ff2-4c4d-b485-1c4a8e81dd4d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:22:18.823091 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:22:18.823068 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34dd9e84-4ff2-4c4d-b485-1c4a8e81dd4d-kube-api-access-kb2xj" (OuterVolumeSpecName: "kube-api-access-kb2xj") pod "34dd9e84-4ff2-4c4d-b485-1c4a8e81dd4d" (UID: "34dd9e84-4ff2-4c4d-b485-1c4a8e81dd4d"). InnerVolumeSpecName "kube-api-access-kb2xj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:22:18.892572 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:22:18.892530 2583 generic.go:356] "Generic (PLEG): container finished" podID="34dd9e84-4ff2-4c4d-b485-1c4a8e81dd4d" containerID="138d9721a512c84d5c9728348270f73a368a5c1bb936719eac60715e0aebec8b" exitCode=0 Jul 08 19:22:18.892760 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:22:18.892578 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4ntqt" Jul 08 19:22:18.892760 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:22:18.892614 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4ntqt" event={"ID":"34dd9e84-4ff2-4c4d-b485-1c4a8e81dd4d","Type":"ContainerDied","Data":"138d9721a512c84d5c9728348270f73a368a5c1bb936719eac60715e0aebec8b"} Jul 08 19:22:18.892760 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:22:18.892688 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4ntqt" event={"ID":"34dd9e84-4ff2-4c4d-b485-1c4a8e81dd4d","Type":"ContainerDied","Data":"26eb824f15f9e9f57847089af4526cce9531795e4734e6ad471849fc36ede638"} Jul 08 19:22:18.892760 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:22:18.892714 2583 scope.go:117] "RemoveContainer" containerID="138d9721a512c84d5c9728348270f73a368a5c1bb936719eac60715e0aebec8b" Jul 08 19:22:18.902272 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:22:18.902250 2583 scope.go:117] "RemoveContainer" containerID="138d9721a512c84d5c9728348270f73a368a5c1bb936719eac60715e0aebec8b" Jul 08 19:22:18.902602 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:22:18.902581 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"138d9721a512c84d5c9728348270f73a368a5c1bb936719eac60715e0aebec8b\": container with ID starting with 138d9721a512c84d5c9728348270f73a368a5c1bb936719eac60715e0aebec8b not found: ID does not exist" containerID="138d9721a512c84d5c9728348270f73a368a5c1bb936719eac60715e0aebec8b" Jul 08 19:22:18.902669 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:22:18.902613 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"138d9721a512c84d5c9728348270f73a368a5c1bb936719eac60715e0aebec8b"} err="failed to get container status \"138d9721a512c84d5c9728348270f73a368a5c1bb936719eac60715e0aebec8b\": rpc error: code = NotFound desc = could not find container \"138d9721a512c84d5c9728348270f73a368a5c1bb936719eac60715e0aebec8b\": container with ID starting with 138d9721a512c84d5c9728348270f73a368a5c1bb936719eac60715e0aebec8b not found: ID does not exist" Jul 08 19:22:18.915104 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:22:18.915071 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4ntqt"] Jul 08 19:22:18.921796 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:22:18.921770 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4ntqt"] Jul 08 19:22:18.922139 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:22:18.922121 2583 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/34dd9e84-4ff2-4c4d-b485-1c4a8e81dd4d-extensions-socket-volume\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:22:18.922203 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:22:18.922145 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kb2xj\" (UniqueName: \"kubernetes.io/projected/34dd9e84-4ff2-4c4d-b485-1c4a8e81dd4d-kube-api-access-kb2xj\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:22:19.174565 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:22:19.174533 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34dd9e84-4ff2-4c4d-b485-1c4a8e81dd4d" path="/var/lib/kubelet/pods/34dd9e84-4ff2-4c4d-b485-1c4a8e81dd4d/volumes" Jul 08 19:23:17.291358 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:23:17.291274 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d48fl"] Jul 08 19:23:17.291811 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:23:17.291628 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abb9a538-578d-4699-a837-d522d2967ef6" containerName="authorino" Jul 08 19:23:17.291811 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:23:17.291640 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="abb9a538-578d-4699-a837-d522d2967ef6" containerName="authorino" Jul 08 19:23:17.291811 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:23:17.291649 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34dd9e84-4ff2-4c4d-b485-1c4a8e81dd4d" containerName="manager" Jul 08 19:23:17.291811 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:23:17.291655 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="34dd9e84-4ff2-4c4d-b485-1c4a8e81dd4d" containerName="manager" Jul 08 19:23:17.291811 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:23:17.291708 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="abb9a538-578d-4699-a837-d522d2967ef6" containerName="authorino" Jul 08 19:23:17.291811 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:23:17.291716 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="34dd9e84-4ff2-4c4d-b485-1c4a8e81dd4d" containerName="manager" Jul 08 19:23:17.295800 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:23:17.295784 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d48fl" Jul 08 19:23:17.298339 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:23:17.298315 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-gc28r\"" Jul 08 19:23:17.314283 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:23:17.314258 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d48fl"] Jul 08 19:23:17.426729 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:23:17.426696 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8649d14a-1909-4ce9-a380-d35fa273a605-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-d48fl\" (UID: \"8649d14a-1909-4ce9-a380-d35fa273a605\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d48fl" Jul 08 19:23:17.426902 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:23:17.426747 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvs9g\" (UniqueName: \"kubernetes.io/projected/8649d14a-1909-4ce9-a380-d35fa273a605-kube-api-access-dvs9g\") pod \"kuadrant-operator-controller-manager-86878c769f-d48fl\" (UID: \"8649d14a-1909-4ce9-a380-d35fa273a605\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d48fl" Jul 08 19:23:17.528191 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:23:17.528152 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8649d14a-1909-4ce9-a380-d35fa273a605-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-d48fl\" (UID: \"8649d14a-1909-4ce9-a380-d35fa273a605\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d48fl" Jul 08 19:23:17.528337 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:23:17.528215 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvs9g\" (UniqueName: \"kubernetes.io/projected/8649d14a-1909-4ce9-a380-d35fa273a605-kube-api-access-dvs9g\") pod \"kuadrant-operator-controller-manager-86878c769f-d48fl\" (UID: \"8649d14a-1909-4ce9-a380-d35fa273a605\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d48fl" Jul 08 19:23:17.528580 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:23:17.528560 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8649d14a-1909-4ce9-a380-d35fa273a605-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-d48fl\" (UID: \"8649d14a-1909-4ce9-a380-d35fa273a605\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d48fl" Jul 08 19:23:17.536985 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:23:17.536956 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvs9g\" (UniqueName: \"kubernetes.io/projected/8649d14a-1909-4ce9-a380-d35fa273a605-kube-api-access-dvs9g\") pod \"kuadrant-operator-controller-manager-86878c769f-d48fl\" (UID: \"8649d14a-1909-4ce9-a380-d35fa273a605\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d48fl" Jul 08 19:23:17.606025 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:23:17.605936 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d48fl" Jul 08 19:23:17.748579 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:23:17.748553 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d48fl"] Jul 08 19:23:17.750521 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:23:17.750493 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8649d14a_1909_4ce9_a380_d35fa273a605.slice/crio-ef90021d6dcbc6daafb871124a609760901a6736c8de26ce5bbea4914b268fe5 WatchSource:0}: Error finding container ef90021d6dcbc6daafb871124a609760901a6736c8de26ce5bbea4914b268fe5: Status 404 returned error can't find the container with id ef90021d6dcbc6daafb871124a609760901a6736c8de26ce5bbea4914b268fe5 Jul 08 19:23:18.157908 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:23:18.157869 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d48fl" event={"ID":"8649d14a-1909-4ce9-a380-d35fa273a605","Type":"ContainerStarted","Data":"48ec950247178eb3b1913ccd174c41f8fad1baf82d532be8d04a0dbaaba12ef7"} Jul 08 19:23:18.158073 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:23:18.157934 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d48fl" event={"ID":"8649d14a-1909-4ce9-a380-d35fa273a605","Type":"ContainerStarted","Data":"ef90021d6dcbc6daafb871124a609760901a6736c8de26ce5bbea4914b268fe5"} Jul 08 19:23:18.158073 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:23:18.157992 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d48fl" Jul 08 19:23:18.180243 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:23:18.180192 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d48fl" podStartSLOduration=1.18017572 podStartE2EDuration="1.18017572s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:23:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:23:18.177770926 +0000 UTC m=+1163.456858539" watchObservedRunningTime="2026-07-08 19:23:18.18017572 +0000 UTC m=+1163.459263339" Jul 08 19:23:29.165016 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:23:29.164981 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-d48fl" Jul 08 19:29:49.023069 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:29:49.022986 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd"] Jul 08 19:29:49.026654 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:29:49.026634 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd" Jul 08 19:29:49.029323 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:29:49.029300 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-39a4045b-openshift-default-dockercfg-vcrmz\"" Jul 08 19:29:49.048720 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:29:49.048695 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd"] Jul 08 19:29:49.070512 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:29:49.070476 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91f99702-2cb6-4562-b21f-339de91ea64a-workload-certs\") pod \"e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd\" (UID: \"91f99702-2cb6-4562-b21f-339de91ea64a\") " pod="openshift-ingress/e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd" Jul 08 19:29:49.070635 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:29:49.070519 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hxt2\" (UniqueName: \"kubernetes.io/projected/91f99702-2cb6-4562-b21f-339de91ea64a-kube-api-access-2hxt2\") pod \"e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd\" (UID: \"91f99702-2cb6-4562-b21f-339de91ea64a\") " pod="openshift-ingress/e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd" Jul 08 19:29:49.070635 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:29:49.070584 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91f99702-2cb6-4562-b21f-339de91ea64a-credential-socket\") pod \"e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd\" (UID: \"91f99702-2cb6-4562-b21f-339de91ea64a\") " pod="openshift-ingress/e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd" Jul 08 19:29:49.070635 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:29:49.070610 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91f99702-2cb6-4562-b21f-339de91ea64a-istio-data\") pod \"e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd\" (UID: \"91f99702-2cb6-4562-b21f-339de91ea64a\") " pod="openshift-ingress/e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd" Jul 08 19:29:49.070635 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:29:49.070631 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91f99702-2cb6-4562-b21f-339de91ea64a-istio-token\") pod \"e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd\" (UID: \"91f99702-2cb6-4562-b21f-339de91ea64a\") " pod="openshift-ingress/e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd" Jul 08 19:29:49.070838 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:29:49.070653 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91f99702-2cb6-4562-b21f-339de91ea64a-workload-socket\") pod \"e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd\" (UID: \"91f99702-2cb6-4562-b21f-339de91ea64a\") " pod="openshift-ingress/e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd" Jul 08 19:29:49.070838 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:29:49.070672 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91f99702-2cb6-4562-b21f-339de91ea64a-istio-envoy\") pod \"e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd\" (UID: \"91f99702-2cb6-4562-b21f-339de91ea64a\") " pod="openshift-ingress/e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd" Jul 08 19:29:49.070838 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:29:49.070778 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91f99702-2cb6-4562-b21f-339de91ea64a-istiod-ca-cert\") pod \"e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd\" (UID: \"91f99702-2cb6-4562-b21f-339de91ea64a\") " pod="openshift-ingress/e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd" Jul 08 19:29:49.070838 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:29:49.070827 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91f99702-2cb6-4562-b21f-339de91ea64a-istio-podinfo\") pod \"e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd\" (UID: \"91f99702-2cb6-4562-b21f-339de91ea64a\") " pod="openshift-ingress/e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd" Jul 08 19:29:49.171523 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:29:49.171489 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91f99702-2cb6-4562-b21f-339de91ea64a-istiod-ca-cert\") pod \"e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd\" (UID: \"91f99702-2cb6-4562-b21f-339de91ea64a\") " pod="openshift-ingress/e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd" Jul 08 19:29:49.171523 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:29:49.171526 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91f99702-2cb6-4562-b21f-339de91ea64a-istio-podinfo\") pod \"e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd\" (UID: \"91f99702-2cb6-4562-b21f-339de91ea64a\") " pod="openshift-ingress/e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd" Jul 08 19:29:49.171773 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:29:49.171662 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91f99702-2cb6-4562-b21f-339de91ea64a-workload-certs\") pod \"e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd\" (UID: \"91f99702-2cb6-4562-b21f-339de91ea64a\") " pod="openshift-ingress/e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd" Jul 08 19:29:49.171773 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:29:49.171710 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hxt2\" (UniqueName: \"kubernetes.io/projected/91f99702-2cb6-4562-b21f-339de91ea64a-kube-api-access-2hxt2\") pod \"e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd\" (UID: \"91f99702-2cb6-4562-b21f-339de91ea64a\") " pod="openshift-ingress/e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd" Jul 08 19:29:49.171773 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:29:49.171756 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91f99702-2cb6-4562-b21f-339de91ea64a-credential-socket\") pod \"e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd\" (UID: \"91f99702-2cb6-4562-b21f-339de91ea64a\") " pod="openshift-ingress/e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd" Jul 08 19:29:49.171939 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:29:49.171780 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91f99702-2cb6-4562-b21f-339de91ea64a-istio-data\") pod \"e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd\" (UID: \"91f99702-2cb6-4562-b21f-339de91ea64a\") " pod="openshift-ingress/e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd" Jul 08 19:29:49.171939 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:29:49.171810 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91f99702-2cb6-4562-b21f-339de91ea64a-istio-token\") pod \"e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd\" (UID: \"91f99702-2cb6-4562-b21f-339de91ea64a\") " pod="openshift-ingress/e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd" Jul 08 19:29:49.171939 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:29:49.171844 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91f99702-2cb6-4562-b21f-339de91ea64a-workload-socket\") pod \"e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd\" (UID: \"91f99702-2cb6-4562-b21f-339de91ea64a\") " pod="openshift-ingress/e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd" Jul 08 19:29:49.171939 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:29:49.171869 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91f99702-2cb6-4562-b21f-339de91ea64a-istio-envoy\") pod \"e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd\" (UID: \"91f99702-2cb6-4562-b21f-339de91ea64a\") " pod="openshift-ingress/e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd" Jul 08 19:29:49.172157 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:29:49.172018 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91f99702-2cb6-4562-b21f-339de91ea64a-workload-certs\") pod \"e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd\" (UID: \"91f99702-2cb6-4562-b21f-339de91ea64a\") " pod="openshift-ingress/e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd" Jul 08 19:29:49.172157 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:29:49.172120 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91f99702-2cb6-4562-b21f-339de91ea64a-credential-socket\") pod \"e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd\" (UID: \"91f99702-2cb6-4562-b21f-339de91ea64a\") " pod="openshift-ingress/e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd" Jul 08 19:29:49.172305 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:29:49.172281 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91f99702-2cb6-4562-b21f-339de91ea64a-istiod-ca-cert\") pod \"e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd\" (UID: \"91f99702-2cb6-4562-b21f-339de91ea64a\") " pod="openshift-ingress/e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd" Jul 08 19:29:49.172395 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:29:49.172319 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91f99702-2cb6-4562-b21f-339de91ea64a-istio-data\") pod \"e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd\" (UID: \"91f99702-2cb6-4562-b21f-339de91ea64a\") " pod="openshift-ingress/e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd" Jul 08 19:29:49.172476 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:29:49.172389 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91f99702-2cb6-4562-b21f-339de91ea64a-workload-socket\") pod \"e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd\" (UID: \"91f99702-2cb6-4562-b21f-339de91ea64a\") " pod="openshift-ingress/e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd" Jul 08 19:29:49.174179 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:29:49.174157 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91f99702-2cb6-4562-b21f-339de91ea64a-istio-podinfo\") pod \"e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd\" (UID: \"91f99702-2cb6-4562-b21f-339de91ea64a\") " pod="openshift-ingress/e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd" Jul 08 19:29:49.174494 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:29:49.174475 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91f99702-2cb6-4562-b21f-339de91ea64a-istio-envoy\") pod \"e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd\" (UID: \"91f99702-2cb6-4562-b21f-339de91ea64a\") " pod="openshift-ingress/e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd" Jul 08 19:29:49.180119 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:29:49.180092 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91f99702-2cb6-4562-b21f-339de91ea64a-istio-token\") pod \"e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd\" (UID: \"91f99702-2cb6-4562-b21f-339de91ea64a\") " pod="openshift-ingress/e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd" Jul 08 19:29:49.180551 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:29:49.180530 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hxt2\" (UniqueName: \"kubernetes.io/projected/91f99702-2cb6-4562-b21f-339de91ea64a-kube-api-access-2hxt2\") pod \"e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd\" (UID: \"91f99702-2cb6-4562-b21f-339de91ea64a\") " pod="openshift-ingress/e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd" Jul 08 19:29:49.337501 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:29:49.337408 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd" Jul 08 19:29:49.473901 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:29:49.473871 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd"] Jul 08 19:29:49.475938 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:29:49.475911 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91f99702_2cb6_4562_b21f_339de91ea64a.slice/crio-37b25347484aeef7c2a721643eecfa4419bc79d81b83037f61825bd2a06f0e9f WatchSource:0}: Error finding container 37b25347484aeef7c2a721643eecfa4419bc79d81b83037f61825bd2a06f0e9f: Status 404 returned error can't find the container with id 37b25347484aeef7c2a721643eecfa4419bc79d81b83037f61825bd2a06f0e9f Jul 08 19:29:49.478024 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:29:49.477982 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:29:49.478110 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:29:49.478058 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:29:49.478110 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:29:49.478093 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:29:49.973020 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:29:49.972970 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd" event={"ID":"91f99702-2cb6-4562-b21f-339de91ea64a","Type":"ContainerStarted","Data":"fb8afba9a73b642ee0b5b3fb2ab727386e8c9c98b3d0d6349caa61f09b81c62c"} Jul 08 19:29:49.973194 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:29:49.973071 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd" event={"ID":"91f99702-2cb6-4562-b21f-339de91ea64a","Type":"ContainerStarted","Data":"37b25347484aeef7c2a721643eecfa4419bc79d81b83037f61825bd2a06f0e9f"} Jul 08 19:29:49.994426 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:29:49.994373 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd" podStartSLOduration=0.994358875 podStartE2EDuration="994.358875ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:29:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:29:49.993221754 +0000 UTC m=+1555.272309373" watchObservedRunningTime="2026-07-08 19:29:49.994358875 +0000 UTC m=+1555.273446491" Jul 08 19:29:50.338503 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:29:50.338379 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd" Jul 08 19:29:50.360200 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:29:50.360173 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd" Jul 08 19:29:50.978304 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:29:50.978269 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd" Jul 08 19:29:50.979301 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:29:50.979277 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd" Jul 08 19:29:55.394414 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:29:55.394377 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd"] Jul 08 19:29:55.394983 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:29:55.394620 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd" podUID="91f99702-2cb6-4562-b21f-339de91ea64a" containerName="istio-proxy" containerID="cri-o://fb8afba9a73b642ee0b5b3fb2ab727386e8c9c98b3d0d6349caa61f09b81c62c" gracePeriod=30 Jul 08 19:30:01.792732 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:01.792692 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2"] Jul 08 19:30:01.797065 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:01.797042 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2" Jul 08 19:30:01.800501 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:01.800475 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-19b7c553-openshift-default-dockercfg-nmlqj\"" Jul 08 19:30:01.811146 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:01.811114 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2"] Jul 08 19:30:01.896168 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:01.896125 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97bb796b-1ec4-4c0b-8171-d59a80145448-istio-podinfo\") pod \"e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2\" (UID: \"97bb796b-1ec4-4c0b-8171-d59a80145448\") " pod="openshift-ingress/e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2" Jul 08 19:30:01.896363 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:01.896183 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97bb796b-1ec4-4c0b-8171-d59a80145448-istio-data\") pod \"e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2\" (UID: \"97bb796b-1ec4-4c0b-8171-d59a80145448\") " pod="openshift-ingress/e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2" Jul 08 19:30:01.896363 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:01.896206 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mrqr\" (UniqueName: \"kubernetes.io/projected/97bb796b-1ec4-4c0b-8171-d59a80145448-kube-api-access-4mrqr\") pod \"e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2\" (UID: \"97bb796b-1ec4-4c0b-8171-d59a80145448\") " pod="openshift-ingress/e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2" Jul 08 19:30:01.896363 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:01.896254 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97bb796b-1ec4-4c0b-8171-d59a80145448-credential-socket\") pod \"e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2\" (UID: \"97bb796b-1ec4-4c0b-8171-d59a80145448\") " pod="openshift-ingress/e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2" Jul 08 19:30:01.896363 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:01.896323 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97bb796b-1ec4-4c0b-8171-d59a80145448-workload-socket\") pod \"e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2\" (UID: \"97bb796b-1ec4-4c0b-8171-d59a80145448\") " pod="openshift-ingress/e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2" Jul 08 19:30:01.896363 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:01.896344 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97bb796b-1ec4-4c0b-8171-d59a80145448-istio-token\") pod \"e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2\" (UID: \"97bb796b-1ec4-4c0b-8171-d59a80145448\") " pod="openshift-ingress/e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2" Jul 08 19:30:01.896601 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:01.896366 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97bb796b-1ec4-4c0b-8171-d59a80145448-istiod-ca-cert\") pod \"e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2\" (UID: \"97bb796b-1ec4-4c0b-8171-d59a80145448\") " pod="openshift-ingress/e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2" Jul 08 19:30:01.896601 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:01.896402 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97bb796b-1ec4-4c0b-8171-d59a80145448-istio-envoy\") pod \"e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2\" (UID: \"97bb796b-1ec4-4c0b-8171-d59a80145448\") " pod="openshift-ingress/e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2" Jul 08 19:30:01.896601 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:01.896505 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97bb796b-1ec4-4c0b-8171-d59a80145448-workload-certs\") pod \"e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2\" (UID: \"97bb796b-1ec4-4c0b-8171-d59a80145448\") " pod="openshift-ingress/e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2" Jul 08 19:30:01.997768 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:01.997730 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97bb796b-1ec4-4c0b-8171-d59a80145448-istio-envoy\") pod \"e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2\" (UID: \"97bb796b-1ec4-4c0b-8171-d59a80145448\") " pod="openshift-ingress/e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2" Jul 08 19:30:02.000082 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:01.997780 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97bb796b-1ec4-4c0b-8171-d59a80145448-workload-certs\") pod \"e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2\" (UID: \"97bb796b-1ec4-4c0b-8171-d59a80145448\") " pod="openshift-ingress/e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2" Jul 08 19:30:02.000082 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:01.997814 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97bb796b-1ec4-4c0b-8171-d59a80145448-istio-podinfo\") pod \"e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2\" (UID: \"97bb796b-1ec4-4c0b-8171-d59a80145448\") " pod="openshift-ingress/e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2" Jul 08 19:30:02.000082 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:01.997839 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97bb796b-1ec4-4c0b-8171-d59a80145448-istio-data\") pod \"e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2\" (UID: \"97bb796b-1ec4-4c0b-8171-d59a80145448\") " pod="openshift-ingress/e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2" Jul 08 19:30:02.000082 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:01.997855 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mrqr\" (UniqueName: \"kubernetes.io/projected/97bb796b-1ec4-4c0b-8171-d59a80145448-kube-api-access-4mrqr\") pod \"e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2\" (UID: \"97bb796b-1ec4-4c0b-8171-d59a80145448\") " pod="openshift-ingress/e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2" Jul 08 19:30:02.000082 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:01.997883 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97bb796b-1ec4-4c0b-8171-d59a80145448-credential-socket\") pod \"e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2\" (UID: \"97bb796b-1ec4-4c0b-8171-d59a80145448\") " pod="openshift-ingress/e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2" Jul 08 19:30:02.000082 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:01.997956 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97bb796b-1ec4-4c0b-8171-d59a80145448-workload-socket\") pod \"e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2\" (UID: \"97bb796b-1ec4-4c0b-8171-d59a80145448\") " pod="openshift-ingress/e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2" Jul 08 19:30:02.000082 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:01.997982 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97bb796b-1ec4-4c0b-8171-d59a80145448-istio-token\") pod \"e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2\" (UID: \"97bb796b-1ec4-4c0b-8171-d59a80145448\") " pod="openshift-ingress/e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2" Jul 08 19:30:02.000082 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:01.998007 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97bb796b-1ec4-4c0b-8171-d59a80145448-istiod-ca-cert\") pod \"e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2\" (UID: \"97bb796b-1ec4-4c0b-8171-d59a80145448\") " pod="openshift-ingress/e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2" Jul 08 19:30:02.000082 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:01.998276 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97bb796b-1ec4-4c0b-8171-d59a80145448-workload-certs\") pod \"e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2\" (UID: \"97bb796b-1ec4-4c0b-8171-d59a80145448\") " pod="openshift-ingress/e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2" Jul 08 19:30:02.000082 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:01.998331 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97bb796b-1ec4-4c0b-8171-d59a80145448-workload-socket\") pod \"e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2\" (UID: \"97bb796b-1ec4-4c0b-8171-d59a80145448\") " pod="openshift-ingress/e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2" Jul 08 19:30:02.000082 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:01.998460 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97bb796b-1ec4-4c0b-8171-d59a80145448-credential-socket\") pod \"e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2\" (UID: \"97bb796b-1ec4-4c0b-8171-d59a80145448\") " pod="openshift-ingress/e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2" Jul 08 19:30:02.000082 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:01.998590 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97bb796b-1ec4-4c0b-8171-d59a80145448-istio-data\") pod \"e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2\" (UID: \"97bb796b-1ec4-4c0b-8171-d59a80145448\") " pod="openshift-ingress/e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2" Jul 08 19:30:02.000082 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:01.998877 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97bb796b-1ec4-4c0b-8171-d59a80145448-istiod-ca-cert\") pod \"e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2\" (UID: \"97bb796b-1ec4-4c0b-8171-d59a80145448\") " pod="openshift-ingress/e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2" Jul 08 19:30:02.000521 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:02.000300 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97bb796b-1ec4-4c0b-8171-d59a80145448-istio-envoy\") pod \"e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2\" (UID: \"97bb796b-1ec4-4c0b-8171-d59a80145448\") " pod="openshift-ingress/e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2" Jul 08 19:30:02.000605 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:02.000589 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97bb796b-1ec4-4c0b-8171-d59a80145448-istio-podinfo\") pod \"e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2\" (UID: \"97bb796b-1ec4-4c0b-8171-d59a80145448\") " pod="openshift-ingress/e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2" Jul 08 19:30:02.008748 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:02.008723 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97bb796b-1ec4-4c0b-8171-d59a80145448-istio-token\") pod \"e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2\" (UID: \"97bb796b-1ec4-4c0b-8171-d59a80145448\") " pod="openshift-ingress/e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2" Jul 08 19:30:02.009270 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:02.009250 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mrqr\" (UniqueName: \"kubernetes.io/projected/97bb796b-1ec4-4c0b-8171-d59a80145448-kube-api-access-4mrqr\") pod \"e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2\" (UID: \"97bb796b-1ec4-4c0b-8171-d59a80145448\") " pod="openshift-ingress/e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2" Jul 08 19:30:02.109080 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:02.109053 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2" Jul 08 19:30:02.260752 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:02.260707 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2"] Jul 08 19:30:02.264470 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:30:02.264425 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97bb796b_1ec4_4c0b_8171_d59a80145448.slice/crio-0ee9c0f54913f4a6e72e6e8b3dc6e06ccf3ce38e34f128f23e0b6dadad683f73 WatchSource:0}: Error finding container 0ee9c0f54913f4a6e72e6e8b3dc6e06ccf3ce38e34f128f23e0b6dadad683f73: Status 404 returned error can't find the container with id 0ee9c0f54913f4a6e72e6e8b3dc6e06ccf3ce38e34f128f23e0b6dadad683f73 Jul 08 19:30:02.266467 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:02.266419 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:30:02.266547 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:02.266498 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:30:02.266547 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:02.266525 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:30:02.269126 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:02.269106 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd" Jul 08 19:30:02.402514 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:02.402409 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91f99702-2cb6-4562-b21f-339de91ea64a-istiod-ca-cert\") pod \"91f99702-2cb6-4562-b21f-339de91ea64a\" (UID: \"91f99702-2cb6-4562-b21f-339de91ea64a\") " Jul 08 19:30:02.402514 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:02.402480 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91f99702-2cb6-4562-b21f-339de91ea64a-istio-podinfo\") pod \"91f99702-2cb6-4562-b21f-339de91ea64a\" (UID: \"91f99702-2cb6-4562-b21f-339de91ea64a\") " Jul 08 19:30:02.402746 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:02.402582 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91f99702-2cb6-4562-b21f-339de91ea64a-istio-token\") pod \"91f99702-2cb6-4562-b21f-339de91ea64a\" (UID: \"91f99702-2cb6-4562-b21f-339de91ea64a\") " Jul 08 19:30:02.402746 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:02.402639 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91f99702-2cb6-4562-b21f-339de91ea64a-credential-socket\") pod \"91f99702-2cb6-4562-b21f-339de91ea64a\" (UID: \"91f99702-2cb6-4562-b21f-339de91ea64a\") " Jul 08 19:30:02.402746 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:02.402674 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91f99702-2cb6-4562-b21f-339de91ea64a-workload-socket\") pod \"91f99702-2cb6-4562-b21f-339de91ea64a\" (UID: \"91f99702-2cb6-4562-b21f-339de91ea64a\") " Jul 08 19:30:02.402746 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:02.402711 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91f99702-2cb6-4562-b21f-339de91ea64a-istio-data\") pod \"91f99702-2cb6-4562-b21f-339de91ea64a\" (UID: \"91f99702-2cb6-4562-b21f-339de91ea64a\") " Jul 08 19:30:02.402950 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:02.402756 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2hxt2\" (UniqueName: \"kubernetes.io/projected/91f99702-2cb6-4562-b21f-339de91ea64a-kube-api-access-2hxt2\") pod \"91f99702-2cb6-4562-b21f-339de91ea64a\" (UID: \"91f99702-2cb6-4562-b21f-339de91ea64a\") " Jul 08 19:30:02.402950 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:02.402793 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91f99702-2cb6-4562-b21f-339de91ea64a-workload-certs\") pod \"91f99702-2cb6-4562-b21f-339de91ea64a\" (UID: \"91f99702-2cb6-4562-b21f-339de91ea64a\") " Jul 08 19:30:02.402950 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:02.402816 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91f99702-2cb6-4562-b21f-339de91ea64a-istio-envoy\") pod \"91f99702-2cb6-4562-b21f-339de91ea64a\" (UID: \"91f99702-2cb6-4562-b21f-339de91ea64a\") " Jul 08 19:30:02.402950 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:02.402841 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91f99702-2cb6-4562-b21f-339de91ea64a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "91f99702-2cb6-4562-b21f-339de91ea64a" (UID: "91f99702-2cb6-4562-b21f-339de91ea64a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:30:02.403158 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:02.402966 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91f99702-2cb6-4562-b21f-339de91ea64a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "91f99702-2cb6-4562-b21f-339de91ea64a" (UID: "91f99702-2cb6-4562-b21f-339de91ea64a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:30:02.403244 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:02.403208 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91f99702-2cb6-4562-b21f-339de91ea64a-istiod-ca-cert\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:30:02.403244 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:02.403232 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91f99702-2cb6-4562-b21f-339de91ea64a-credential-socket\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:30:02.403708 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:02.403672 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91f99702-2cb6-4562-b21f-339de91ea64a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "91f99702-2cb6-4562-b21f-339de91ea64a" (UID: "91f99702-2cb6-4562-b21f-339de91ea64a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:30:02.403832 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:02.403805 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91f99702-2cb6-4562-b21f-339de91ea64a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "91f99702-2cb6-4562-b21f-339de91ea64a" (UID: "91f99702-2cb6-4562-b21f-339de91ea64a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:30:02.403893 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:02.403837 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91f99702-2cb6-4562-b21f-339de91ea64a-istio-data" (OuterVolumeSpecName: "istio-data") pod "91f99702-2cb6-4562-b21f-339de91ea64a" (UID: "91f99702-2cb6-4562-b21f-339de91ea64a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:30:02.405151 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:02.405125 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/91f99702-2cb6-4562-b21f-339de91ea64a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "91f99702-2cb6-4562-b21f-339de91ea64a" (UID: "91f99702-2cb6-4562-b21f-339de91ea64a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 19:30:02.405283 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:02.405254 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91f99702-2cb6-4562-b21f-339de91ea64a-istio-token" (OuterVolumeSpecName: "istio-token") pod "91f99702-2cb6-4562-b21f-339de91ea64a" (UID: "91f99702-2cb6-4562-b21f-339de91ea64a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:30:02.405672 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:02.405651 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91f99702-2cb6-4562-b21f-339de91ea64a-kube-api-access-2hxt2" (OuterVolumeSpecName: "kube-api-access-2hxt2") pod "91f99702-2cb6-4562-b21f-339de91ea64a" (UID: "91f99702-2cb6-4562-b21f-339de91ea64a"). InnerVolumeSpecName "kube-api-access-2hxt2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:30:02.405978 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:02.405955 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91f99702-2cb6-4562-b21f-339de91ea64a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "91f99702-2cb6-4562-b21f-339de91ea64a" (UID: "91f99702-2cb6-4562-b21f-339de91ea64a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:30:02.504114 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:02.504084 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91f99702-2cb6-4562-b21f-339de91ea64a-istio-podinfo\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:30:02.504114 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:02.504112 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91f99702-2cb6-4562-b21f-339de91ea64a-istio-token\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:30:02.504114 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:02.504121 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91f99702-2cb6-4562-b21f-339de91ea64a-workload-socket\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:30:02.504324 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:02.504130 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91f99702-2cb6-4562-b21f-339de91ea64a-istio-data\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:30:02.504324 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:02.504139 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2hxt2\" (UniqueName: \"kubernetes.io/projected/91f99702-2cb6-4562-b21f-339de91ea64a-kube-api-access-2hxt2\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:30:02.504324 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:02.504149 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91f99702-2cb6-4562-b21f-339de91ea64a-workload-certs\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:30:02.504324 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:02.504157 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91f99702-2cb6-4562-b21f-339de91ea64a-istio-envoy\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:30:03.032239 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:03.032205 2583 generic.go:356] "Generic (PLEG): container finished" podID="91f99702-2cb6-4562-b21f-339de91ea64a" containerID="fb8afba9a73b642ee0b5b3fb2ab727386e8c9c98b3d0d6349caa61f09b81c62c" exitCode=0 Jul 08 19:30:03.032239 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:03.032254 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd" Jul 08 19:30:03.032779 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:03.032293 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd" event={"ID":"91f99702-2cb6-4562-b21f-339de91ea64a","Type":"ContainerDied","Data":"fb8afba9a73b642ee0b5b3fb2ab727386e8c9c98b3d0d6349caa61f09b81c62c"} Jul 08 19:30:03.032779 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:03.032348 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd" event={"ID":"91f99702-2cb6-4562-b21f-339de91ea64a","Type":"ContainerDied","Data":"37b25347484aeef7c2a721643eecfa4419bc79d81b83037f61825bd2a06f0e9f"} Jul 08 19:30:03.032779 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:03.032371 2583 scope.go:117] "RemoveContainer" containerID="fb8afba9a73b642ee0b5b3fb2ab727386e8c9c98b3d0d6349caa61f09b81c62c" Jul 08 19:30:03.034092 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:03.034067 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2" event={"ID":"97bb796b-1ec4-4c0b-8171-d59a80145448","Type":"ContainerStarted","Data":"501f602373d24165b9800054d843cdb2df0f20b04fd18be629e4d4618a129178"} Jul 08 19:30:03.034271 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:03.034244 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2" event={"ID":"97bb796b-1ec4-4c0b-8171-d59a80145448","Type":"ContainerStarted","Data":"0ee9c0f54913f4a6e72e6e8b3dc6e06ccf3ce38e34f128f23e0b6dadad683f73"} Jul 08 19:30:03.042989 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:03.042971 2583 scope.go:117] "RemoveContainer" containerID="fb8afba9a73b642ee0b5b3fb2ab727386e8c9c98b3d0d6349caa61f09b81c62c" Jul 08 19:30:03.043341 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:30:03.043316 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb8afba9a73b642ee0b5b3fb2ab727386e8c9c98b3d0d6349caa61f09b81c62c\": container with ID starting with fb8afba9a73b642ee0b5b3fb2ab727386e8c9c98b3d0d6349caa61f09b81c62c not found: ID does not exist" containerID="fb8afba9a73b642ee0b5b3fb2ab727386e8c9c98b3d0d6349caa61f09b81c62c" Jul 08 19:30:03.043408 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:03.043354 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb8afba9a73b642ee0b5b3fb2ab727386e8c9c98b3d0d6349caa61f09b81c62c"} err="failed to get container status \"fb8afba9a73b642ee0b5b3fb2ab727386e8c9c98b3d0d6349caa61f09b81c62c\": rpc error: code = NotFound desc = could not find container \"fb8afba9a73b642ee0b5b3fb2ab727386e8c9c98b3d0d6349caa61f09b81c62c\": container with ID starting with fb8afba9a73b642ee0b5b3fb2ab727386e8c9c98b3d0d6349caa61f09b81c62c not found: ID does not exist" Jul 08 19:30:03.063738 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:03.063693 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2" podStartSLOduration=2.06367463 podStartE2EDuration="2.06367463s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:30:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:30:03.062699241 +0000 UTC m=+1568.341786860" watchObservedRunningTime="2026-07-08 19:30:03.06367463 +0000 UTC m=+1568.342762249" Jul 08 19:30:03.087253 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:03.087221 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd"] Jul 08 19:30:03.095012 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:03.094982 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-39a4045b-openshift-default-668dfffcff-zshjd"] Jul 08 19:30:03.109949 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:03.109925 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2" Jul 08 19:30:03.114536 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:03.114514 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2" Jul 08 19:30:03.176174 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:03.176145 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91f99702-2cb6-4562-b21f-339de91ea64a" path="/var/lib/kubelet/pods/91f99702-2cb6-4562-b21f-339de91ea64a/volumes" Jul 08 19:30:04.039912 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:04.039884 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2" Jul 08 19:30:04.040819 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:04.040801 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2" Jul 08 19:30:14.126131 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:14.126095 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2"] Jul 08 19:30:14.126536 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:14.126305 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2" podUID="97bb796b-1ec4-4c0b-8171-d59a80145448" containerName="istio-proxy" containerID="cri-o://501f602373d24165b9800054d843cdb2df0f20b04fd18be629e4d4618a129178" gracePeriod=30 Jul 08 19:30:14.482599 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:14.482574 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2" Jul 08 19:30:14.622537 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:14.622507 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97bb796b-1ec4-4c0b-8171-d59a80145448-istio-podinfo\") pod \"97bb796b-1ec4-4c0b-8171-d59a80145448\" (UID: \"97bb796b-1ec4-4c0b-8171-d59a80145448\") " Jul 08 19:30:14.622537 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:14.622539 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4mrqr\" (UniqueName: \"kubernetes.io/projected/97bb796b-1ec4-4c0b-8171-d59a80145448-kube-api-access-4mrqr\") pod \"97bb796b-1ec4-4c0b-8171-d59a80145448\" (UID: \"97bb796b-1ec4-4c0b-8171-d59a80145448\") " Jul 08 19:30:14.622784 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:14.622564 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97bb796b-1ec4-4c0b-8171-d59a80145448-credential-socket\") pod \"97bb796b-1ec4-4c0b-8171-d59a80145448\" (UID: \"97bb796b-1ec4-4c0b-8171-d59a80145448\") " Jul 08 19:30:14.622784 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:14.622588 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97bb796b-1ec4-4c0b-8171-d59a80145448-workload-socket\") pod \"97bb796b-1ec4-4c0b-8171-d59a80145448\" (UID: \"97bb796b-1ec4-4c0b-8171-d59a80145448\") " Jul 08 19:30:14.622784 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:14.622643 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97bb796b-1ec4-4c0b-8171-d59a80145448-workload-certs\") pod \"97bb796b-1ec4-4c0b-8171-d59a80145448\" (UID: \"97bb796b-1ec4-4c0b-8171-d59a80145448\") " Jul 08 19:30:14.622784 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:14.622671 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97bb796b-1ec4-4c0b-8171-d59a80145448-istio-data\") pod \"97bb796b-1ec4-4c0b-8171-d59a80145448\" (UID: \"97bb796b-1ec4-4c0b-8171-d59a80145448\") " Jul 08 19:30:14.622784 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:14.622698 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97bb796b-1ec4-4c0b-8171-d59a80145448-istio-envoy\") pod \"97bb796b-1ec4-4c0b-8171-d59a80145448\" (UID: \"97bb796b-1ec4-4c0b-8171-d59a80145448\") " Jul 08 19:30:14.622784 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:14.622735 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97bb796b-1ec4-4c0b-8171-d59a80145448-istio-token\") pod \"97bb796b-1ec4-4c0b-8171-d59a80145448\" (UID: \"97bb796b-1ec4-4c0b-8171-d59a80145448\") " Jul 08 19:30:14.622784 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:14.622780 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97bb796b-1ec4-4c0b-8171-d59a80145448-istiod-ca-cert\") pod \"97bb796b-1ec4-4c0b-8171-d59a80145448\" (UID: \"97bb796b-1ec4-4c0b-8171-d59a80145448\") " Jul 08 19:30:14.623112 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:14.622901 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97bb796b-1ec4-4c0b-8171-d59a80145448-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "97bb796b-1ec4-4c0b-8171-d59a80145448" (UID: "97bb796b-1ec4-4c0b-8171-d59a80145448"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:30:14.623112 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:14.622994 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97bb796b-1ec4-4c0b-8171-d59a80145448-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "97bb796b-1ec4-4c0b-8171-d59a80145448" (UID: "97bb796b-1ec4-4c0b-8171-d59a80145448"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:30:14.623112 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:14.622999 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97bb796b-1ec4-4c0b-8171-d59a80145448-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "97bb796b-1ec4-4c0b-8171-d59a80145448" (UID: "97bb796b-1ec4-4c0b-8171-d59a80145448"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:30:14.623271 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:14.623151 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97bb796b-1ec4-4c0b-8171-d59a80145448-credential-socket\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:30:14.623271 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:14.623172 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97bb796b-1ec4-4c0b-8171-d59a80145448-workload-socket\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:30:14.623271 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:14.623189 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97bb796b-1ec4-4c0b-8171-d59a80145448-workload-certs\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:30:14.623271 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:14.623230 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97bb796b-1ec4-4c0b-8171-d59a80145448-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "97bb796b-1ec4-4c0b-8171-d59a80145448" (UID: "97bb796b-1ec4-4c0b-8171-d59a80145448"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:30:14.624019 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:14.623989 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97bb796b-1ec4-4c0b-8171-d59a80145448-istio-data" (OuterVolumeSpecName: "istio-data") pod "97bb796b-1ec4-4c0b-8171-d59a80145448" (UID: "97bb796b-1ec4-4c0b-8171-d59a80145448"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:30:14.624894 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:14.624875 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/97bb796b-1ec4-4c0b-8171-d59a80145448-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "97bb796b-1ec4-4c0b-8171-d59a80145448" (UID: "97bb796b-1ec4-4c0b-8171-d59a80145448"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 19:30:14.625106 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:14.625090 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97bb796b-1ec4-4c0b-8171-d59a80145448-istio-token" (OuterVolumeSpecName: "istio-token") pod "97bb796b-1ec4-4c0b-8171-d59a80145448" (UID: "97bb796b-1ec4-4c0b-8171-d59a80145448"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:30:14.625390 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:14.625375 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97bb796b-1ec4-4c0b-8171-d59a80145448-kube-api-access-4mrqr" (OuterVolumeSpecName: "kube-api-access-4mrqr") pod "97bb796b-1ec4-4c0b-8171-d59a80145448" (UID: "97bb796b-1ec4-4c0b-8171-d59a80145448"). InnerVolumeSpecName "kube-api-access-4mrqr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:30:14.625486 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:14.625422 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97bb796b-1ec4-4c0b-8171-d59a80145448-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "97bb796b-1ec4-4c0b-8171-d59a80145448" (UID: "97bb796b-1ec4-4c0b-8171-d59a80145448"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:30:14.724232 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:14.724199 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97bb796b-1ec4-4c0b-8171-d59a80145448-istio-podinfo\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:30:14.724232 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:14.724227 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4mrqr\" (UniqueName: \"kubernetes.io/projected/97bb796b-1ec4-4c0b-8171-d59a80145448-kube-api-access-4mrqr\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:30:14.724232 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:14.724237 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97bb796b-1ec4-4c0b-8171-d59a80145448-istio-data\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:30:14.724481 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:14.724245 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97bb796b-1ec4-4c0b-8171-d59a80145448-istio-envoy\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:30:14.724481 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:14.724252 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97bb796b-1ec4-4c0b-8171-d59a80145448-istio-token\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:30:14.724481 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:14.724262 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97bb796b-1ec4-4c0b-8171-d59a80145448-istiod-ca-cert\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:30:15.105637 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:15.105546 2583 generic.go:356] "Generic (PLEG): container finished" podID="97bb796b-1ec4-4c0b-8171-d59a80145448" containerID="501f602373d24165b9800054d843cdb2df0f20b04fd18be629e4d4618a129178" exitCode=0 Jul 08 19:30:15.105637 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:15.105599 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2" Jul 08 19:30:15.105637 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:15.105612 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2" event={"ID":"97bb796b-1ec4-4c0b-8171-d59a80145448","Type":"ContainerDied","Data":"501f602373d24165b9800054d843cdb2df0f20b04fd18be629e4d4618a129178"} Jul 08 19:30:15.105872 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:15.105657 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2" event={"ID":"97bb796b-1ec4-4c0b-8171-d59a80145448","Type":"ContainerDied","Data":"0ee9c0f54913f4a6e72e6e8b3dc6e06ccf3ce38e34f128f23e0b6dadad683f73"} Jul 08 19:30:15.105872 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:15.105678 2583 scope.go:117] "RemoveContainer" containerID="501f602373d24165b9800054d843cdb2df0f20b04fd18be629e4d4618a129178" Jul 08 19:30:15.114534 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:15.114515 2583 scope.go:117] "RemoveContainer" containerID="501f602373d24165b9800054d843cdb2df0f20b04fd18be629e4d4618a129178" Jul 08 19:30:15.114840 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:30:15.114821 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"501f602373d24165b9800054d843cdb2df0f20b04fd18be629e4d4618a129178\": container with ID starting with 501f602373d24165b9800054d843cdb2df0f20b04fd18be629e4d4618a129178 not found: ID does not exist" containerID="501f602373d24165b9800054d843cdb2df0f20b04fd18be629e4d4618a129178" Jul 08 19:30:15.114902 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:15.114852 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"501f602373d24165b9800054d843cdb2df0f20b04fd18be629e4d4618a129178"} err="failed to get container status \"501f602373d24165b9800054d843cdb2df0f20b04fd18be629e4d4618a129178\": rpc error: code = NotFound desc = could not find container \"501f602373d24165b9800054d843cdb2df0f20b04fd18be629e4d4618a129178\": container with ID starting with 501f602373d24165b9800054d843cdb2df0f20b04fd18be629e4d4618a129178 not found: ID does not exist" Jul 08 19:30:15.126456 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:15.126409 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2"] Jul 08 19:30:15.130757 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:15.130731 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-19b7c553-openshift-default-655f4945bf-g25g2"] Jul 08 19:30:15.174914 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:15.174881 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97bb796b-1ec4-4c0b-8171-d59a80145448" path="/var/lib/kubelet/pods/97bb796b-1ec4-4c0b-8171-d59a80145448/volumes" Jul 08 19:30:20.219068 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:20.219037 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm"] Jul 08 19:30:20.219478 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:20.219451 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91f99702-2cb6-4562-b21f-339de91ea64a" containerName="istio-proxy" Jul 08 19:30:20.219478 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:20.219469 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="91f99702-2cb6-4562-b21f-339de91ea64a" containerName="istio-proxy" Jul 08 19:30:20.219554 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:20.219480 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97bb796b-1ec4-4c0b-8171-d59a80145448" containerName="istio-proxy" Jul 08 19:30:20.219554 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:20.219485 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="97bb796b-1ec4-4c0b-8171-d59a80145448" containerName="istio-proxy" Jul 08 19:30:20.219554 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:20.219538 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="97bb796b-1ec4-4c0b-8171-d59a80145448" containerName="istio-proxy" Jul 08 19:30:20.219554 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:20.219553 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="91f99702-2cb6-4562-b21f-339de91ea64a" containerName="istio-proxy" Jul 08 19:30:20.223947 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:20.223927 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm" Jul 08 19:30:20.226637 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:20.226614 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-f7c8045c-openshift-default-dockercfg-ptdnq\"" Jul 08 19:30:20.234719 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:20.234694 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm"] Jul 08 19:30:20.268504 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:20.268465 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7ea8c93-5763-493b-a39a-122822ee3174-credential-socket\") pod \"e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm\" (UID: \"a7ea8c93-5763-493b-a39a-122822ee3174\") " pod="openshift-ingress/e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm" Jul 08 19:30:20.268504 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:20.268510 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7ea8c93-5763-493b-a39a-122822ee3174-istio-podinfo\") pod \"e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm\" (UID: \"a7ea8c93-5763-493b-a39a-122822ee3174\") " pod="openshift-ingress/e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm" Jul 08 19:30:20.268738 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:20.268565 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7ea8c93-5763-493b-a39a-122822ee3174-workload-socket\") pod \"e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm\" (UID: \"a7ea8c93-5763-493b-a39a-122822ee3174\") " pod="openshift-ingress/e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm" Jul 08 19:30:20.268738 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:20.268580 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7ea8c93-5763-493b-a39a-122822ee3174-istio-envoy\") pod \"e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm\" (UID: \"a7ea8c93-5763-493b-a39a-122822ee3174\") " pod="openshift-ingress/e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm" Jul 08 19:30:20.268738 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:20.268598 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7ea8c93-5763-493b-a39a-122822ee3174-istio-token\") pod \"e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm\" (UID: \"a7ea8c93-5763-493b-a39a-122822ee3174\") " pod="openshift-ingress/e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm" Jul 08 19:30:20.268738 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:20.268671 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7ea8c93-5763-493b-a39a-122822ee3174-workload-certs\") pod \"e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm\" (UID: \"a7ea8c93-5763-493b-a39a-122822ee3174\") " pod="openshift-ingress/e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm" Jul 08 19:30:20.268738 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:20.268689 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7ea8c93-5763-493b-a39a-122822ee3174-istio-data\") pod \"e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm\" (UID: \"a7ea8c93-5763-493b-a39a-122822ee3174\") " pod="openshift-ingress/e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm" Jul 08 19:30:20.268738 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:20.268708 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8j9f\" (UniqueName: \"kubernetes.io/projected/a7ea8c93-5763-493b-a39a-122822ee3174-kube-api-access-s8j9f\") pod \"e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm\" (UID: \"a7ea8c93-5763-493b-a39a-122822ee3174\") " pod="openshift-ingress/e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm" Jul 08 19:30:20.268939 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:20.268748 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7ea8c93-5763-493b-a39a-122822ee3174-istiod-ca-cert\") pod \"e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm\" (UID: \"a7ea8c93-5763-493b-a39a-122822ee3174\") " pod="openshift-ingress/e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm" Jul 08 19:30:20.369413 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:20.369380 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7ea8c93-5763-493b-a39a-122822ee3174-workload-socket\") pod \"e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm\" (UID: \"a7ea8c93-5763-493b-a39a-122822ee3174\") " pod="openshift-ingress/e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm" Jul 08 19:30:20.369413 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:20.369414 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7ea8c93-5763-493b-a39a-122822ee3174-istio-envoy\") pod \"e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm\" (UID: \"a7ea8c93-5763-493b-a39a-122822ee3174\") " pod="openshift-ingress/e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm" Jul 08 19:30:20.369675 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:20.369469 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7ea8c93-5763-493b-a39a-122822ee3174-istio-token\") pod \"e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm\" (UID: \"a7ea8c93-5763-493b-a39a-122822ee3174\") " pod="openshift-ingress/e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm" Jul 08 19:30:20.369675 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:20.369498 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7ea8c93-5763-493b-a39a-122822ee3174-workload-certs\") pod \"e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm\" (UID: \"a7ea8c93-5763-493b-a39a-122822ee3174\") " pod="openshift-ingress/e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm" Jul 08 19:30:20.369675 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:20.369524 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7ea8c93-5763-493b-a39a-122822ee3174-istio-data\") pod \"e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm\" (UID: \"a7ea8c93-5763-493b-a39a-122822ee3174\") " pod="openshift-ingress/e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm" Jul 08 19:30:20.369675 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:20.369556 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8j9f\" (UniqueName: \"kubernetes.io/projected/a7ea8c93-5763-493b-a39a-122822ee3174-kube-api-access-s8j9f\") pod \"e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm\" (UID: \"a7ea8c93-5763-493b-a39a-122822ee3174\") " pod="openshift-ingress/e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm" Jul 08 19:30:20.369675 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:20.369625 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7ea8c93-5763-493b-a39a-122822ee3174-istiod-ca-cert\") pod \"e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm\" (UID: \"a7ea8c93-5763-493b-a39a-122822ee3174\") " pod="openshift-ingress/e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm" Jul 08 19:30:20.369912 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:20.369694 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7ea8c93-5763-493b-a39a-122822ee3174-credential-socket\") pod \"e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm\" (UID: \"a7ea8c93-5763-493b-a39a-122822ee3174\") " pod="openshift-ingress/e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm" Jul 08 19:30:20.369912 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:20.369719 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7ea8c93-5763-493b-a39a-122822ee3174-istio-podinfo\") pod \"e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm\" (UID: \"a7ea8c93-5763-493b-a39a-122822ee3174\") " pod="openshift-ingress/e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm" Jul 08 19:30:20.369912 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:20.369831 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7ea8c93-5763-493b-a39a-122822ee3174-workload-socket\") pod \"e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm\" (UID: \"a7ea8c93-5763-493b-a39a-122822ee3174\") " pod="openshift-ingress/e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm" Jul 08 19:30:20.370043 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:20.369905 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7ea8c93-5763-493b-a39a-122822ee3174-workload-certs\") pod \"e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm\" (UID: \"a7ea8c93-5763-493b-a39a-122822ee3174\") " pod="openshift-ingress/e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm" Jul 08 19:30:20.370043 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:20.369936 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7ea8c93-5763-493b-a39a-122822ee3174-istio-data\") pod \"e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm\" (UID: \"a7ea8c93-5763-493b-a39a-122822ee3174\") " pod="openshift-ingress/e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm" Jul 08 19:30:20.370112 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:20.370051 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7ea8c93-5763-493b-a39a-122822ee3174-credential-socket\") pod \"e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm\" (UID: \"a7ea8c93-5763-493b-a39a-122822ee3174\") " pod="openshift-ingress/e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm" Jul 08 19:30:20.370376 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:20.370355 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7ea8c93-5763-493b-a39a-122822ee3174-istiod-ca-cert\") pod \"e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm\" (UID: \"a7ea8c93-5763-493b-a39a-122822ee3174\") " pod="openshift-ingress/e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm" Jul 08 19:30:20.372036 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:20.372014 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7ea8c93-5763-493b-a39a-122822ee3174-istio-envoy\") pod \"e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm\" (UID: \"a7ea8c93-5763-493b-a39a-122822ee3174\") " pod="openshift-ingress/e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm" Jul 08 19:30:20.372160 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:20.372143 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7ea8c93-5763-493b-a39a-122822ee3174-istio-podinfo\") pod \"e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm\" (UID: \"a7ea8c93-5763-493b-a39a-122822ee3174\") " pod="openshift-ingress/e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm" Jul 08 19:30:20.378334 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:20.378308 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8j9f\" (UniqueName: \"kubernetes.io/projected/a7ea8c93-5763-493b-a39a-122822ee3174-kube-api-access-s8j9f\") pod \"e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm\" (UID: \"a7ea8c93-5763-493b-a39a-122822ee3174\") " pod="openshift-ingress/e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm" Jul 08 19:30:20.378461 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:20.378340 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7ea8c93-5763-493b-a39a-122822ee3174-istio-token\") pod \"e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm\" (UID: \"a7ea8c93-5763-493b-a39a-122822ee3174\") " pod="openshift-ingress/e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm" Jul 08 19:30:20.537837 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:20.537749 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm" Jul 08 19:30:20.680638 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:20.680590 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm"] Jul 08 19:30:20.683003 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:30:20.682968 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7ea8c93_5763_493b_a39a_122822ee3174.slice/crio-a536e07b03f6b21541b50ed71eb24dae10952cff7022d2f4e7f079759dba6654 WatchSource:0}: Error finding container a536e07b03f6b21541b50ed71eb24dae10952cff7022d2f4e7f079759dba6654: Status 404 returned error can't find the container with id a536e07b03f6b21541b50ed71eb24dae10952cff7022d2f4e7f079759dba6654 Jul 08 19:30:20.685679 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:20.685646 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:30:20.685763 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:20.685717 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:30:20.685763 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:20.685746 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:30:21.134788 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:21.134755 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm" event={"ID":"a7ea8c93-5763-493b-a39a-122822ee3174","Type":"ContainerStarted","Data":"cb929d51d10738d0f0fc485b405cfa5b98d81e5fd252a8e3d64ca9e465e14832"} Jul 08 19:30:21.134997 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:21.134804 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm" event={"ID":"a7ea8c93-5763-493b-a39a-122822ee3174","Type":"ContainerStarted","Data":"a536e07b03f6b21541b50ed71eb24dae10952cff7022d2f4e7f079759dba6654"} Jul 08 19:30:21.159163 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:21.159111 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm" podStartSLOduration=1.159095065 podStartE2EDuration="1.159095065s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:30:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:30:21.157928256 +0000 UTC m=+1586.437015913" watchObservedRunningTime="2026-07-08 19:30:21.159095065 +0000 UTC m=+1586.438182685" Jul 08 19:30:21.538515 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:21.538477 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm" Jul 08 19:30:21.543680 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:21.543657 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm" Jul 08 19:30:22.139458 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:22.139406 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm" Jul 08 19:30:22.140340 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:22.140324 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm" Jul 08 19:30:25.977692 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:25.977649 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm"] Jul 08 19:30:25.978176 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:25.977899 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm" podUID="a7ea8c93-5763-493b-a39a-122822ee3174" containerName="istio-proxy" containerID="cri-o://cb929d51d10738d0f0fc485b405cfa5b98d81e5fd252a8e3d64ca9e465e14832" gracePeriod=30 Jul 08 19:30:31.436425 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:31.436397 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm" Jul 08 19:30:31.473073 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:31.473032 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7ea8c93-5763-493b-a39a-122822ee3174-istio-podinfo\") pod \"a7ea8c93-5763-493b-a39a-122822ee3174\" (UID: \"a7ea8c93-5763-493b-a39a-122822ee3174\") " Jul 08 19:30:31.473229 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:31.473080 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7ea8c93-5763-493b-a39a-122822ee3174-istio-data\") pod \"a7ea8c93-5763-493b-a39a-122822ee3174\" (UID: \"a7ea8c93-5763-493b-a39a-122822ee3174\") " Jul 08 19:30:31.473229 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:31.473191 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s8j9f\" (UniqueName: \"kubernetes.io/projected/a7ea8c93-5763-493b-a39a-122822ee3174-kube-api-access-s8j9f\") pod \"a7ea8c93-5763-493b-a39a-122822ee3174\" (UID: \"a7ea8c93-5763-493b-a39a-122822ee3174\") " Jul 08 19:30:31.473343 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:31.473234 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7ea8c93-5763-493b-a39a-122822ee3174-istiod-ca-cert\") pod \"a7ea8c93-5763-493b-a39a-122822ee3174\" (UID: \"a7ea8c93-5763-493b-a39a-122822ee3174\") " Jul 08 19:30:31.473343 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:31.473286 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7ea8c93-5763-493b-a39a-122822ee3174-istio-envoy\") pod \"a7ea8c93-5763-493b-a39a-122822ee3174\" (UID: \"a7ea8c93-5763-493b-a39a-122822ee3174\") " Jul 08 19:30:31.473461 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:31.473344 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7ea8c93-5763-493b-a39a-122822ee3174-workload-socket\") pod \"a7ea8c93-5763-493b-a39a-122822ee3174\" (UID: \"a7ea8c93-5763-493b-a39a-122822ee3174\") " Jul 08 19:30:31.473461 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:31.473370 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7ea8c93-5763-493b-a39a-122822ee3174-istio-token\") pod \"a7ea8c93-5763-493b-a39a-122822ee3174\" (UID: \"a7ea8c93-5763-493b-a39a-122822ee3174\") " Jul 08 19:30:31.473461 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:31.473423 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7ea8c93-5763-493b-a39a-122822ee3174-credential-socket\") pod \"a7ea8c93-5763-493b-a39a-122822ee3174\" (UID: \"a7ea8c93-5763-493b-a39a-122822ee3174\") " Jul 08 19:30:31.473642 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:31.473494 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7ea8c93-5763-493b-a39a-122822ee3174-workload-certs\") pod \"a7ea8c93-5763-493b-a39a-122822ee3174\" (UID: \"a7ea8c93-5763-493b-a39a-122822ee3174\") " Jul 08 19:30:31.473697 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:31.473628 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7ea8c93-5763-493b-a39a-122822ee3174-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a7ea8c93-5763-493b-a39a-122822ee3174" (UID: "a7ea8c93-5763-493b-a39a-122822ee3174"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:30:31.474208 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:31.473842 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7ea8c93-5763-493b-a39a-122822ee3174-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a7ea8c93-5763-493b-a39a-122822ee3174" (UID: "a7ea8c93-5763-493b-a39a-122822ee3174"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:30:31.474208 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:31.473905 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7ea8c93-5763-493b-a39a-122822ee3174-istiod-ca-cert\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:30:31.474208 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:31.474126 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7ea8c93-5763-493b-a39a-122822ee3174-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a7ea8c93-5763-493b-a39a-122822ee3174" (UID: "a7ea8c93-5763-493b-a39a-122822ee3174"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:30:31.474208 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:31.474138 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7ea8c93-5763-493b-a39a-122822ee3174-istio-data" (OuterVolumeSpecName: "istio-data") pod "a7ea8c93-5763-493b-a39a-122822ee3174" (UID: "a7ea8c93-5763-493b-a39a-122822ee3174"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:30:31.474685 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:31.474246 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7ea8c93-5763-493b-a39a-122822ee3174-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a7ea8c93-5763-493b-a39a-122822ee3174" (UID: "a7ea8c93-5763-493b-a39a-122822ee3174"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:30:31.475758 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:31.475729 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7ea8c93-5763-493b-a39a-122822ee3174-kube-api-access-s8j9f" (OuterVolumeSpecName: "kube-api-access-s8j9f") pod "a7ea8c93-5763-493b-a39a-122822ee3174" (UID: "a7ea8c93-5763-493b-a39a-122822ee3174"). InnerVolumeSpecName "kube-api-access-s8j9f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:30:31.476031 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:31.476003 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7ea8c93-5763-493b-a39a-122822ee3174-istio-token" (OuterVolumeSpecName: "istio-token") pod "a7ea8c93-5763-493b-a39a-122822ee3174" (UID: "a7ea8c93-5763-493b-a39a-122822ee3174"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:30:31.476162 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:31.476128 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a7ea8c93-5763-493b-a39a-122822ee3174-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a7ea8c93-5763-493b-a39a-122822ee3174" (UID: "a7ea8c93-5763-493b-a39a-122822ee3174"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 19:30:31.476162 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:31.476145 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7ea8c93-5763-493b-a39a-122822ee3174-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a7ea8c93-5763-493b-a39a-122822ee3174" (UID: "a7ea8c93-5763-493b-a39a-122822ee3174"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:30:31.574799 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:31.574710 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s8j9f\" (UniqueName: \"kubernetes.io/projected/a7ea8c93-5763-493b-a39a-122822ee3174-kube-api-access-s8j9f\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:30:31.574799 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:31.574739 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7ea8c93-5763-493b-a39a-122822ee3174-istio-envoy\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:30:31.574799 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:31.574749 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7ea8c93-5763-493b-a39a-122822ee3174-workload-socket\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:30:31.574799 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:31.574760 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7ea8c93-5763-493b-a39a-122822ee3174-istio-token\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:30:31.574799 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:31.574768 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7ea8c93-5763-493b-a39a-122822ee3174-credential-socket\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:30:31.574799 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:31.574776 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7ea8c93-5763-493b-a39a-122822ee3174-workload-certs\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:30:31.574799 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:31.574784 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7ea8c93-5763-493b-a39a-122822ee3174-istio-podinfo\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:30:31.574799 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:31.574790 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7ea8c93-5763-493b-a39a-122822ee3174-istio-data\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:30:32.199214 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:32.199180 2583 generic.go:356] "Generic (PLEG): container finished" podID="a7ea8c93-5763-493b-a39a-122822ee3174" containerID="cb929d51d10738d0f0fc485b405cfa5b98d81e5fd252a8e3d64ca9e465e14832" exitCode=0 Jul 08 19:30:32.199397 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:32.199231 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm" Jul 08 19:30:32.199397 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:32.199268 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm" event={"ID":"a7ea8c93-5763-493b-a39a-122822ee3174","Type":"ContainerDied","Data":"cb929d51d10738d0f0fc485b405cfa5b98d81e5fd252a8e3d64ca9e465e14832"} Jul 08 19:30:32.199397 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:32.199314 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm" event={"ID":"a7ea8c93-5763-493b-a39a-122822ee3174","Type":"ContainerDied","Data":"a536e07b03f6b21541b50ed71eb24dae10952cff7022d2f4e7f079759dba6654"} Jul 08 19:30:32.199397 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:32.199336 2583 scope.go:117] "RemoveContainer" containerID="cb929d51d10738d0f0fc485b405cfa5b98d81e5fd252a8e3d64ca9e465e14832" Jul 08 19:30:32.208344 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:32.208326 2583 scope.go:117] "RemoveContainer" containerID="cb929d51d10738d0f0fc485b405cfa5b98d81e5fd252a8e3d64ca9e465e14832" Jul 08 19:30:32.208637 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:30:32.208617 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb929d51d10738d0f0fc485b405cfa5b98d81e5fd252a8e3d64ca9e465e14832\": container with ID starting with cb929d51d10738d0f0fc485b405cfa5b98d81e5fd252a8e3d64ca9e465e14832 not found: ID does not exist" containerID="cb929d51d10738d0f0fc485b405cfa5b98d81e5fd252a8e3d64ca9e465e14832" Jul 08 19:30:32.208713 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:32.208645 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb929d51d10738d0f0fc485b405cfa5b98d81e5fd252a8e3d64ca9e465e14832"} err="failed to get container status \"cb929d51d10738d0f0fc485b405cfa5b98d81e5fd252a8e3d64ca9e465e14832\": rpc error: code = NotFound desc = could not find container \"cb929d51d10738d0f0fc485b405cfa5b98d81e5fd252a8e3d64ca9e465e14832\": container with ID starting with cb929d51d10738d0f0fc485b405cfa5b98d81e5fd252a8e3d64ca9e465e14832 not found: ID does not exist" Jul 08 19:30:32.220463 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:32.220421 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm"] Jul 08 19:30:32.223848 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:32.223830 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-f7c8045c-openshift-default-c5f8bf855-r4scm"] Jul 08 19:30:33.175251 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:30:33.175220 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7ea8c93-5763-493b-a39a-122822ee3174" path="/var/lib/kubelet/pods/a7ea8c93-5763-493b-a39a-122822ee3174/volumes" Jul 08 19:32:08.622629 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:08.622584 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh"] Jul 08 19:32:08.623186 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:08.623032 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7ea8c93-5763-493b-a39a-122822ee3174" containerName="istio-proxy" Jul 08 19:32:08.623186 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:08.623076 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7ea8c93-5763-493b-a39a-122822ee3174" containerName="istio-proxy" Jul 08 19:32:08.623304 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:08.623277 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7ea8c93-5763-493b-a39a-122822ee3174" containerName="istio-proxy" Jul 08 19:32:08.626539 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:08.626517 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh" Jul 08 19:32:08.629263 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:08.629242 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-94233cba-openshift-default-dockercfg-tlzvj\"" Jul 08 19:32:08.640506 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:08.640471 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh"] Jul 08 19:32:08.732722 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:08.732684 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68f5ba38-736f-486b-bbcc-ef2a97220906-credential-socket\") pod \"e2e-mt-94233cba-openshift-default-64b9c97775-z97nh\" (UID: \"68f5ba38-736f-486b-bbcc-ef2a97220906\") " pod="openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh" Jul 08 19:32:08.732942 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:08.732749 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68f5ba38-736f-486b-bbcc-ef2a97220906-istio-data\") pod \"e2e-mt-94233cba-openshift-default-64b9c97775-z97nh\" (UID: \"68f5ba38-736f-486b-bbcc-ef2a97220906\") " pod="openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh" Jul 08 19:32:08.732942 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:08.732797 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7h48\" (UniqueName: \"kubernetes.io/projected/68f5ba38-736f-486b-bbcc-ef2a97220906-kube-api-access-h7h48\") pod \"e2e-mt-94233cba-openshift-default-64b9c97775-z97nh\" (UID: \"68f5ba38-736f-486b-bbcc-ef2a97220906\") " pod="openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh" Jul 08 19:32:08.732942 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:08.732928 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68f5ba38-736f-486b-bbcc-ef2a97220906-istiod-ca-cert\") pod \"e2e-mt-94233cba-openshift-default-64b9c97775-z97nh\" (UID: \"68f5ba38-736f-486b-bbcc-ef2a97220906\") " pod="openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh" Jul 08 19:32:08.733107 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:08.733004 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68f5ba38-736f-486b-bbcc-ef2a97220906-workload-certs\") pod \"e2e-mt-94233cba-openshift-default-64b9c97775-z97nh\" (UID: \"68f5ba38-736f-486b-bbcc-ef2a97220906\") " pod="openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh" Jul 08 19:32:08.733107 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:08.733038 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68f5ba38-736f-486b-bbcc-ef2a97220906-workload-socket\") pod \"e2e-mt-94233cba-openshift-default-64b9c97775-z97nh\" (UID: \"68f5ba38-736f-486b-bbcc-ef2a97220906\") " pod="openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh" Jul 08 19:32:08.733107 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:08.733064 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68f5ba38-736f-486b-bbcc-ef2a97220906-istio-token\") pod \"e2e-mt-94233cba-openshift-default-64b9c97775-z97nh\" (UID: \"68f5ba38-736f-486b-bbcc-ef2a97220906\") " pod="openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh" Jul 08 19:32:08.733107 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:08.733099 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68f5ba38-736f-486b-bbcc-ef2a97220906-istio-envoy\") pod \"e2e-mt-94233cba-openshift-default-64b9c97775-z97nh\" (UID: \"68f5ba38-736f-486b-bbcc-ef2a97220906\") " pod="openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh" Jul 08 19:32:08.733289 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:08.733136 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68f5ba38-736f-486b-bbcc-ef2a97220906-istio-podinfo\") pod \"e2e-mt-94233cba-openshift-default-64b9c97775-z97nh\" (UID: \"68f5ba38-736f-486b-bbcc-ef2a97220906\") " pod="openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh" Jul 08 19:32:08.833841 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:08.833802 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68f5ba38-736f-486b-bbcc-ef2a97220906-istio-data\") pod \"e2e-mt-94233cba-openshift-default-64b9c97775-z97nh\" (UID: \"68f5ba38-736f-486b-bbcc-ef2a97220906\") " pod="openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh" Jul 08 19:32:08.833841 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:08.833844 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7h48\" (UniqueName: \"kubernetes.io/projected/68f5ba38-736f-486b-bbcc-ef2a97220906-kube-api-access-h7h48\") pod \"e2e-mt-94233cba-openshift-default-64b9c97775-z97nh\" (UID: \"68f5ba38-736f-486b-bbcc-ef2a97220906\") " pod="openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh" Jul 08 19:32:08.834107 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:08.833877 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68f5ba38-736f-486b-bbcc-ef2a97220906-istiod-ca-cert\") pod \"e2e-mt-94233cba-openshift-default-64b9c97775-z97nh\" (UID: \"68f5ba38-736f-486b-bbcc-ef2a97220906\") " pod="openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh" Jul 08 19:32:08.834107 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:08.833921 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68f5ba38-736f-486b-bbcc-ef2a97220906-workload-certs\") pod \"e2e-mt-94233cba-openshift-default-64b9c97775-z97nh\" (UID: \"68f5ba38-736f-486b-bbcc-ef2a97220906\") " pod="openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh" Jul 08 19:32:08.834107 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:08.833941 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68f5ba38-736f-486b-bbcc-ef2a97220906-workload-socket\") pod \"e2e-mt-94233cba-openshift-default-64b9c97775-z97nh\" (UID: \"68f5ba38-736f-486b-bbcc-ef2a97220906\") " pod="openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh" Jul 08 19:32:08.834107 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:08.833958 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68f5ba38-736f-486b-bbcc-ef2a97220906-istio-token\") pod \"e2e-mt-94233cba-openshift-default-64b9c97775-z97nh\" (UID: \"68f5ba38-736f-486b-bbcc-ef2a97220906\") " pod="openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh" Jul 08 19:32:08.834316 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:08.834122 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68f5ba38-736f-486b-bbcc-ef2a97220906-istio-envoy\") pod \"e2e-mt-94233cba-openshift-default-64b9c97775-z97nh\" (UID: \"68f5ba38-736f-486b-bbcc-ef2a97220906\") " pod="openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh" Jul 08 19:32:08.834316 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:08.834154 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68f5ba38-736f-486b-bbcc-ef2a97220906-istio-podinfo\") pod \"e2e-mt-94233cba-openshift-default-64b9c97775-z97nh\" (UID: \"68f5ba38-736f-486b-bbcc-ef2a97220906\") " pod="openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh" Jul 08 19:32:08.834316 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:08.834195 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68f5ba38-736f-486b-bbcc-ef2a97220906-credential-socket\") pod \"e2e-mt-94233cba-openshift-default-64b9c97775-z97nh\" (UID: \"68f5ba38-736f-486b-bbcc-ef2a97220906\") " pod="openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh" Jul 08 19:32:08.834498 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:08.834310 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68f5ba38-736f-486b-bbcc-ef2a97220906-istio-data\") pod \"e2e-mt-94233cba-openshift-default-64b9c97775-z97nh\" (UID: \"68f5ba38-736f-486b-bbcc-ef2a97220906\") " pod="openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh" Jul 08 19:32:08.834498 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:08.834342 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68f5ba38-736f-486b-bbcc-ef2a97220906-workload-certs\") pod \"e2e-mt-94233cba-openshift-default-64b9c97775-z97nh\" (UID: \"68f5ba38-736f-486b-bbcc-ef2a97220906\") " pod="openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh" Jul 08 19:32:08.834498 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:08.834392 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68f5ba38-736f-486b-bbcc-ef2a97220906-workload-socket\") pod \"e2e-mt-94233cba-openshift-default-64b9c97775-z97nh\" (UID: \"68f5ba38-736f-486b-bbcc-ef2a97220906\") " pod="openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh" Jul 08 19:32:08.834498 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:08.834497 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68f5ba38-736f-486b-bbcc-ef2a97220906-credential-socket\") pod \"e2e-mt-94233cba-openshift-default-64b9c97775-z97nh\" (UID: \"68f5ba38-736f-486b-bbcc-ef2a97220906\") " pod="openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh" Jul 08 19:32:08.834802 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:08.834772 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68f5ba38-736f-486b-bbcc-ef2a97220906-istiod-ca-cert\") pod \"e2e-mt-94233cba-openshift-default-64b9c97775-z97nh\" (UID: \"68f5ba38-736f-486b-bbcc-ef2a97220906\") " pod="openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh" Jul 08 19:32:08.836502 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:08.836465 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68f5ba38-736f-486b-bbcc-ef2a97220906-istio-envoy\") pod \"e2e-mt-94233cba-openshift-default-64b9c97775-z97nh\" (UID: \"68f5ba38-736f-486b-bbcc-ef2a97220906\") " pod="openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh" Jul 08 19:32:08.836833 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:08.836811 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68f5ba38-736f-486b-bbcc-ef2a97220906-istio-podinfo\") pod \"e2e-mt-94233cba-openshift-default-64b9c97775-z97nh\" (UID: \"68f5ba38-736f-486b-bbcc-ef2a97220906\") " pod="openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh" Jul 08 19:32:08.842083 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:08.842061 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7h48\" (UniqueName: \"kubernetes.io/projected/68f5ba38-736f-486b-bbcc-ef2a97220906-kube-api-access-h7h48\") pod \"e2e-mt-94233cba-openshift-default-64b9c97775-z97nh\" (UID: \"68f5ba38-736f-486b-bbcc-ef2a97220906\") " pod="openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh" Jul 08 19:32:08.842164 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:08.842059 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68f5ba38-736f-486b-bbcc-ef2a97220906-istio-token\") pod \"e2e-mt-94233cba-openshift-default-64b9c97775-z97nh\" (UID: \"68f5ba38-736f-486b-bbcc-ef2a97220906\") " pod="openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh" Jul 08 19:32:08.938964 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:08.938929 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh" Jul 08 19:32:09.088883 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:09.088858 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh"] Jul 08 19:32:09.090619 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:32:09.090591 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68f5ba38_736f_486b_bbcc_ef2a97220906.slice/crio-77d3f8f1edd70b7bb0c499ec2e6e5c70f471afbb77513d2f38058f7deef1124d WatchSource:0}: Error finding container 77d3f8f1edd70b7bb0c499ec2e6e5c70f471afbb77513d2f38058f7deef1124d: Status 404 returned error can't find the container with id 77d3f8f1edd70b7bb0c499ec2e6e5c70f471afbb77513d2f38058f7deef1124d Jul 08 19:32:09.092901 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:09.092860 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:32:09.093011 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:09.092949 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:32:09.093011 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:09.092990 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:32:09.634096 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:09.634059 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh" event={"ID":"68f5ba38-736f-486b-bbcc-ef2a97220906","Type":"ContainerStarted","Data":"225980ff9495b2390c8ab228f646a457a691bd8d5da8898aaddeb361e8ba8fc8"} Jul 08 19:32:09.634571 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:09.634128 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh" event={"ID":"68f5ba38-736f-486b-bbcc-ef2a97220906","Type":"ContainerStarted","Data":"77d3f8f1edd70b7bb0c499ec2e6e5c70f471afbb77513d2f38058f7deef1124d"} Jul 08 19:32:09.656880 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:09.656822 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh" podStartSLOduration=1.656804639 podStartE2EDuration="1.656804639s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:32:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:32:09.654107544 +0000 UTC m=+1694.933195175" watchObservedRunningTime="2026-07-08 19:32:09.656804639 +0000 UTC m=+1694.935892260" Jul 08 19:32:09.939529 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:09.939497 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh" Jul 08 19:32:10.944553 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:10.944522 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh" Jul 08 19:32:11.642501 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:11.642473 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh" Jul 08 19:32:11.643646 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:11.643624 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh" Jul 08 19:32:22.703029 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:22.702928 2583 generic.go:356] "Generic (PLEG): container finished" podID="68f5ba38-736f-486b-bbcc-ef2a97220906" containerID="225980ff9495b2390c8ab228f646a457a691bd8d5da8898aaddeb361e8ba8fc8" exitCode=0 Jul 08 19:32:22.703029 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:22.703005 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh" event={"ID":"68f5ba38-736f-486b-bbcc-ef2a97220906","Type":"ContainerDied","Data":"225980ff9495b2390c8ab228f646a457a691bd8d5da8898aaddeb361e8ba8fc8"} Jul 08 19:32:22.703580 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:22.703450 2583 scope.go:117] "RemoveContainer" containerID="225980ff9495b2390c8ab228f646a457a691bd8d5da8898aaddeb361e8ba8fc8" Jul 08 19:32:22.704476 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:22.704426 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:32:22.704534 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:22.704518 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:32:22.704568 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:22.704551 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:32:23.709922 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:23.709888 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh" event={"ID":"68f5ba38-736f-486b-bbcc-ef2a97220906","Type":"ContainerStarted","Data":"0c36e0504418358f9dc0d0f209e76530d9c4d4c22d40c7464fbac5059e5f2dbf"} Jul 08 19:32:23.939754 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:23.939713 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh" Jul 08 19:32:23.941013 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:23.940985 2583 patch_prober.go:28] interesting pod/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Jul 08 19:32:23.941154 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:23.941044 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh" podUID="68f5ba38-736f-486b-bbcc-ef2a97220906" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Jul 08 19:32:24.939796 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:24.939749 2583 patch_prober.go:28] interesting pod/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Jul 08 19:32:24.940261 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:24.939817 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh" podUID="68f5ba38-736f-486b-bbcc-ef2a97220906" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Jul 08 19:32:25.940409 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:25.940366 2583 patch_prober.go:28] interesting pod/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Jul 08 19:32:25.940850 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:25.940455 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh" podUID="68f5ba38-736f-486b-bbcc-ef2a97220906" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Jul 08 19:32:26.643329 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:26.643283 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh" Jul 08 19:32:26.940177 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:26.940139 2583 patch_prober.go:28] interesting pod/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Jul 08 19:32:26.940362 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:26.940196 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh" podUID="68f5ba38-736f-486b-bbcc-ef2a97220906" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Jul 08 19:32:27.939617 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:27.939577 2583 patch_prober.go:28] interesting pod/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Jul 08 19:32:27.940024 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:27.939636 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh" podUID="68f5ba38-736f-486b-bbcc-ef2a97220906" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Jul 08 19:32:28.943167 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:28.943134 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh" Jul 08 19:32:28.944115 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:28.944091 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh" Jul 08 19:32:30.412537 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:30.412487 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh"] Jul 08 19:32:30.742898 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:30.742816 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh" podUID="68f5ba38-736f-486b-bbcc-ef2a97220906" containerName="istio-proxy" containerID="cri-o://0c36e0504418358f9dc0d0f209e76530d9c4d4c22d40c7464fbac5059e5f2dbf" gracePeriod=30 Jul 08 19:32:35.999147 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:35.999121 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh" Jul 08 19:32:36.088497 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:36.088388 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68f5ba38-736f-486b-bbcc-ef2a97220906-credential-socket\") pod \"68f5ba38-736f-486b-bbcc-ef2a97220906\" (UID: \"68f5ba38-736f-486b-bbcc-ef2a97220906\") " Jul 08 19:32:36.088497 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:36.088450 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68f5ba38-736f-486b-bbcc-ef2a97220906-istio-token\") pod \"68f5ba38-736f-486b-bbcc-ef2a97220906\" (UID: \"68f5ba38-736f-486b-bbcc-ef2a97220906\") " Jul 08 19:32:36.088497 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:36.088470 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68f5ba38-736f-486b-bbcc-ef2a97220906-istio-podinfo\") pod \"68f5ba38-736f-486b-bbcc-ef2a97220906\" (UID: \"68f5ba38-736f-486b-bbcc-ef2a97220906\") " Jul 08 19:32:36.088497 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:36.088490 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68f5ba38-736f-486b-bbcc-ef2a97220906-istio-envoy\") pod \"68f5ba38-736f-486b-bbcc-ef2a97220906\" (UID: \"68f5ba38-736f-486b-bbcc-ef2a97220906\") " Jul 08 19:32:36.089042 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:36.088601 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68f5ba38-736f-486b-bbcc-ef2a97220906-istio-data\") pod \"68f5ba38-736f-486b-bbcc-ef2a97220906\" (UID: \"68f5ba38-736f-486b-bbcc-ef2a97220906\") " Jul 08 19:32:36.089042 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:36.088634 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68f5ba38-736f-486b-bbcc-ef2a97220906-workload-certs\") pod \"68f5ba38-736f-486b-bbcc-ef2a97220906\" (UID: \"68f5ba38-736f-486b-bbcc-ef2a97220906\") " Jul 08 19:32:36.089042 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:36.088660 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68f5ba38-736f-486b-bbcc-ef2a97220906-istiod-ca-cert\") pod \"68f5ba38-736f-486b-bbcc-ef2a97220906\" (UID: \"68f5ba38-736f-486b-bbcc-ef2a97220906\") " Jul 08 19:32:36.089042 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:36.088741 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68f5ba38-736f-486b-bbcc-ef2a97220906-workload-socket\") pod \"68f5ba38-736f-486b-bbcc-ef2a97220906\" (UID: \"68f5ba38-736f-486b-bbcc-ef2a97220906\") " Jul 08 19:32:36.089042 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:36.088777 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h7h48\" (UniqueName: \"kubernetes.io/projected/68f5ba38-736f-486b-bbcc-ef2a97220906-kube-api-access-h7h48\") pod \"68f5ba38-736f-486b-bbcc-ef2a97220906\" (UID: \"68f5ba38-736f-486b-bbcc-ef2a97220906\") " Jul 08 19:32:36.089042 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:36.088787 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68f5ba38-736f-486b-bbcc-ef2a97220906-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "68f5ba38-736f-486b-bbcc-ef2a97220906" (UID: "68f5ba38-736f-486b-bbcc-ef2a97220906"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:32:36.089042 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:36.088894 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68f5ba38-736f-486b-bbcc-ef2a97220906-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "68f5ba38-736f-486b-bbcc-ef2a97220906" (UID: "68f5ba38-736f-486b-bbcc-ef2a97220906"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:32:36.089419 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:36.089089 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68f5ba38-736f-486b-bbcc-ef2a97220906-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "68f5ba38-736f-486b-bbcc-ef2a97220906" (UID: "68f5ba38-736f-486b-bbcc-ef2a97220906"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:32:36.089419 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:36.089105 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68f5ba38-736f-486b-bbcc-ef2a97220906-credential-socket\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:32:36.089419 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:36.089125 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68f5ba38-736f-486b-bbcc-ef2a97220906-workload-certs\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:32:36.089419 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:36.089120 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68f5ba38-736f-486b-bbcc-ef2a97220906-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "68f5ba38-736f-486b-bbcc-ef2a97220906" (UID: "68f5ba38-736f-486b-bbcc-ef2a97220906"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:32:36.089791 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:36.089732 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68f5ba38-736f-486b-bbcc-ef2a97220906-istio-data" (OuterVolumeSpecName: "istio-data") pod "68f5ba38-736f-486b-bbcc-ef2a97220906" (UID: "68f5ba38-736f-486b-bbcc-ef2a97220906"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:32:36.090862 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:36.090833 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/68f5ba38-736f-486b-bbcc-ef2a97220906-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "68f5ba38-736f-486b-bbcc-ef2a97220906" (UID: "68f5ba38-736f-486b-bbcc-ef2a97220906"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 19:32:36.090954 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:36.090882 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68f5ba38-736f-486b-bbcc-ef2a97220906-istio-token" (OuterVolumeSpecName: "istio-token") pod "68f5ba38-736f-486b-bbcc-ef2a97220906" (UID: "68f5ba38-736f-486b-bbcc-ef2a97220906"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:32:36.090954 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:36.090888 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68f5ba38-736f-486b-bbcc-ef2a97220906-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "68f5ba38-736f-486b-bbcc-ef2a97220906" (UID: "68f5ba38-736f-486b-bbcc-ef2a97220906"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:32:36.091184 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:36.091170 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68f5ba38-736f-486b-bbcc-ef2a97220906-kube-api-access-h7h48" (OuterVolumeSpecName: "kube-api-access-h7h48") pod "68f5ba38-736f-486b-bbcc-ef2a97220906" (UID: "68f5ba38-736f-486b-bbcc-ef2a97220906"). InnerVolumeSpecName "kube-api-access-h7h48". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:32:36.190152 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:36.190116 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h7h48\" (UniqueName: \"kubernetes.io/projected/68f5ba38-736f-486b-bbcc-ef2a97220906-kube-api-access-h7h48\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:32:36.190152 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:36.190145 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68f5ba38-736f-486b-bbcc-ef2a97220906-istio-token\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:32:36.190152 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:36.190154 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68f5ba38-736f-486b-bbcc-ef2a97220906-istio-podinfo\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:32:36.190152 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:36.190162 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68f5ba38-736f-486b-bbcc-ef2a97220906-istio-envoy\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:32:36.190415 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:36.190170 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68f5ba38-736f-486b-bbcc-ef2a97220906-istio-data\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:32:36.190415 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:36.190178 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68f5ba38-736f-486b-bbcc-ef2a97220906-istiod-ca-cert\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:32:36.190415 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:36.190185 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68f5ba38-736f-486b-bbcc-ef2a97220906-workload-socket\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:32:36.774231 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:36.774197 2583 generic.go:356] "Generic (PLEG): container finished" podID="68f5ba38-736f-486b-bbcc-ef2a97220906" containerID="0c36e0504418358f9dc0d0f209e76530d9c4d4c22d40c7464fbac5059e5f2dbf" exitCode=0 Jul 08 19:32:36.774401 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:36.774247 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh" Jul 08 19:32:36.774401 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:36.774248 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh" event={"ID":"68f5ba38-736f-486b-bbcc-ef2a97220906","Type":"ContainerDied","Data":"0c36e0504418358f9dc0d0f209e76530d9c4d4c22d40c7464fbac5059e5f2dbf"} Jul 08 19:32:36.774401 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:36.774374 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh" event={"ID":"68f5ba38-736f-486b-bbcc-ef2a97220906","Type":"ContainerDied","Data":"77d3f8f1edd70b7bb0c499ec2e6e5c70f471afbb77513d2f38058f7deef1124d"} Jul 08 19:32:36.774548 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:36.774407 2583 scope.go:117] "RemoveContainer" containerID="0c36e0504418358f9dc0d0f209e76530d9c4d4c22d40c7464fbac5059e5f2dbf" Jul 08 19:32:36.784094 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:36.784079 2583 scope.go:117] "RemoveContainer" containerID="225980ff9495b2390c8ab228f646a457a691bd8d5da8898aaddeb361e8ba8fc8" Jul 08 19:32:36.792226 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:36.792206 2583 scope.go:117] "RemoveContainer" containerID="0c36e0504418358f9dc0d0f209e76530d9c4d4c22d40c7464fbac5059e5f2dbf" Jul 08 19:32:36.792533 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:32:36.792514 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c36e0504418358f9dc0d0f209e76530d9c4d4c22d40c7464fbac5059e5f2dbf\": container with ID starting with 0c36e0504418358f9dc0d0f209e76530d9c4d4c22d40c7464fbac5059e5f2dbf not found: ID does not exist" containerID="0c36e0504418358f9dc0d0f209e76530d9c4d4c22d40c7464fbac5059e5f2dbf" Jul 08 19:32:36.792582 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:36.792540 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c36e0504418358f9dc0d0f209e76530d9c4d4c22d40c7464fbac5059e5f2dbf"} err="failed to get container status \"0c36e0504418358f9dc0d0f209e76530d9c4d4c22d40c7464fbac5059e5f2dbf\": rpc error: code = NotFound desc = could not find container \"0c36e0504418358f9dc0d0f209e76530d9c4d4c22d40c7464fbac5059e5f2dbf\": container with ID starting with 0c36e0504418358f9dc0d0f209e76530d9c4d4c22d40c7464fbac5059e5f2dbf not found: ID does not exist" Jul 08 19:32:36.792582 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:36.792557 2583 scope.go:117] "RemoveContainer" containerID="225980ff9495b2390c8ab228f646a457a691bd8d5da8898aaddeb361e8ba8fc8" Jul 08 19:32:36.792753 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:32:36.792738 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"225980ff9495b2390c8ab228f646a457a691bd8d5da8898aaddeb361e8ba8fc8\": container with ID starting with 225980ff9495b2390c8ab228f646a457a691bd8d5da8898aaddeb361e8ba8fc8 not found: ID does not exist" containerID="225980ff9495b2390c8ab228f646a457a691bd8d5da8898aaddeb361e8ba8fc8" Jul 08 19:32:36.792790 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:36.792759 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"225980ff9495b2390c8ab228f646a457a691bd8d5da8898aaddeb361e8ba8fc8"} err="failed to get container status \"225980ff9495b2390c8ab228f646a457a691bd8d5da8898aaddeb361e8ba8fc8\": rpc error: code = NotFound desc = could not find container \"225980ff9495b2390c8ab228f646a457a691bd8d5da8898aaddeb361e8ba8fc8\": container with ID starting with 225980ff9495b2390c8ab228f646a457a691bd8d5da8898aaddeb361e8ba8fc8 not found: ID does not exist" Jul 08 19:32:36.801454 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:36.797600 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh"] Jul 08 19:32:36.804470 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:36.804428 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-94233cba-openshift-default-64b9c97775-z97nh"] Jul 08 19:32:37.174264 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:37.174220 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68f5ba38-736f-486b-bbcc-ef2a97220906" path="/var/lib/kubelet/pods/68f5ba38-736f-486b-bbcc-ef2a97220906/volumes" Jul 08 19:32:50.860042 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:50.860007 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc"] Jul 08 19:32:50.860629 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:50.860347 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68f5ba38-736f-486b-bbcc-ef2a97220906" containerName="istio-proxy" Jul 08 19:32:50.860629 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:50.860358 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="68f5ba38-736f-486b-bbcc-ef2a97220906" containerName="istio-proxy" Jul 08 19:32:50.860629 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:50.860373 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68f5ba38-736f-486b-bbcc-ef2a97220906" containerName="istio-proxy" Jul 08 19:32:50.860629 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:50.860378 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="68f5ba38-736f-486b-bbcc-ef2a97220906" containerName="istio-proxy" Jul 08 19:32:50.860629 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:50.860447 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="68f5ba38-736f-486b-bbcc-ef2a97220906" containerName="istio-proxy" Jul 08 19:32:50.864933 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:50.864911 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc" Jul 08 19:32:50.867464 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:50.867427 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-7c480b69-openshift-default-dockercfg-t4749\"" Jul 08 19:32:50.877509 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:50.877482 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc"] Jul 08 19:32:50.922293 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:50.922256 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3ea1410-d890-4baa-b5d8-4057d3f0b757-istio-data\") pod \"e2e-mt-7c480b69-openshift-default-89456895-7r8sc\" (UID: \"d3ea1410-d890-4baa-b5d8-4057d3f0b757\") " pod="openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc" Jul 08 19:32:50.922293 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:50.922297 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3ea1410-d890-4baa-b5d8-4057d3f0b757-istiod-ca-cert\") pod \"e2e-mt-7c480b69-openshift-default-89456895-7r8sc\" (UID: \"d3ea1410-d890-4baa-b5d8-4057d3f0b757\") " pod="openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc" Jul 08 19:32:50.922585 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:50.922327 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3ea1410-d890-4baa-b5d8-4057d3f0b757-workload-socket\") pod \"e2e-mt-7c480b69-openshift-default-89456895-7r8sc\" (UID: \"d3ea1410-d890-4baa-b5d8-4057d3f0b757\") " pod="openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc" Jul 08 19:32:50.922585 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:50.922383 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3ea1410-d890-4baa-b5d8-4057d3f0b757-credential-socket\") pod \"e2e-mt-7c480b69-openshift-default-89456895-7r8sc\" (UID: \"d3ea1410-d890-4baa-b5d8-4057d3f0b757\") " pod="openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc" Jul 08 19:32:50.922585 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:50.922473 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3ea1410-d890-4baa-b5d8-4057d3f0b757-istio-token\") pod \"e2e-mt-7c480b69-openshift-default-89456895-7r8sc\" (UID: \"d3ea1410-d890-4baa-b5d8-4057d3f0b757\") " pod="openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc" Jul 08 19:32:50.922585 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:50.922531 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7vx8\" (UniqueName: \"kubernetes.io/projected/d3ea1410-d890-4baa-b5d8-4057d3f0b757-kube-api-access-w7vx8\") pod \"e2e-mt-7c480b69-openshift-default-89456895-7r8sc\" (UID: \"d3ea1410-d890-4baa-b5d8-4057d3f0b757\") " pod="openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc" Jul 08 19:32:50.922585 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:50.922577 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3ea1410-d890-4baa-b5d8-4057d3f0b757-istio-envoy\") pod \"e2e-mt-7c480b69-openshift-default-89456895-7r8sc\" (UID: \"d3ea1410-d890-4baa-b5d8-4057d3f0b757\") " pod="openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc" Jul 08 19:32:50.922768 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:50.922603 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3ea1410-d890-4baa-b5d8-4057d3f0b757-workload-certs\") pod \"e2e-mt-7c480b69-openshift-default-89456895-7r8sc\" (UID: \"d3ea1410-d890-4baa-b5d8-4057d3f0b757\") " pod="openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc" Jul 08 19:32:50.922768 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:50.922618 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3ea1410-d890-4baa-b5d8-4057d3f0b757-istio-podinfo\") pod \"e2e-mt-7c480b69-openshift-default-89456895-7r8sc\" (UID: \"d3ea1410-d890-4baa-b5d8-4057d3f0b757\") " pod="openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc" Jul 08 19:32:51.023712 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:51.023654 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3ea1410-d890-4baa-b5d8-4057d3f0b757-istio-token\") pod \"e2e-mt-7c480b69-openshift-default-89456895-7r8sc\" (UID: \"d3ea1410-d890-4baa-b5d8-4057d3f0b757\") " pod="openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc" Jul 08 19:32:51.023916 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:51.023740 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7vx8\" (UniqueName: \"kubernetes.io/projected/d3ea1410-d890-4baa-b5d8-4057d3f0b757-kube-api-access-w7vx8\") pod \"e2e-mt-7c480b69-openshift-default-89456895-7r8sc\" (UID: \"d3ea1410-d890-4baa-b5d8-4057d3f0b757\") " pod="openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc" Jul 08 19:32:51.023916 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:51.023794 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3ea1410-d890-4baa-b5d8-4057d3f0b757-istio-envoy\") pod \"e2e-mt-7c480b69-openshift-default-89456895-7r8sc\" (UID: \"d3ea1410-d890-4baa-b5d8-4057d3f0b757\") " pod="openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc" Jul 08 19:32:51.023916 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:51.023829 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3ea1410-d890-4baa-b5d8-4057d3f0b757-workload-certs\") pod \"e2e-mt-7c480b69-openshift-default-89456895-7r8sc\" (UID: \"d3ea1410-d890-4baa-b5d8-4057d3f0b757\") " pod="openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc" Jul 08 19:32:51.023916 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:51.023854 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3ea1410-d890-4baa-b5d8-4057d3f0b757-istio-podinfo\") pod \"e2e-mt-7c480b69-openshift-default-89456895-7r8sc\" (UID: \"d3ea1410-d890-4baa-b5d8-4057d3f0b757\") " pod="openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc" Jul 08 19:32:51.023916 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:51.023902 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3ea1410-d890-4baa-b5d8-4057d3f0b757-istio-data\") pod \"e2e-mt-7c480b69-openshift-default-89456895-7r8sc\" (UID: \"d3ea1410-d890-4baa-b5d8-4057d3f0b757\") " pod="openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc" Jul 08 19:32:51.024174 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:51.023982 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3ea1410-d890-4baa-b5d8-4057d3f0b757-istiod-ca-cert\") pod \"e2e-mt-7c480b69-openshift-default-89456895-7r8sc\" (UID: \"d3ea1410-d890-4baa-b5d8-4057d3f0b757\") " pod="openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc" Jul 08 19:32:51.024174 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:51.024027 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3ea1410-d890-4baa-b5d8-4057d3f0b757-workload-socket\") pod \"e2e-mt-7c480b69-openshift-default-89456895-7r8sc\" (UID: \"d3ea1410-d890-4baa-b5d8-4057d3f0b757\") " pod="openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc" Jul 08 19:32:51.024174 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:51.024050 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3ea1410-d890-4baa-b5d8-4057d3f0b757-credential-socket\") pod \"e2e-mt-7c480b69-openshift-default-89456895-7r8sc\" (UID: \"d3ea1410-d890-4baa-b5d8-4057d3f0b757\") " pod="openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc" Jul 08 19:32:51.024328 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:51.024294 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3ea1410-d890-4baa-b5d8-4057d3f0b757-istio-data\") pod \"e2e-mt-7c480b69-openshift-default-89456895-7r8sc\" (UID: \"d3ea1410-d890-4baa-b5d8-4057d3f0b757\") " pod="openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc" Jul 08 19:32:51.024419 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:51.024395 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3ea1410-d890-4baa-b5d8-4057d3f0b757-workload-certs\") pod \"e2e-mt-7c480b69-openshift-default-89456895-7r8sc\" (UID: \"d3ea1410-d890-4baa-b5d8-4057d3f0b757\") " pod="openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc" Jul 08 19:32:51.024496 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:51.024459 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3ea1410-d890-4baa-b5d8-4057d3f0b757-credential-socket\") pod \"e2e-mt-7c480b69-openshift-default-89456895-7r8sc\" (UID: \"d3ea1410-d890-4baa-b5d8-4057d3f0b757\") " pod="openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc" Jul 08 19:32:51.024496 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:51.024463 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3ea1410-d890-4baa-b5d8-4057d3f0b757-workload-socket\") pod \"e2e-mt-7c480b69-openshift-default-89456895-7r8sc\" (UID: \"d3ea1410-d890-4baa-b5d8-4057d3f0b757\") " pod="openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc" Jul 08 19:32:51.024946 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:51.024929 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3ea1410-d890-4baa-b5d8-4057d3f0b757-istiod-ca-cert\") pod \"e2e-mt-7c480b69-openshift-default-89456895-7r8sc\" (UID: \"d3ea1410-d890-4baa-b5d8-4057d3f0b757\") " pod="openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc" Jul 08 19:32:51.026705 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:51.026680 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3ea1410-d890-4baa-b5d8-4057d3f0b757-istio-envoy\") pod \"e2e-mt-7c480b69-openshift-default-89456895-7r8sc\" (UID: \"d3ea1410-d890-4baa-b5d8-4057d3f0b757\") " pod="openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc" Jul 08 19:32:51.026797 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:51.026680 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3ea1410-d890-4baa-b5d8-4057d3f0b757-istio-podinfo\") pod \"e2e-mt-7c480b69-openshift-default-89456895-7r8sc\" (UID: \"d3ea1410-d890-4baa-b5d8-4057d3f0b757\") " pod="openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc" Jul 08 19:32:51.031932 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:51.031902 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3ea1410-d890-4baa-b5d8-4057d3f0b757-istio-token\") pod \"e2e-mt-7c480b69-openshift-default-89456895-7r8sc\" (UID: \"d3ea1410-d890-4baa-b5d8-4057d3f0b757\") " pod="openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc" Jul 08 19:32:51.032132 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:51.032110 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7vx8\" (UniqueName: \"kubernetes.io/projected/d3ea1410-d890-4baa-b5d8-4057d3f0b757-kube-api-access-w7vx8\") pod \"e2e-mt-7c480b69-openshift-default-89456895-7r8sc\" (UID: \"d3ea1410-d890-4baa-b5d8-4057d3f0b757\") " pod="openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc" Jul 08 19:32:51.175627 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:51.175591 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc" Jul 08 19:32:51.321803 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:51.321776 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc"] Jul 08 19:32:51.324314 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:32:51.324287 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3ea1410_d890_4baa_b5d8_4057d3f0b757.slice/crio-4dd5fe2636bb0ca60e800e1807b4ec2c3c8b44928dc4125dc7be3788366d4b9f WatchSource:0}: Error finding container 4dd5fe2636bb0ca60e800e1807b4ec2c3c8b44928dc4125dc7be3788366d4b9f: Status 404 returned error can't find the container with id 4dd5fe2636bb0ca60e800e1807b4ec2c3c8b44928dc4125dc7be3788366d4b9f Jul 08 19:32:51.326616 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:51.326571 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:32:51.326711 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:51.326648 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:32:51.326711 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:51.326682 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:32:51.846642 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:51.846607 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc" event={"ID":"d3ea1410-d890-4baa-b5d8-4057d3f0b757","Type":"ContainerStarted","Data":"f9a232c04352f3ae890a7a4157c50be097611004efd39936d744972059909d70"} Jul 08 19:32:51.846801 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:51.846674 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc" event={"ID":"d3ea1410-d890-4baa-b5d8-4057d3f0b757","Type":"ContainerStarted","Data":"4dd5fe2636bb0ca60e800e1807b4ec2c3c8b44928dc4125dc7be3788366d4b9f"} Jul 08 19:32:51.868358 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:51.868307 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc" podStartSLOduration=1.8682886189999999 podStartE2EDuration="1.868288619s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:32:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:32:51.865905253 +0000 UTC m=+1737.144992873" watchObservedRunningTime="2026-07-08 19:32:51.868288619 +0000 UTC m=+1737.147376239" Jul 08 19:32:52.176464 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:52.176398 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc" Jul 08 19:32:52.181730 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:52.181706 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc" Jul 08 19:32:52.853250 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:52.853220 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc" Jul 08 19:32:52.854235 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:32:52.854213 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc" Jul 08 19:33:07.854484 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:07.854357 2583 patch_prober.go:28] interesting pod/e2e-mt-7c480b69-openshift-default-89456895-7r8sc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 08 19:33:07.854484 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:07.854427 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc" podUID="d3ea1410-d890-4baa-b5d8-4057d3f0b757" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 08 19:33:07.925850 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:07.925815 2583 generic.go:356] "Generic (PLEG): container finished" podID="d3ea1410-d890-4baa-b5d8-4057d3f0b757" containerID="f9a232c04352f3ae890a7a4157c50be097611004efd39936d744972059909d70" exitCode=0 Jul 08 19:33:07.926019 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:07.925895 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc" event={"ID":"d3ea1410-d890-4baa-b5d8-4057d3f0b757","Type":"ContainerDied","Data":"f9a232c04352f3ae890a7a4157c50be097611004efd39936d744972059909d70"} Jul 08 19:33:07.926274 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:07.926262 2583 scope.go:117] "RemoveContainer" containerID="f9a232c04352f3ae890a7a4157c50be097611004efd39936d744972059909d70" Jul 08 19:33:07.927243 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:07.927218 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:33:07.927302 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:07.927289 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:33:07.927349 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:07.927322 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:33:08.933550 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:08.933520 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc" event={"ID":"d3ea1410-d890-4baa-b5d8-4057d3f0b757","Type":"ContainerStarted","Data":"b593a7ef682e6eb5e31fd7a6f29b6397b45e6ea68d0ed7b180520e9f348929cf"} Jul 08 19:33:09.177721 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:09.177679 2583 patch_prober.go:28] interesting pod/e2e-mt-7c480b69-openshift-default-89456895-7r8sc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 08 19:33:09.177943 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:09.177736 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc" podUID="d3ea1410-d890-4baa-b5d8-4057d3f0b757" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 08 19:33:09.178024 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:09.177975 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc" Jul 08 19:33:10.176789 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:10.176747 2583 patch_prober.go:28] interesting pod/e2e-mt-7c480b69-openshift-default-89456895-7r8sc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 08 19:33:10.177204 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:10.176819 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc" podUID="d3ea1410-d890-4baa-b5d8-4057d3f0b757" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 08 19:33:11.175922 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:11.175886 2583 patch_prober.go:28] interesting pod/e2e-mt-7c480b69-openshift-default-89456895-7r8sc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 08 19:33:11.176108 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:11.175939 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc" podUID="d3ea1410-d890-4baa-b5d8-4057d3f0b757" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 08 19:33:12.176070 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:12.176032 2583 patch_prober.go:28] interesting pod/e2e-mt-7c480b69-openshift-default-89456895-7r8sc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 08 19:33:12.176605 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:12.176095 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc" podUID="d3ea1410-d890-4baa-b5d8-4057d3f0b757" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 08 19:33:13.176637 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:13.176604 2583 patch_prober.go:28] interesting pod/e2e-mt-7c480b69-openshift-default-89456895-7r8sc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 08 19:33:13.177026 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:13.176658 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc" podUID="d3ea1410-d890-4baa-b5d8-4057d3f0b757" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 08 19:33:14.179849 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:14.179820 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc" Jul 08 19:33:14.180265 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:14.180151 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc" Jul 08 19:33:14.180979 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:14.180956 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc" Jul 08 19:33:14.605339 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:14.605261 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-7c480b69/e2e-lifecycle-model-7c480b69-kserve-776ccfd85f-nkgkt"] Jul 08 19:33:14.605933 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:14.605913 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="68f5ba38-736f-486b-bbcc-ef2a97220906" containerName="istio-proxy" Jul 08 19:33:14.609900 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:14.609880 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7c480b69/e2e-lifecycle-model-7c480b69-kserve-776ccfd85f-nkgkt" Jul 08 19:33:14.612373 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:14.612348 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-7c480b69\"/\"kube-root-ca.crt\"" Jul 08 19:33:14.613237 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:14.613218 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-7c480b69\"/\"e2e-lifecycle-model-7c480b69-kserve-self-signed-certs\"" Jul 08 19:33:14.613373 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:14.613221 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-7c480b69\"/\"default-dockercfg-qq2vk\"" Jul 08 19:33:14.613373 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:14.613259 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-7c480b69\"/\"openshift-service-ca.crt\"" Jul 08 19:33:14.616722 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:14.616693 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-7c480b69/e2e-lifecycle-model-7c480b69-kserve-776ccfd85f-nkgkt"] Jul 08 19:33:14.731261 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:14.731230 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1332a8e6-83a7-4e20-b432-677f31c1bb9e-model-cache\") pod \"e2e-lifecycle-model-7c480b69-kserve-776ccfd85f-nkgkt\" (UID: \"1332a8e6-83a7-4e20-b432-677f31c1bb9e\") " pod="ai-tenant-e2e-mt-7c480b69/e2e-lifecycle-model-7c480b69-kserve-776ccfd85f-nkgkt" Jul 08 19:33:14.731261 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:14.731268 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1332a8e6-83a7-4e20-b432-677f31c1bb9e-tmp-dir\") pod \"e2e-lifecycle-model-7c480b69-kserve-776ccfd85f-nkgkt\" (UID: \"1332a8e6-83a7-4e20-b432-677f31c1bb9e\") " pod="ai-tenant-e2e-mt-7c480b69/e2e-lifecycle-model-7c480b69-kserve-776ccfd85f-nkgkt" Jul 08 19:33:14.731599 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:14.731294 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1332a8e6-83a7-4e20-b432-677f31c1bb9e-tls-certs\") pod \"e2e-lifecycle-model-7c480b69-kserve-776ccfd85f-nkgkt\" (UID: \"1332a8e6-83a7-4e20-b432-677f31c1bb9e\") " pod="ai-tenant-e2e-mt-7c480b69/e2e-lifecycle-model-7c480b69-kserve-776ccfd85f-nkgkt" Jul 08 19:33:14.731599 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:14.731391 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxzwh\" (UniqueName: \"kubernetes.io/projected/1332a8e6-83a7-4e20-b432-677f31c1bb9e-kube-api-access-jxzwh\") pod \"e2e-lifecycle-model-7c480b69-kserve-776ccfd85f-nkgkt\" (UID: \"1332a8e6-83a7-4e20-b432-677f31c1bb9e\") " pod="ai-tenant-e2e-mt-7c480b69/e2e-lifecycle-model-7c480b69-kserve-776ccfd85f-nkgkt" Jul 08 19:33:14.731599 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:14.731473 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1332a8e6-83a7-4e20-b432-677f31c1bb9e-dshm\") pod \"e2e-lifecycle-model-7c480b69-kserve-776ccfd85f-nkgkt\" (UID: \"1332a8e6-83a7-4e20-b432-677f31c1bb9e\") " pod="ai-tenant-e2e-mt-7c480b69/e2e-lifecycle-model-7c480b69-kserve-776ccfd85f-nkgkt" Jul 08 19:33:14.731599 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:14.731534 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1332a8e6-83a7-4e20-b432-677f31c1bb9e-home\") pod \"e2e-lifecycle-model-7c480b69-kserve-776ccfd85f-nkgkt\" (UID: \"1332a8e6-83a7-4e20-b432-677f31c1bb9e\") " pod="ai-tenant-e2e-mt-7c480b69/e2e-lifecycle-model-7c480b69-kserve-776ccfd85f-nkgkt" Jul 08 19:33:14.832038 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:14.831984 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxzwh\" (UniqueName: \"kubernetes.io/projected/1332a8e6-83a7-4e20-b432-677f31c1bb9e-kube-api-access-jxzwh\") pod \"e2e-lifecycle-model-7c480b69-kserve-776ccfd85f-nkgkt\" (UID: \"1332a8e6-83a7-4e20-b432-677f31c1bb9e\") " pod="ai-tenant-e2e-mt-7c480b69/e2e-lifecycle-model-7c480b69-kserve-776ccfd85f-nkgkt" Jul 08 19:33:14.832247 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:14.832048 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1332a8e6-83a7-4e20-b432-677f31c1bb9e-dshm\") pod \"e2e-lifecycle-model-7c480b69-kserve-776ccfd85f-nkgkt\" (UID: \"1332a8e6-83a7-4e20-b432-677f31c1bb9e\") " pod="ai-tenant-e2e-mt-7c480b69/e2e-lifecycle-model-7c480b69-kserve-776ccfd85f-nkgkt" Jul 08 19:33:14.832247 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:14.832104 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1332a8e6-83a7-4e20-b432-677f31c1bb9e-home\") pod \"e2e-lifecycle-model-7c480b69-kserve-776ccfd85f-nkgkt\" (UID: \"1332a8e6-83a7-4e20-b432-677f31c1bb9e\") " pod="ai-tenant-e2e-mt-7c480b69/e2e-lifecycle-model-7c480b69-kserve-776ccfd85f-nkgkt" Jul 08 19:33:14.832247 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:14.832156 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1332a8e6-83a7-4e20-b432-677f31c1bb9e-model-cache\") pod \"e2e-lifecycle-model-7c480b69-kserve-776ccfd85f-nkgkt\" (UID: \"1332a8e6-83a7-4e20-b432-677f31c1bb9e\") " pod="ai-tenant-e2e-mt-7c480b69/e2e-lifecycle-model-7c480b69-kserve-776ccfd85f-nkgkt" Jul 08 19:33:14.832247 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:14.832189 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1332a8e6-83a7-4e20-b432-677f31c1bb9e-tmp-dir\") pod \"e2e-lifecycle-model-7c480b69-kserve-776ccfd85f-nkgkt\" (UID: \"1332a8e6-83a7-4e20-b432-677f31c1bb9e\") " pod="ai-tenant-e2e-mt-7c480b69/e2e-lifecycle-model-7c480b69-kserve-776ccfd85f-nkgkt" Jul 08 19:33:14.832247 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:14.832220 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1332a8e6-83a7-4e20-b432-677f31c1bb9e-tls-certs\") pod \"e2e-lifecycle-model-7c480b69-kserve-776ccfd85f-nkgkt\" (UID: \"1332a8e6-83a7-4e20-b432-677f31c1bb9e\") " pod="ai-tenant-e2e-mt-7c480b69/e2e-lifecycle-model-7c480b69-kserve-776ccfd85f-nkgkt" Jul 08 19:33:14.832610 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:14.832585 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1332a8e6-83a7-4e20-b432-677f31c1bb9e-model-cache\") pod \"e2e-lifecycle-model-7c480b69-kserve-776ccfd85f-nkgkt\" (UID: \"1332a8e6-83a7-4e20-b432-677f31c1bb9e\") " pod="ai-tenant-e2e-mt-7c480b69/e2e-lifecycle-model-7c480b69-kserve-776ccfd85f-nkgkt" Jul 08 19:33:14.832684 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:14.832622 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1332a8e6-83a7-4e20-b432-677f31c1bb9e-tmp-dir\") pod \"e2e-lifecycle-model-7c480b69-kserve-776ccfd85f-nkgkt\" (UID: \"1332a8e6-83a7-4e20-b432-677f31c1bb9e\") " pod="ai-tenant-e2e-mt-7c480b69/e2e-lifecycle-model-7c480b69-kserve-776ccfd85f-nkgkt" Jul 08 19:33:14.832684 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:14.832648 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1332a8e6-83a7-4e20-b432-677f31c1bb9e-home\") pod \"e2e-lifecycle-model-7c480b69-kserve-776ccfd85f-nkgkt\" (UID: \"1332a8e6-83a7-4e20-b432-677f31c1bb9e\") " pod="ai-tenant-e2e-mt-7c480b69/e2e-lifecycle-model-7c480b69-kserve-776ccfd85f-nkgkt" Jul 08 19:33:14.834568 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:14.834544 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1332a8e6-83a7-4e20-b432-677f31c1bb9e-dshm\") pod \"e2e-lifecycle-model-7c480b69-kserve-776ccfd85f-nkgkt\" (UID: \"1332a8e6-83a7-4e20-b432-677f31c1bb9e\") " pod="ai-tenant-e2e-mt-7c480b69/e2e-lifecycle-model-7c480b69-kserve-776ccfd85f-nkgkt" Jul 08 19:33:14.834762 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:14.834743 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1332a8e6-83a7-4e20-b432-677f31c1bb9e-tls-certs\") pod \"e2e-lifecycle-model-7c480b69-kserve-776ccfd85f-nkgkt\" (UID: \"1332a8e6-83a7-4e20-b432-677f31c1bb9e\") " pod="ai-tenant-e2e-mt-7c480b69/e2e-lifecycle-model-7c480b69-kserve-776ccfd85f-nkgkt" Jul 08 19:33:14.839876 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:14.839853 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxzwh\" (UniqueName: \"kubernetes.io/projected/1332a8e6-83a7-4e20-b432-677f31c1bb9e-kube-api-access-jxzwh\") pod \"e2e-lifecycle-model-7c480b69-kserve-776ccfd85f-nkgkt\" (UID: \"1332a8e6-83a7-4e20-b432-677f31c1bb9e\") " pod="ai-tenant-e2e-mt-7c480b69/e2e-lifecycle-model-7c480b69-kserve-776ccfd85f-nkgkt" Jul 08 19:33:14.920789 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:14.920759 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7c480b69/e2e-lifecycle-model-7c480b69-kserve-776ccfd85f-nkgkt" Jul 08 19:33:15.067007 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:15.066984 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-7c480b69/e2e-lifecycle-model-7c480b69-kserve-776ccfd85f-nkgkt"] Jul 08 19:33:15.069011 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:33:15.068981 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1332a8e6_83a7_4e20_b432_677f31c1bb9e.slice/crio-c79e7cb62da3a161c336378584d3da74add6f0f111a462b43b73c184f747401e WatchSource:0}: Error finding container c79e7cb62da3a161c336378584d3da74add6f0f111a462b43b73c184f747401e: Status 404 returned error can't find the container with id c79e7cb62da3a161c336378584d3da74add6f0f111a462b43b73c184f747401e Jul 08 19:33:15.979040 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:15.979001 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7c480b69/e2e-lifecycle-model-7c480b69-kserve-776ccfd85f-nkgkt" event={"ID":"1332a8e6-83a7-4e20-b432-677f31c1bb9e","Type":"ContainerStarted","Data":"2dd2f06baa34361d20cbf9d3452a229489f0795d6560780d4888efcaa04f0116"} Jul 08 19:33:15.979040 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:15.979046 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7c480b69/e2e-lifecycle-model-7c480b69-kserve-776ccfd85f-nkgkt" event={"ID":"1332a8e6-83a7-4e20-b432-677f31c1bb9e","Type":"ContainerStarted","Data":"c79e7cb62da3a161c336378584d3da74add6f0f111a462b43b73c184f747401e"} Jul 08 19:33:15.997764 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:15.997703 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-7c480b69/e2e-lifecycle-model-7c480b69-kserve-776ccfd85f-nkgkt" podStartSLOduration=1.997686387 podStartE2EDuration="1.997686387s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:33:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:33:15.995813402 +0000 UTC m=+1761.274901025" watchObservedRunningTime="2026-07-08 19:33:15.997686387 +0000 UTC m=+1761.276774007" Jul 08 19:33:24.921417 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:24.921374 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-7c480b69/e2e-lifecycle-model-7c480b69-kserve-776ccfd85f-nkgkt" Jul 08 19:33:24.921417 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:24.921426 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-7c480b69/e2e-lifecycle-model-7c480b69-kserve-776ccfd85f-nkgkt" Jul 08 19:33:24.934127 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:24.934096 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-7c480b69/e2e-lifecycle-model-7c480b69-kserve-776ccfd85f-nkgkt" Jul 08 19:33:25.031591 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:25.031563 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-7c480b69/e2e-lifecycle-model-7c480b69-kserve-776ccfd85f-nkgkt" Jul 08 19:33:46.879578 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:33:46.879502 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-7c480b69/e2e-lifecycle-model-7c480b69-kserve-776ccfd85f-nkgkt"] Jul 08 19:34:01.901098 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:01.901049 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-7c480b69/e2e-lifecycle-model-7c480b69-kserve-776ccfd85f-nkgkt" podUID="1332a8e6-83a7-4e20-b432-677f31c1bb9e" containerName="main" containerID="cri-o://2dd2f06baa34361d20cbf9d3452a229489f0795d6560780d4888efcaa04f0116" gracePeriod=45 Jul 08 19:34:02.161600 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:02.161576 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7c480b69/e2e-lifecycle-model-7c480b69-kserve-776ccfd85f-nkgkt" Jul 08 19:34:02.193621 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:02.193587 2583 generic.go:356] "Generic (PLEG): container finished" podID="1332a8e6-83a7-4e20-b432-677f31c1bb9e" containerID="2dd2f06baa34361d20cbf9d3452a229489f0795d6560780d4888efcaa04f0116" exitCode=0 Jul 08 19:34:02.193621 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:02.193617 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7c480b69/e2e-lifecycle-model-7c480b69-kserve-776ccfd85f-nkgkt" Jul 08 19:34:02.193822 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:02.193670 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7c480b69/e2e-lifecycle-model-7c480b69-kserve-776ccfd85f-nkgkt" event={"ID":"1332a8e6-83a7-4e20-b432-677f31c1bb9e","Type":"ContainerDied","Data":"2dd2f06baa34361d20cbf9d3452a229489f0795d6560780d4888efcaa04f0116"} Jul 08 19:34:02.193822 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:02.193717 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7c480b69/e2e-lifecycle-model-7c480b69-kserve-776ccfd85f-nkgkt" event={"ID":"1332a8e6-83a7-4e20-b432-677f31c1bb9e","Type":"ContainerDied","Data":"c79e7cb62da3a161c336378584d3da74add6f0f111a462b43b73c184f747401e"} Jul 08 19:34:02.193822 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:02.193744 2583 scope.go:117] "RemoveContainer" containerID="2dd2f06baa34361d20cbf9d3452a229489f0795d6560780d4888efcaa04f0116" Jul 08 19:34:02.202344 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:02.202322 2583 scope.go:117] "RemoveContainer" containerID="2dd2f06baa34361d20cbf9d3452a229489f0795d6560780d4888efcaa04f0116" Jul 08 19:34:02.202645 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:34:02.202627 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2dd2f06baa34361d20cbf9d3452a229489f0795d6560780d4888efcaa04f0116\": container with ID starting with 2dd2f06baa34361d20cbf9d3452a229489f0795d6560780d4888efcaa04f0116 not found: ID does not exist" containerID="2dd2f06baa34361d20cbf9d3452a229489f0795d6560780d4888efcaa04f0116" Jul 08 19:34:02.202729 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:02.202653 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2dd2f06baa34361d20cbf9d3452a229489f0795d6560780d4888efcaa04f0116"} err="failed to get container status \"2dd2f06baa34361d20cbf9d3452a229489f0795d6560780d4888efcaa04f0116\": rpc error: code = NotFound desc = could not find container \"2dd2f06baa34361d20cbf9d3452a229489f0795d6560780d4888efcaa04f0116\": container with ID starting with 2dd2f06baa34361d20cbf9d3452a229489f0795d6560780d4888efcaa04f0116 not found: ID does not exist" Jul 08 19:34:02.266170 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:02.266140 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1332a8e6-83a7-4e20-b432-677f31c1bb9e-home\") pod \"1332a8e6-83a7-4e20-b432-677f31c1bb9e\" (UID: \"1332a8e6-83a7-4e20-b432-677f31c1bb9e\") " Jul 08 19:34:02.266170 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:02.266177 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1332a8e6-83a7-4e20-b432-677f31c1bb9e-model-cache\") pod \"1332a8e6-83a7-4e20-b432-677f31c1bb9e\" (UID: \"1332a8e6-83a7-4e20-b432-677f31c1bb9e\") " Jul 08 19:34:02.266378 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:02.266266 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1332a8e6-83a7-4e20-b432-677f31c1bb9e-dshm\") pod \"1332a8e6-83a7-4e20-b432-677f31c1bb9e\" (UID: \"1332a8e6-83a7-4e20-b432-677f31c1bb9e\") " Jul 08 19:34:02.266378 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:02.266284 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1332a8e6-83a7-4e20-b432-677f31c1bb9e-tls-certs\") pod \"1332a8e6-83a7-4e20-b432-677f31c1bb9e\" (UID: \"1332a8e6-83a7-4e20-b432-677f31c1bb9e\") " Jul 08 19:34:02.266378 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:02.266308 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jxzwh\" (UniqueName: \"kubernetes.io/projected/1332a8e6-83a7-4e20-b432-677f31c1bb9e-kube-api-access-jxzwh\") pod \"1332a8e6-83a7-4e20-b432-677f31c1bb9e\" (UID: \"1332a8e6-83a7-4e20-b432-677f31c1bb9e\") " Jul 08 19:34:02.266378 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:02.266368 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1332a8e6-83a7-4e20-b432-677f31c1bb9e-tmp-dir\") pod \"1332a8e6-83a7-4e20-b432-677f31c1bb9e\" (UID: \"1332a8e6-83a7-4e20-b432-677f31c1bb9e\") " Jul 08 19:34:02.266568 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:02.266505 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1332a8e6-83a7-4e20-b432-677f31c1bb9e-home" (OuterVolumeSpecName: "home") pod "1332a8e6-83a7-4e20-b432-677f31c1bb9e" (UID: "1332a8e6-83a7-4e20-b432-677f31c1bb9e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:34:02.266713 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:02.266485 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1332a8e6-83a7-4e20-b432-677f31c1bb9e-model-cache" (OuterVolumeSpecName: "model-cache") pod "1332a8e6-83a7-4e20-b432-677f31c1bb9e" (UID: "1332a8e6-83a7-4e20-b432-677f31c1bb9e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:34:02.267101 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:02.267070 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1332a8e6-83a7-4e20-b432-677f31c1bb9e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1332a8e6-83a7-4e20-b432-677f31c1bb9e" (UID: "1332a8e6-83a7-4e20-b432-677f31c1bb9e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:34:02.267223 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:02.267168 2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1332a8e6-83a7-4e20-b432-677f31c1bb9e-tmp-dir\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:34:02.267223 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:02.267204 2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1332a8e6-83a7-4e20-b432-677f31c1bb9e-home\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:34:02.267223 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:02.267217 2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1332a8e6-83a7-4e20-b432-677f31c1bb9e-model-cache\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:34:02.275079 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:02.275047 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1332a8e6-83a7-4e20-b432-677f31c1bb9e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1332a8e6-83a7-4e20-b432-677f31c1bb9e" (UID: "1332a8e6-83a7-4e20-b432-677f31c1bb9e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:34:02.275647 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:02.275623 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1332a8e6-83a7-4e20-b432-677f31c1bb9e-dshm" (OuterVolumeSpecName: "dshm") pod "1332a8e6-83a7-4e20-b432-677f31c1bb9e" (UID: "1332a8e6-83a7-4e20-b432-677f31c1bb9e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:34:02.276229 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:02.276206 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1332a8e6-83a7-4e20-b432-677f31c1bb9e-kube-api-access-jxzwh" (OuterVolumeSpecName: "kube-api-access-jxzwh") pod "1332a8e6-83a7-4e20-b432-677f31c1bb9e" (UID: "1332a8e6-83a7-4e20-b432-677f31c1bb9e"). InnerVolumeSpecName "kube-api-access-jxzwh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:34:02.367844 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:02.367809 2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1332a8e6-83a7-4e20-b432-677f31c1bb9e-dshm\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:34:02.367844 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:02.367839 2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1332a8e6-83a7-4e20-b432-677f31c1bb9e-tls-certs\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:34:02.367844 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:02.367851 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jxzwh\" (UniqueName: \"kubernetes.io/projected/1332a8e6-83a7-4e20-b432-677f31c1bb9e-kube-api-access-jxzwh\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:34:02.514714 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:02.514680 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-7c480b69/e2e-lifecycle-model-7c480b69-kserve-776ccfd85f-nkgkt"] Jul 08 19:34:02.519476 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:02.519413 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-7c480b69/e2e-lifecycle-model-7c480b69-kserve-776ccfd85f-nkgkt"] Jul 08 19:34:03.174575 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:03.174540 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1332a8e6-83a7-4e20-b432-677f31c1bb9e" path="/var/lib/kubelet/pods/1332a8e6-83a7-4e20-b432-677f31c1bb9e/volumes" Jul 08 19:34:30.181817 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:30.181782 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc"] Jul 08 19:34:30.182393 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:30.182078 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc" podUID="d3ea1410-d890-4baa-b5d8-4057d3f0b757" containerName="istio-proxy" containerID="cri-o://b593a7ef682e6eb5e31fd7a6f29b6397b45e6ea68d0ed7b180520e9f348929cf" gracePeriod=30 Jul 08 19:34:35.351561 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:35.351306 2583 generic.go:356] "Generic (PLEG): container finished" podID="d3ea1410-d890-4baa-b5d8-4057d3f0b757" containerID="b593a7ef682e6eb5e31fd7a6f29b6397b45e6ea68d0ed7b180520e9f348929cf" exitCode=0 Jul 08 19:34:35.351561 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:35.351474 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc" event={"ID":"d3ea1410-d890-4baa-b5d8-4057d3f0b757","Type":"ContainerDied","Data":"b593a7ef682e6eb5e31fd7a6f29b6397b45e6ea68d0ed7b180520e9f348929cf"} Jul 08 19:34:35.351561 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:35.351539 2583 scope.go:117] "RemoveContainer" containerID="f9a232c04352f3ae890a7a4157c50be097611004efd39936d744972059909d70" Jul 08 19:34:35.430742 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:35.430720 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc" Jul 08 19:34:35.453699 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:35.452720 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3ea1410-d890-4baa-b5d8-4057d3f0b757-credential-socket\") pod \"d3ea1410-d890-4baa-b5d8-4057d3f0b757\" (UID: \"d3ea1410-d890-4baa-b5d8-4057d3f0b757\") " Jul 08 19:34:35.453699 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:35.452795 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3ea1410-d890-4baa-b5d8-4057d3f0b757-istio-data\") pod \"d3ea1410-d890-4baa-b5d8-4057d3f0b757\" (UID: \"d3ea1410-d890-4baa-b5d8-4057d3f0b757\") " Jul 08 19:34:35.453699 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:35.452834 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3ea1410-d890-4baa-b5d8-4057d3f0b757-istio-podinfo\") pod \"d3ea1410-d890-4baa-b5d8-4057d3f0b757\" (UID: \"d3ea1410-d890-4baa-b5d8-4057d3f0b757\") " Jul 08 19:34:35.453699 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:35.452901 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3ea1410-d890-4baa-b5d8-4057d3f0b757-workload-certs\") pod \"d3ea1410-d890-4baa-b5d8-4057d3f0b757\" (UID: \"d3ea1410-d890-4baa-b5d8-4057d3f0b757\") " Jul 08 19:34:35.453699 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:35.452934 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3ea1410-d890-4baa-b5d8-4057d3f0b757-istiod-ca-cert\") pod \"d3ea1410-d890-4baa-b5d8-4057d3f0b757\" (UID: \"d3ea1410-d890-4baa-b5d8-4057d3f0b757\") " Jul 08 19:34:35.453699 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:35.452970 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3ea1410-d890-4baa-b5d8-4057d3f0b757-istio-envoy\") pod \"d3ea1410-d890-4baa-b5d8-4057d3f0b757\" (UID: \"d3ea1410-d890-4baa-b5d8-4057d3f0b757\") " Jul 08 19:34:35.453699 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:35.453009 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3ea1410-d890-4baa-b5d8-4057d3f0b757-workload-socket\") pod \"d3ea1410-d890-4baa-b5d8-4057d3f0b757\" (UID: \"d3ea1410-d890-4baa-b5d8-4057d3f0b757\") " Jul 08 19:34:35.453699 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:35.453032 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w7vx8\" (UniqueName: \"kubernetes.io/projected/d3ea1410-d890-4baa-b5d8-4057d3f0b757-kube-api-access-w7vx8\") pod \"d3ea1410-d890-4baa-b5d8-4057d3f0b757\" (UID: \"d3ea1410-d890-4baa-b5d8-4057d3f0b757\") " Jul 08 19:34:35.453699 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:35.453061 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3ea1410-d890-4baa-b5d8-4057d3f0b757-istio-token\") pod \"d3ea1410-d890-4baa-b5d8-4057d3f0b757\" (UID: \"d3ea1410-d890-4baa-b5d8-4057d3f0b757\") " Jul 08 19:34:35.453699 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:35.453101 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3ea1410-d890-4baa-b5d8-4057d3f0b757-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d3ea1410-d890-4baa-b5d8-4057d3f0b757" (UID: "d3ea1410-d890-4baa-b5d8-4057d3f0b757"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:34:35.453699 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:35.453340 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3ea1410-d890-4baa-b5d8-4057d3f0b757-credential-socket\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:34:35.453699 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:35.453416 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3ea1410-d890-4baa-b5d8-4057d3f0b757-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d3ea1410-d890-4baa-b5d8-4057d3f0b757" (UID: "d3ea1410-d890-4baa-b5d8-4057d3f0b757"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:34:35.455767 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:35.455068 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3ea1410-d890-4baa-b5d8-4057d3f0b757-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d3ea1410-d890-4baa-b5d8-4057d3f0b757" (UID: "d3ea1410-d890-4baa-b5d8-4057d3f0b757"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:34:35.455999 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:35.455960 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3ea1410-d890-4baa-b5d8-4057d3f0b757-istio-token" (OuterVolumeSpecName: "istio-token") pod "d3ea1410-d890-4baa-b5d8-4057d3f0b757" (UID: "d3ea1410-d890-4baa-b5d8-4057d3f0b757"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:34:35.456150 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:35.456124 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3ea1410-d890-4baa-b5d8-4057d3f0b757-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d3ea1410-d890-4baa-b5d8-4057d3f0b757" (UID: "d3ea1410-d890-4baa-b5d8-4057d3f0b757"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:34:35.457018 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:35.456983 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3ea1410-d890-4baa-b5d8-4057d3f0b757-kube-api-access-w7vx8" (OuterVolumeSpecName: "kube-api-access-w7vx8") pod "d3ea1410-d890-4baa-b5d8-4057d3f0b757" (UID: "d3ea1410-d890-4baa-b5d8-4057d3f0b757"). InnerVolumeSpecName "kube-api-access-w7vx8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:34:35.457489 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:35.457452 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3ea1410-d890-4baa-b5d8-4057d3f0b757-istio-data" (OuterVolumeSpecName: "istio-data") pod "d3ea1410-d890-4baa-b5d8-4057d3f0b757" (UID: "d3ea1410-d890-4baa-b5d8-4057d3f0b757"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:34:35.458682 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:35.458649 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3ea1410-d890-4baa-b5d8-4057d3f0b757-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d3ea1410-d890-4baa-b5d8-4057d3f0b757" (UID: "d3ea1410-d890-4baa-b5d8-4057d3f0b757"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:34:35.458775 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:35.458758 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d3ea1410-d890-4baa-b5d8-4057d3f0b757-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d3ea1410-d890-4baa-b5d8-4057d3f0b757" (UID: "d3ea1410-d890-4baa-b5d8-4057d3f0b757"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 19:34:35.553825 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:35.553796 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3ea1410-d890-4baa-b5d8-4057d3f0b757-workload-certs\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:34:35.553825 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:35.553823 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3ea1410-d890-4baa-b5d8-4057d3f0b757-istiod-ca-cert\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:34:35.554147 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:35.553834 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3ea1410-d890-4baa-b5d8-4057d3f0b757-istio-envoy\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:34:35.554147 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:35.553845 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3ea1410-d890-4baa-b5d8-4057d3f0b757-workload-socket\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:34:35.554147 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:35.553861 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w7vx8\" (UniqueName: \"kubernetes.io/projected/d3ea1410-d890-4baa-b5d8-4057d3f0b757-kube-api-access-w7vx8\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:34:35.554147 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:35.553873 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3ea1410-d890-4baa-b5d8-4057d3f0b757-istio-token\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:34:35.554147 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:35.553884 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3ea1410-d890-4baa-b5d8-4057d3f0b757-istio-data\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:34:35.554147 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:35.553894 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3ea1410-d890-4baa-b5d8-4057d3f0b757-istio-podinfo\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:34:36.358661 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:36.358629 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc" Jul 08 19:34:36.359074 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:36.358636 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc" event={"ID":"d3ea1410-d890-4baa-b5d8-4057d3f0b757","Type":"ContainerDied","Data":"4dd5fe2636bb0ca60e800e1807b4ec2c3c8b44928dc4125dc7be3788366d4b9f"} Jul 08 19:34:36.359074 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:36.358761 2583 scope.go:117] "RemoveContainer" containerID="b593a7ef682e6eb5e31fd7a6f29b6397b45e6ea68d0ed7b180520e9f348929cf" Jul 08 19:34:36.381069 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:36.381047 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc"] Jul 08 19:34:36.387746 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:36.387726 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-7c480b69-openshift-default-89456895-7r8sc"] Jul 08 19:34:37.176024 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:34:37.175980 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3ea1410-d890-4baa-b5d8-4057d3f0b757" path="/var/lib/kubelet/pods/d3ea1410-d890-4baa-b5d8-4057d3f0b757/volumes" Jul 08 19:35:26.259757 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:26.259671 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p"] Jul 08 19:35:26.260384 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:26.260182 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3ea1410-d890-4baa-b5d8-4057d3f0b757" containerName="istio-proxy" Jul 08 19:35:26.260384 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:26.260201 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3ea1410-d890-4baa-b5d8-4057d3f0b757" containerName="istio-proxy" Jul 08 19:35:26.260384 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:26.260214 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1332a8e6-83a7-4e20-b432-677f31c1bb9e" containerName="main" Jul 08 19:35:26.260384 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:26.260223 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="1332a8e6-83a7-4e20-b432-677f31c1bb9e" containerName="main" Jul 08 19:35:26.260384 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:26.260251 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3ea1410-d890-4baa-b5d8-4057d3f0b757" containerName="istio-proxy" Jul 08 19:35:26.260384 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:26.260259 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3ea1410-d890-4baa-b5d8-4057d3f0b757" containerName="istio-proxy" Jul 08 19:35:26.260384 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:26.260351 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3ea1410-d890-4baa-b5d8-4057d3f0b757" containerName="istio-proxy" Jul 08 19:35:26.260384 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:26.260367 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="1332a8e6-83a7-4e20-b432-677f31c1bb9e" containerName="main" Jul 08 19:35:26.263352 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:26.263331 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p" Jul 08 19:35:26.265779 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:26.265760 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-8ab38c-openshift-default-dockercfg-p72sk\"" Jul 08 19:35:26.274799 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:26.274775 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p"] Jul 08 19:35:26.295064 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:26.295032 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/034f96e9-baa5-4c2e-b011-04791ce8d7ed-istiod-ca-cert\") pod \"e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p\" (UID: \"034f96e9-baa5-4c2e-b011-04791ce8d7ed\") " pod="openshift-ingress/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p" Jul 08 19:35:26.295206 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:26.295074 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/034f96e9-baa5-4c2e-b011-04791ce8d7ed-istio-token\") pod \"e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p\" (UID: \"034f96e9-baa5-4c2e-b011-04791ce8d7ed\") " pod="openshift-ingress/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p" Jul 08 19:35:26.295206 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:26.295151 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/034f96e9-baa5-4c2e-b011-04791ce8d7ed-istio-data\") pod \"e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p\" (UID: \"034f96e9-baa5-4c2e-b011-04791ce8d7ed\") " pod="openshift-ingress/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p" Jul 08 19:35:26.295278 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:26.295218 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/034f96e9-baa5-4c2e-b011-04791ce8d7ed-istio-podinfo\") pod \"e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p\" (UID: \"034f96e9-baa5-4c2e-b011-04791ce8d7ed\") " pod="openshift-ingress/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p" Jul 08 19:35:26.295278 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:26.295248 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/034f96e9-baa5-4c2e-b011-04791ce8d7ed-credential-socket\") pod \"e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p\" (UID: \"034f96e9-baa5-4c2e-b011-04791ce8d7ed\") " pod="openshift-ingress/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p" Jul 08 19:35:26.295346 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:26.295296 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgpg8\" (UniqueName: \"kubernetes.io/projected/034f96e9-baa5-4c2e-b011-04791ce8d7ed-kube-api-access-vgpg8\") pod \"e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p\" (UID: \"034f96e9-baa5-4c2e-b011-04791ce8d7ed\") " pod="openshift-ingress/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p" Jul 08 19:35:26.295346 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:26.295324 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/034f96e9-baa5-4c2e-b011-04791ce8d7ed-istio-envoy\") pod \"e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p\" (UID: \"034f96e9-baa5-4c2e-b011-04791ce8d7ed\") " pod="openshift-ingress/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p" Jul 08 19:35:26.295413 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:26.295361 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/034f96e9-baa5-4c2e-b011-04791ce8d7ed-workload-socket\") pod \"e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p\" (UID: \"034f96e9-baa5-4c2e-b011-04791ce8d7ed\") " pod="openshift-ingress/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p" Jul 08 19:35:26.295413 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:26.295400 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/034f96e9-baa5-4c2e-b011-04791ce8d7ed-workload-certs\") pod \"e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p\" (UID: \"034f96e9-baa5-4c2e-b011-04791ce8d7ed\") " pod="openshift-ingress/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p" Jul 08 19:35:26.396671 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:26.396635 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/034f96e9-baa5-4c2e-b011-04791ce8d7ed-istiod-ca-cert\") pod \"e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p\" (UID: \"034f96e9-baa5-4c2e-b011-04791ce8d7ed\") " pod="openshift-ingress/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p" Jul 08 19:35:26.396847 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:26.396678 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/034f96e9-baa5-4c2e-b011-04791ce8d7ed-istio-token\") pod \"e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p\" (UID: \"034f96e9-baa5-4c2e-b011-04791ce8d7ed\") " pod="openshift-ingress/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p" Jul 08 19:35:26.396847 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:26.396800 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/034f96e9-baa5-4c2e-b011-04791ce8d7ed-istio-data\") pod \"e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p\" (UID: \"034f96e9-baa5-4c2e-b011-04791ce8d7ed\") " pod="openshift-ingress/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p" Jul 08 19:35:26.396935 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:26.396853 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/034f96e9-baa5-4c2e-b011-04791ce8d7ed-istio-podinfo\") pod \"e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p\" (UID: \"034f96e9-baa5-4c2e-b011-04791ce8d7ed\") " pod="openshift-ingress/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p" Jul 08 19:35:26.396935 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:26.396875 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/034f96e9-baa5-4c2e-b011-04791ce8d7ed-credential-socket\") pod \"e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p\" (UID: \"034f96e9-baa5-4c2e-b011-04791ce8d7ed\") " pod="openshift-ingress/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p" Jul 08 19:35:26.396935 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:26.396915 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgpg8\" (UniqueName: \"kubernetes.io/projected/034f96e9-baa5-4c2e-b011-04791ce8d7ed-kube-api-access-vgpg8\") pod \"e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p\" (UID: \"034f96e9-baa5-4c2e-b011-04791ce8d7ed\") " pod="openshift-ingress/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p" Jul 08 19:35:26.397073 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:26.396944 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/034f96e9-baa5-4c2e-b011-04791ce8d7ed-istio-envoy\") pod \"e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p\" (UID: \"034f96e9-baa5-4c2e-b011-04791ce8d7ed\") " pod="openshift-ingress/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p" Jul 08 19:35:26.397073 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:26.396984 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/034f96e9-baa5-4c2e-b011-04791ce8d7ed-workload-socket\") pod \"e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p\" (UID: \"034f96e9-baa5-4c2e-b011-04791ce8d7ed\") " pod="openshift-ingress/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p" Jul 08 19:35:26.397073 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:26.397035 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/034f96e9-baa5-4c2e-b011-04791ce8d7ed-workload-certs\") pod \"e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p\" (UID: \"034f96e9-baa5-4c2e-b011-04791ce8d7ed\") " pod="openshift-ingress/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p" Jul 08 19:35:26.397229 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:26.397151 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/034f96e9-baa5-4c2e-b011-04791ce8d7ed-istio-data\") pod \"e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p\" (UID: \"034f96e9-baa5-4c2e-b011-04791ce8d7ed\") " pod="openshift-ingress/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p" Jul 08 19:35:26.397289 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:26.397247 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/034f96e9-baa5-4c2e-b011-04791ce8d7ed-credential-socket\") pod \"e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p\" (UID: \"034f96e9-baa5-4c2e-b011-04791ce8d7ed\") " pod="openshift-ingress/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p" Jul 08 19:35:26.397410 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:26.397380 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/034f96e9-baa5-4c2e-b011-04791ce8d7ed-workload-socket\") pod \"e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p\" (UID: \"034f96e9-baa5-4c2e-b011-04791ce8d7ed\") " pod="openshift-ingress/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p" Jul 08 19:35:26.397578 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:26.397490 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/034f96e9-baa5-4c2e-b011-04791ce8d7ed-istiod-ca-cert\") pod \"e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p\" (UID: \"034f96e9-baa5-4c2e-b011-04791ce8d7ed\") " pod="openshift-ingress/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p" Jul 08 19:35:26.397578 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:26.397570 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/034f96e9-baa5-4c2e-b011-04791ce8d7ed-workload-certs\") pod \"e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p\" (UID: \"034f96e9-baa5-4c2e-b011-04791ce8d7ed\") " pod="openshift-ingress/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p" Jul 08 19:35:26.399330 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:26.399309 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/034f96e9-baa5-4c2e-b011-04791ce8d7ed-istio-envoy\") pod \"e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p\" (UID: \"034f96e9-baa5-4c2e-b011-04791ce8d7ed\") " pod="openshift-ingress/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p" Jul 08 19:35:26.399503 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:26.399486 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/034f96e9-baa5-4c2e-b011-04791ce8d7ed-istio-podinfo\") pod \"e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p\" (UID: \"034f96e9-baa5-4c2e-b011-04791ce8d7ed\") " pod="openshift-ingress/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p" Jul 08 19:35:26.405867 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:26.405843 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/034f96e9-baa5-4c2e-b011-04791ce8d7ed-istio-token\") pod \"e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p\" (UID: \"034f96e9-baa5-4c2e-b011-04791ce8d7ed\") " pod="openshift-ingress/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p" Jul 08 19:35:26.406025 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:26.406006 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgpg8\" (UniqueName: \"kubernetes.io/projected/034f96e9-baa5-4c2e-b011-04791ce8d7ed-kube-api-access-vgpg8\") pod \"e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p\" (UID: \"034f96e9-baa5-4c2e-b011-04791ce8d7ed\") " pod="openshift-ingress/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p" Jul 08 19:35:26.575742 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:26.575656 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p" Jul 08 19:35:26.716113 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:26.716088 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p"] Jul 08 19:35:26.718769 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:35:26.718734 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod034f96e9_baa5_4c2e_b011_04791ce8d7ed.slice/crio-6f64474683f196199a54b8bf3e93a924715334e514b4c0ef4ef147098a630f95 WatchSource:0}: Error finding container 6f64474683f196199a54b8bf3e93a924715334e514b4c0ef4ef147098a630f95: Status 404 returned error can't find the container with id 6f64474683f196199a54b8bf3e93a924715334e514b4c0ef4ef147098a630f95 Jul 08 19:35:26.720957 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:26.720927 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:35:26.721050 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:26.720991 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:35:26.721050 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:26.721019 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:35:27.583914 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:27.583867 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p" event={"ID":"034f96e9-baa5-4c2e-b011-04791ce8d7ed","Type":"ContainerStarted","Data":"0208aebb8979c017b895d4ec078720efd61ca636bc623bd3e54ecd7e409661c0"} Jul 08 19:35:27.584399 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:27.583930 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p" event={"ID":"034f96e9-baa5-4c2e-b011-04791ce8d7ed","Type":"ContainerStarted","Data":"6f64474683f196199a54b8bf3e93a924715334e514b4c0ef4ef147098a630f95"} Jul 08 19:35:27.610625 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:27.610579 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p" podStartSLOduration=1.6105645709999998 podStartE2EDuration="1.610564571s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:35:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:35:27.60947743 +0000 UTC m=+1892.888565050" watchObservedRunningTime="2026-07-08 19:35:27.610564571 +0000 UTC m=+1892.889652190" Jul 08 19:35:28.575990 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:28.575947 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p" Jul 08 19:35:28.580812 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:28.580790 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p" Jul 08 19:35:28.588320 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:28.588294 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p" Jul 08 19:35:28.589211 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:28.589189 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p" Jul 08 19:35:34.793679 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:34.793645 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8"] Jul 08 19:35:34.794104 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:34.794065 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3ea1410-d890-4baa-b5d8-4057d3f0b757" containerName="istio-proxy" Jul 08 19:35:34.797302 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:34.797278 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8" Jul 08 19:35:34.800534 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:34.800509 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-1709ef-openshift-default-dockercfg-lmmjd\"" Jul 08 19:35:34.815202 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:34.815177 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8"] Jul 08 19:35:34.877397 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:34.876962 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b57a2e6a-e2f3-422b-8fb7-c4c6f409845c-workload-certs\") pod \"e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8\" (UID: \"b57a2e6a-e2f3-422b-8fb7-c4c6f409845c\") " pod="openshift-ingress/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8" Jul 08 19:35:34.877397 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:34.877020 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b57a2e6a-e2f3-422b-8fb7-c4c6f409845c-istiod-ca-cert\") pod \"e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8\" (UID: \"b57a2e6a-e2f3-422b-8fb7-c4c6f409845c\") " pod="openshift-ingress/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8" Jul 08 19:35:34.877397 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:34.877066 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b57a2e6a-e2f3-422b-8fb7-c4c6f409845c-credential-socket\") pod \"e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8\" (UID: \"b57a2e6a-e2f3-422b-8fb7-c4c6f409845c\") " pod="openshift-ingress/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8" Jul 08 19:35:34.877397 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:34.877096 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b57a2e6a-e2f3-422b-8fb7-c4c6f409845c-istio-envoy\") pod \"e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8\" (UID: \"b57a2e6a-e2f3-422b-8fb7-c4c6f409845c\") " pod="openshift-ingress/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8" Jul 08 19:35:34.877397 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:34.877142 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b57a2e6a-e2f3-422b-8fb7-c4c6f409845c-istio-token\") pod \"e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8\" (UID: \"b57a2e6a-e2f3-422b-8fb7-c4c6f409845c\") " pod="openshift-ingress/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8" Jul 08 19:35:34.877397 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:34.877191 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b57a2e6a-e2f3-422b-8fb7-c4c6f409845c-workload-socket\") pod \"e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8\" (UID: \"b57a2e6a-e2f3-422b-8fb7-c4c6f409845c\") " pod="openshift-ingress/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8" Jul 08 19:35:34.877397 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:34.877219 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b57a2e6a-e2f3-422b-8fb7-c4c6f409845c-istio-data\") pod \"e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8\" (UID: \"b57a2e6a-e2f3-422b-8fb7-c4c6f409845c\") " pod="openshift-ingress/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8" Jul 08 19:35:34.877397 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:34.877243 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j25pg\" (UniqueName: \"kubernetes.io/projected/b57a2e6a-e2f3-422b-8fb7-c4c6f409845c-kube-api-access-j25pg\") pod \"e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8\" (UID: \"b57a2e6a-e2f3-422b-8fb7-c4c6f409845c\") " pod="openshift-ingress/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8" Jul 08 19:35:34.877397 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:34.877273 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b57a2e6a-e2f3-422b-8fb7-c4c6f409845c-istio-podinfo\") pod \"e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8\" (UID: \"b57a2e6a-e2f3-422b-8fb7-c4c6f409845c\") " pod="openshift-ingress/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8" Jul 08 19:35:34.978230 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:34.978187 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b57a2e6a-e2f3-422b-8fb7-c4c6f409845c-credential-socket\") pod \"e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8\" (UID: \"b57a2e6a-e2f3-422b-8fb7-c4c6f409845c\") " pod="openshift-ingress/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8" Jul 08 19:35:34.978422 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:34.978238 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b57a2e6a-e2f3-422b-8fb7-c4c6f409845c-istio-envoy\") pod \"e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8\" (UID: \"b57a2e6a-e2f3-422b-8fb7-c4c6f409845c\") " pod="openshift-ingress/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8" Jul 08 19:35:34.978422 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:34.978288 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b57a2e6a-e2f3-422b-8fb7-c4c6f409845c-istio-token\") pod \"e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8\" (UID: \"b57a2e6a-e2f3-422b-8fb7-c4c6f409845c\") " pod="openshift-ingress/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8" Jul 08 19:35:34.978422 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:34.978333 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b57a2e6a-e2f3-422b-8fb7-c4c6f409845c-workload-socket\") pod \"e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8\" (UID: \"b57a2e6a-e2f3-422b-8fb7-c4c6f409845c\") " pod="openshift-ingress/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8" Jul 08 19:35:34.978422 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:34.978359 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b57a2e6a-e2f3-422b-8fb7-c4c6f409845c-istio-data\") pod \"e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8\" (UID: \"b57a2e6a-e2f3-422b-8fb7-c4c6f409845c\") " pod="openshift-ingress/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8" Jul 08 19:35:34.978422 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:34.978386 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j25pg\" (UniqueName: \"kubernetes.io/projected/b57a2e6a-e2f3-422b-8fb7-c4c6f409845c-kube-api-access-j25pg\") pod \"e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8\" (UID: \"b57a2e6a-e2f3-422b-8fb7-c4c6f409845c\") " pod="openshift-ingress/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8" Jul 08 19:35:34.978422 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:34.978413 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b57a2e6a-e2f3-422b-8fb7-c4c6f409845c-istio-podinfo\") pod \"e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8\" (UID: \"b57a2e6a-e2f3-422b-8fb7-c4c6f409845c\") " pod="openshift-ingress/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8" Jul 08 19:35:34.978784 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:34.978518 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b57a2e6a-e2f3-422b-8fb7-c4c6f409845c-workload-certs\") pod \"e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8\" (UID: \"b57a2e6a-e2f3-422b-8fb7-c4c6f409845c\") " pod="openshift-ingress/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8" Jul 08 19:35:34.978784 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:34.978553 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b57a2e6a-e2f3-422b-8fb7-c4c6f409845c-istiod-ca-cert\") pod \"e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8\" (UID: \"b57a2e6a-e2f3-422b-8fb7-c4c6f409845c\") " pod="openshift-ingress/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8" Jul 08 19:35:34.978784 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:34.978694 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b57a2e6a-e2f3-422b-8fb7-c4c6f409845c-credential-socket\") pod \"e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8\" (UID: \"b57a2e6a-e2f3-422b-8fb7-c4c6f409845c\") " pod="openshift-ingress/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8" Jul 08 19:35:34.978784 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:34.978724 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b57a2e6a-e2f3-422b-8fb7-c4c6f409845c-workload-socket\") pod \"e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8\" (UID: \"b57a2e6a-e2f3-422b-8fb7-c4c6f409845c\") " pod="openshift-ingress/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8" Jul 08 19:35:34.979048 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:34.979028 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b57a2e6a-e2f3-422b-8fb7-c4c6f409845c-istio-data\") pod \"e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8\" (UID: \"b57a2e6a-e2f3-422b-8fb7-c4c6f409845c\") " pod="openshift-ingress/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8" Jul 08 19:35:34.979199 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:34.979064 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b57a2e6a-e2f3-422b-8fb7-c4c6f409845c-workload-certs\") pod \"e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8\" (UID: \"b57a2e6a-e2f3-422b-8fb7-c4c6f409845c\") " pod="openshift-ingress/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8" Jul 08 19:35:34.979303 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:34.979284 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b57a2e6a-e2f3-422b-8fb7-c4c6f409845c-istiod-ca-cert\") pod \"e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8\" (UID: \"b57a2e6a-e2f3-422b-8fb7-c4c6f409845c\") " pod="openshift-ingress/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8" Jul 08 19:35:34.980786 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:34.980758 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b57a2e6a-e2f3-422b-8fb7-c4c6f409845c-istio-envoy\") pod \"e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8\" (UID: \"b57a2e6a-e2f3-422b-8fb7-c4c6f409845c\") " pod="openshift-ingress/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8" Jul 08 19:35:34.981642 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:34.981621 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b57a2e6a-e2f3-422b-8fb7-c4c6f409845c-istio-podinfo\") pod \"e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8\" (UID: \"b57a2e6a-e2f3-422b-8fb7-c4c6f409845c\") " pod="openshift-ingress/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8" Jul 08 19:35:34.987145 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:34.987124 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j25pg\" (UniqueName: \"kubernetes.io/projected/b57a2e6a-e2f3-422b-8fb7-c4c6f409845c-kube-api-access-j25pg\") pod \"e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8\" (UID: \"b57a2e6a-e2f3-422b-8fb7-c4c6f409845c\") " pod="openshift-ingress/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8" Jul 08 19:35:34.987286 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:34.987266 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b57a2e6a-e2f3-422b-8fb7-c4c6f409845c-istio-token\") pod \"e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8\" (UID: \"b57a2e6a-e2f3-422b-8fb7-c4c6f409845c\") " pod="openshift-ingress/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8" Jul 08 19:35:35.107737 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:35.107647 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8" Jul 08 19:35:35.307531 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:35.307504 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8"] Jul 08 19:35:35.309012 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:35:35.308981 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb57a2e6a_e2f3_422b_8fb7_c4c6f409845c.slice/crio-4639402140e8eee8d81b0194893bfdaa3fc8721cc883d20567c9c3f01cdb69f9 WatchSource:0}: Error finding container 4639402140e8eee8d81b0194893bfdaa3fc8721cc883d20567c9c3f01cdb69f9: Status 404 returned error can't find the container with id 4639402140e8eee8d81b0194893bfdaa3fc8721cc883d20567c9c3f01cdb69f9 Jul 08 19:35:35.311180 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:35.311150 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:35:35.311267 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:35.311212 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:35:35.311267 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:35.311241 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:35:35.621666 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:35.621628 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8" event={"ID":"b57a2e6a-e2f3-422b-8fb7-c4c6f409845c","Type":"ContainerStarted","Data":"321ce26d41120044e4281e4e86a1edb2015ebef1eb8b1ac73cd869a12f049e5e"} Jul 08 19:35:35.621855 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:35.621690 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8" event={"ID":"b57a2e6a-e2f3-422b-8fb7-c4c6f409845c","Type":"ContainerStarted","Data":"4639402140e8eee8d81b0194893bfdaa3fc8721cc883d20567c9c3f01cdb69f9"} Jul 08 19:35:35.650837 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:35.650717 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8" podStartSLOduration=1.6506799970000001 podStartE2EDuration="1.650679997s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:35:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:35:35.646581929 +0000 UTC m=+1900.925669550" watchObservedRunningTime="2026-07-08 19:35:35.650679997 +0000 UTC m=+1900.929767617" Jul 08 19:35:36.108078 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:36.108041 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8" Jul 08 19:35:36.113360 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:36.113335 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8" Jul 08 19:35:36.626596 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:36.626561 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8" Jul 08 19:35:36.627692 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:36.627672 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8" Jul 08 19:35:41.652043 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:41.651926 2583 generic.go:356] "Generic (PLEG): container finished" podID="034f96e9-baa5-4c2e-b011-04791ce8d7ed" containerID="0208aebb8979c017b895d4ec078720efd61ca636bc623bd3e54ecd7e409661c0" exitCode=0 Jul 08 19:35:41.652043 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:41.651979 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p" event={"ID":"034f96e9-baa5-4c2e-b011-04791ce8d7ed","Type":"ContainerDied","Data":"0208aebb8979c017b895d4ec078720efd61ca636bc623bd3e54ecd7e409661c0"} Jul 08 19:35:41.652644 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:41.652409 2583 scope.go:117] "RemoveContainer" containerID="0208aebb8979c017b895d4ec078720efd61ca636bc623bd3e54ecd7e409661c0" Jul 08 19:35:41.653542 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:41.653506 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:35:41.653649 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:41.653602 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:35:41.653649 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:41.653642 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:35:42.661725 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:42.661671 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p" event={"ID":"034f96e9-baa5-4c2e-b011-04791ce8d7ed","Type":"ContainerStarted","Data":"f67d2e52c6bb3d757978ab7ae7201199dbd517c905981f9af546c85abc35ea48"} Jul 08 19:35:43.576791 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:43.576690 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p" Jul 08 19:35:43.578060 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:43.578023 2583 patch_prober.go:28] interesting pod/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 08 19:35:43.578210 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:43.578124 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p" podUID="034f96e9-baa5-4c2e-b011-04791ce8d7ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 08 19:35:43.589136 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:43.589045 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p" Jul 08 19:35:44.576967 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:44.576923 2583 patch_prober.go:28] interesting pod/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 08 19:35:44.577491 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:44.576996 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p" podUID="034f96e9-baa5-4c2e-b011-04791ce8d7ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 08 19:35:45.578461 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:45.577943 2583 patch_prober.go:28] interesting pod/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 08 19:35:45.578461 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:45.578015 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p" podUID="034f96e9-baa5-4c2e-b011-04791ce8d7ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 08 19:35:46.580915 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:46.580885 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p" Jul 08 19:35:46.582029 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:46.582008 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p" Jul 08 19:35:49.703248 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:49.703217 2583 generic.go:356] "Generic (PLEG): container finished" podID="b57a2e6a-e2f3-422b-8fb7-c4c6f409845c" containerID="321ce26d41120044e4281e4e86a1edb2015ebef1eb8b1ac73cd869a12f049e5e" exitCode=0 Jul 08 19:35:49.703713 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:49.703299 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8" event={"ID":"b57a2e6a-e2f3-422b-8fb7-c4c6f409845c","Type":"ContainerDied","Data":"321ce26d41120044e4281e4e86a1edb2015ebef1eb8b1ac73cd869a12f049e5e"} Jul 08 19:35:49.703794 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:49.703779 2583 scope.go:117] "RemoveContainer" containerID="321ce26d41120044e4281e4e86a1edb2015ebef1eb8b1ac73cd869a12f049e5e" Jul 08 19:35:49.704831 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:49.704805 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:35:49.704890 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:49.704877 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:35:49.704934 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:49.704909 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:35:50.710577 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:50.710540 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8" event={"ID":"b57a2e6a-e2f3-422b-8fb7-c4c6f409845c","Type":"ContainerStarted","Data":"e30d8ff681a04c561574a4c69168553488a454317639582aef28f8c2630e854b"} Jul 08 19:35:51.108847 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:51.108746 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8" Jul 08 19:35:51.109843 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:51.109817 2583 patch_prober.go:28] interesting pod/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 08 19:35:51.109968 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:51.109875 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8" podUID="b57a2e6a-e2f3-422b-8fb7-c4c6f409845c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 08 19:35:51.627012 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:51.626980 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8" Jul 08 19:35:52.108992 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:52.108960 2583 patch_prober.go:28] interesting pod/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 08 19:35:52.109377 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:52.109028 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8" podUID="b57a2e6a-e2f3-422b-8fb7-c4c6f409845c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 08 19:35:53.108318 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:53.108282 2583 patch_prober.go:28] interesting pod/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 08 19:35:53.108508 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:53.108339 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8" podUID="b57a2e6a-e2f3-422b-8fb7-c4c6f409845c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 08 19:35:54.108554 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:54.108518 2583 patch_prober.go:28] interesting pod/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 08 19:35:54.108963 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:54.108580 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8" podUID="b57a2e6a-e2f3-422b-8fb7-c4c6f409845c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 08 19:35:55.112187 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:55.112152 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8" Jul 08 19:35:55.113224 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:35:55.113206 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8" Jul 08 19:37:40.334997 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:40.334965 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p"] Jul 08 19:37:40.335554 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:40.335165 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p" podUID="034f96e9-baa5-4c2e-b011-04791ce8d7ed" containerName="istio-proxy" containerID="cri-o://f67d2e52c6bb3d757978ab7ae7201199dbd517c905981f9af546c85abc35ea48" gracePeriod=30 Jul 08 19:37:45.596254 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:45.596183 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p" Jul 08 19:37:45.778889 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:45.778853 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vgpg8\" (UniqueName: \"kubernetes.io/projected/034f96e9-baa5-4c2e-b011-04791ce8d7ed-kube-api-access-vgpg8\") pod \"034f96e9-baa5-4c2e-b011-04791ce8d7ed\" (UID: \"034f96e9-baa5-4c2e-b011-04791ce8d7ed\") " Jul 08 19:37:45.779067 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:45.778905 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/034f96e9-baa5-4c2e-b011-04791ce8d7ed-workload-socket\") pod \"034f96e9-baa5-4c2e-b011-04791ce8d7ed\" (UID: \"034f96e9-baa5-4c2e-b011-04791ce8d7ed\") " Jul 08 19:37:45.779067 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:45.778950 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/034f96e9-baa5-4c2e-b011-04791ce8d7ed-istio-podinfo\") pod \"034f96e9-baa5-4c2e-b011-04791ce8d7ed\" (UID: \"034f96e9-baa5-4c2e-b011-04791ce8d7ed\") " Jul 08 19:37:45.779067 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:45.778965 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/034f96e9-baa5-4c2e-b011-04791ce8d7ed-credential-socket\") pod \"034f96e9-baa5-4c2e-b011-04791ce8d7ed\" (UID: \"034f96e9-baa5-4c2e-b011-04791ce8d7ed\") " Jul 08 19:37:45.779067 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:45.778980 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/034f96e9-baa5-4c2e-b011-04791ce8d7ed-workload-certs\") pod \"034f96e9-baa5-4c2e-b011-04791ce8d7ed\" (UID: \"034f96e9-baa5-4c2e-b011-04791ce8d7ed\") " Jul 08 19:37:45.779067 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:45.779004 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/034f96e9-baa5-4c2e-b011-04791ce8d7ed-istiod-ca-cert\") pod \"034f96e9-baa5-4c2e-b011-04791ce8d7ed\" (UID: \"034f96e9-baa5-4c2e-b011-04791ce8d7ed\") " Jul 08 19:37:45.779067 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:45.779040 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/034f96e9-baa5-4c2e-b011-04791ce8d7ed-istio-token\") pod \"034f96e9-baa5-4c2e-b011-04791ce8d7ed\" (UID: \"034f96e9-baa5-4c2e-b011-04791ce8d7ed\") " Jul 08 19:37:45.779377 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:45.779099 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/034f96e9-baa5-4c2e-b011-04791ce8d7ed-istio-data\") pod \"034f96e9-baa5-4c2e-b011-04791ce8d7ed\" (UID: \"034f96e9-baa5-4c2e-b011-04791ce8d7ed\") " Jul 08 19:37:45.779377 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:45.779293 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/034f96e9-baa5-4c2e-b011-04791ce8d7ed-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "034f96e9-baa5-4c2e-b011-04791ce8d7ed" (UID: "034f96e9-baa5-4c2e-b011-04791ce8d7ed"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:37:45.779377 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:45.779316 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/034f96e9-baa5-4c2e-b011-04791ce8d7ed-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "034f96e9-baa5-4c2e-b011-04791ce8d7ed" (UID: "034f96e9-baa5-4c2e-b011-04791ce8d7ed"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:37:45.779377 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:45.779344 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/034f96e9-baa5-4c2e-b011-04791ce8d7ed-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "034f96e9-baa5-4c2e-b011-04791ce8d7ed" (UID: "034f96e9-baa5-4c2e-b011-04791ce8d7ed"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:37:45.779624 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:45.779400 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/034f96e9-baa5-4c2e-b011-04791ce8d7ed-istio-envoy\") pod \"034f96e9-baa5-4c2e-b011-04791ce8d7ed\" (UID: \"034f96e9-baa5-4c2e-b011-04791ce8d7ed\") " Jul 08 19:37:45.779624 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:45.779517 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/034f96e9-baa5-4c2e-b011-04791ce8d7ed-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "034f96e9-baa5-4c2e-b011-04791ce8d7ed" (UID: "034f96e9-baa5-4c2e-b011-04791ce8d7ed"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:37:45.779795 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:45.779773 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/034f96e9-baa5-4c2e-b011-04791ce8d7ed-workload-socket\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:37:45.779878 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:45.779799 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/034f96e9-baa5-4c2e-b011-04791ce8d7ed-credential-socket\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:37:45.779878 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:45.779811 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/034f96e9-baa5-4c2e-b011-04791ce8d7ed-workload-certs\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:37:45.779878 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:45.779819 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/034f96e9-baa5-4c2e-b011-04791ce8d7ed-istiod-ca-cert\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:37:45.780221 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:45.780041 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/034f96e9-baa5-4c2e-b011-04791ce8d7ed-istio-data" (OuterVolumeSpecName: "istio-data") pod "034f96e9-baa5-4c2e-b011-04791ce8d7ed" (UID: "034f96e9-baa5-4c2e-b011-04791ce8d7ed"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:37:45.781551 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:45.781520 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/034f96e9-baa5-4c2e-b011-04791ce8d7ed-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "034f96e9-baa5-4c2e-b011-04791ce8d7ed" (UID: "034f96e9-baa5-4c2e-b011-04791ce8d7ed"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:37:45.781651 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:45.781554 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/034f96e9-baa5-4c2e-b011-04791ce8d7ed-istio-token" (OuterVolumeSpecName: "istio-token") pod "034f96e9-baa5-4c2e-b011-04791ce8d7ed" (UID: "034f96e9-baa5-4c2e-b011-04791ce8d7ed"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:37:45.781719 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:45.781707 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/034f96e9-baa5-4c2e-b011-04791ce8d7ed-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "034f96e9-baa5-4c2e-b011-04791ce8d7ed" (UID: "034f96e9-baa5-4c2e-b011-04791ce8d7ed"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 19:37:45.781780 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:45.781725 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/034f96e9-baa5-4c2e-b011-04791ce8d7ed-kube-api-access-vgpg8" (OuterVolumeSpecName: "kube-api-access-vgpg8") pod "034f96e9-baa5-4c2e-b011-04791ce8d7ed" (UID: "034f96e9-baa5-4c2e-b011-04791ce8d7ed"). InnerVolumeSpecName "kube-api-access-vgpg8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:37:45.880964 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:45.880887 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/034f96e9-baa5-4c2e-b011-04791ce8d7ed-istio-podinfo\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:37:45.880964 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:45.880915 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/034f96e9-baa5-4c2e-b011-04791ce8d7ed-istio-token\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:37:45.880964 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:45.880924 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/034f96e9-baa5-4c2e-b011-04791ce8d7ed-istio-data\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:37:45.880964 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:45.880933 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/034f96e9-baa5-4c2e-b011-04791ce8d7ed-istio-envoy\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:37:45.880964 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:45.880941 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vgpg8\" (UniqueName: \"kubernetes.io/projected/034f96e9-baa5-4c2e-b011-04791ce8d7ed-kube-api-access-vgpg8\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:37:46.250907 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:46.250873 2583 generic.go:356] "Generic (PLEG): container finished" podID="034f96e9-baa5-4c2e-b011-04791ce8d7ed" containerID="f67d2e52c6bb3d757978ab7ae7201199dbd517c905981f9af546c85abc35ea48" exitCode=0 Jul 08 19:37:46.251110 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:46.250939 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p" Jul 08 19:37:46.251110 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:46.250962 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p" event={"ID":"034f96e9-baa5-4c2e-b011-04791ce8d7ed","Type":"ContainerDied","Data":"f67d2e52c6bb3d757978ab7ae7201199dbd517c905981f9af546c85abc35ea48"} Jul 08 19:37:46.251110 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:46.251030 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p" event={"ID":"034f96e9-baa5-4c2e-b011-04791ce8d7ed","Type":"ContainerDied","Data":"6f64474683f196199a54b8bf3e93a924715334e514b4c0ef4ef147098a630f95"} Jul 08 19:37:46.251110 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:46.251060 2583 scope.go:117] "RemoveContainer" containerID="f67d2e52c6bb3d757978ab7ae7201199dbd517c905981f9af546c85abc35ea48" Jul 08 19:37:46.260854 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:46.260838 2583 scope.go:117] "RemoveContainer" containerID="0208aebb8979c017b895d4ec078720efd61ca636bc623bd3e54ecd7e409661c0" Jul 08 19:37:46.268751 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:46.268729 2583 scope.go:117] "RemoveContainer" containerID="f67d2e52c6bb3d757978ab7ae7201199dbd517c905981f9af546c85abc35ea48" Jul 08 19:37:46.268996 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:37:46.268976 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f67d2e52c6bb3d757978ab7ae7201199dbd517c905981f9af546c85abc35ea48\": container with ID starting with f67d2e52c6bb3d757978ab7ae7201199dbd517c905981f9af546c85abc35ea48 not found: ID does not exist" containerID="f67d2e52c6bb3d757978ab7ae7201199dbd517c905981f9af546c85abc35ea48" Jul 08 19:37:46.269050 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:46.269006 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f67d2e52c6bb3d757978ab7ae7201199dbd517c905981f9af546c85abc35ea48"} err="failed to get container status \"f67d2e52c6bb3d757978ab7ae7201199dbd517c905981f9af546c85abc35ea48\": rpc error: code = NotFound desc = could not find container \"f67d2e52c6bb3d757978ab7ae7201199dbd517c905981f9af546c85abc35ea48\": container with ID starting with f67d2e52c6bb3d757978ab7ae7201199dbd517c905981f9af546c85abc35ea48 not found: ID does not exist" Jul 08 19:37:46.269050 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:46.269024 2583 scope.go:117] "RemoveContainer" containerID="0208aebb8979c017b895d4ec078720efd61ca636bc623bd3e54ecd7e409661c0" Jul 08 19:37:46.269216 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:37:46.269202 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0208aebb8979c017b895d4ec078720efd61ca636bc623bd3e54ecd7e409661c0\": container with ID starting with 0208aebb8979c017b895d4ec078720efd61ca636bc623bd3e54ecd7e409661c0 not found: ID does not exist" containerID="0208aebb8979c017b895d4ec078720efd61ca636bc623bd3e54ecd7e409661c0" Jul 08 19:37:46.269258 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:46.269219 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0208aebb8979c017b895d4ec078720efd61ca636bc623bd3e54ecd7e409661c0"} err="failed to get container status \"0208aebb8979c017b895d4ec078720efd61ca636bc623bd3e54ecd7e409661c0\": rpc error: code = NotFound desc = could not find container \"0208aebb8979c017b895d4ec078720efd61ca636bc623bd3e54ecd7e409661c0\": container with ID starting with 0208aebb8979c017b895d4ec078720efd61ca636bc623bd3e54ecd7e409661c0 not found: ID does not exist" Jul 08 19:37:46.272786 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:46.272763 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p"] Jul 08 19:37:46.279421 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:46.279399 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-8ab38c-openshift-default-77f466fc5f-trk6p"] Jul 08 19:37:47.174254 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:47.174222 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="034f96e9-baa5-4c2e-b011-04791ce8d7ed" path="/var/lib/kubelet/pods/034f96e9-baa5-4c2e-b011-04791ce8d7ed/volumes" Jul 08 19:37:47.686819 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:47.686788 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8"] Jul 08 19:37:47.687115 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:47.687087 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8" podUID="b57a2e6a-e2f3-422b-8fb7-c4c6f409845c" containerName="istio-proxy" containerID="cri-o://e30d8ff681a04c561574a4c69168553488a454317639582aef28f8c2630e854b" gracePeriod=30 Jul 08 19:37:48.580379 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:48.580344 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m"] Jul 08 19:37:48.580865 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:48.580774 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="034f96e9-baa5-4c2e-b011-04791ce8d7ed" containerName="istio-proxy" Jul 08 19:37:48.580865 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:48.580787 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="034f96e9-baa5-4c2e-b011-04791ce8d7ed" containerName="istio-proxy" Jul 08 19:37:48.580865 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:48.580865 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="034f96e9-baa5-4c2e-b011-04791ce8d7ed" containerName="istio-proxy" Jul 08 19:37:48.585411 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:48.585388 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m" Jul 08 19:37:48.588113 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:48.588092 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-5f8b33-openshift-default-dockercfg-tngjv\"" Jul 08 19:37:48.599293 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:48.599254 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m"] Jul 08 19:37:48.705111 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:48.705080 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/641bfafc-e012-4233-b772-bf8757e9ccf7-istiod-ca-cert\") pod \"e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m\" (UID: \"641bfafc-e012-4233-b772-bf8757e9ccf7\") " pod="openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m" Jul 08 19:37:48.705284 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:48.705119 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/641bfafc-e012-4233-b772-bf8757e9ccf7-workload-socket\") pod \"e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m\" (UID: \"641bfafc-e012-4233-b772-bf8757e9ccf7\") " pod="openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m" Jul 08 19:37:48.705336 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:48.705292 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/641bfafc-e012-4233-b772-bf8757e9ccf7-istio-data\") pod \"e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m\" (UID: \"641bfafc-e012-4233-b772-bf8757e9ccf7\") " pod="openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m" Jul 08 19:37:48.705375 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:48.705350 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/641bfafc-e012-4233-b772-bf8757e9ccf7-istio-envoy\") pod \"e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m\" (UID: \"641bfafc-e012-4233-b772-bf8757e9ccf7\") " pod="openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m" Jul 08 19:37:48.705420 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:48.705384 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/641bfafc-e012-4233-b772-bf8757e9ccf7-credential-socket\") pod \"e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m\" (UID: \"641bfafc-e012-4233-b772-bf8757e9ccf7\") " pod="openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m" Jul 08 19:37:48.705420 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:48.705411 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/641bfafc-e012-4233-b772-bf8757e9ccf7-istio-podinfo\") pod \"e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m\" (UID: \"641bfafc-e012-4233-b772-bf8757e9ccf7\") " pod="openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m" Jul 08 19:37:48.705525 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:48.705460 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9qmj\" (UniqueName: \"kubernetes.io/projected/641bfafc-e012-4233-b772-bf8757e9ccf7-kube-api-access-s9qmj\") pod \"e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m\" (UID: \"641bfafc-e012-4233-b772-bf8757e9ccf7\") " pod="openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m" Jul 08 19:37:48.705525 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:48.705484 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/641bfafc-e012-4233-b772-bf8757e9ccf7-istio-token\") pod \"e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m\" (UID: \"641bfafc-e012-4233-b772-bf8757e9ccf7\") " pod="openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m" Jul 08 19:37:48.705525 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:48.705511 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/641bfafc-e012-4233-b772-bf8757e9ccf7-workload-certs\") pod \"e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m\" (UID: \"641bfafc-e012-4233-b772-bf8757e9ccf7\") " pod="openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m" Jul 08 19:37:48.805973 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:48.805931 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/641bfafc-e012-4233-b772-bf8757e9ccf7-istio-data\") pod \"e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m\" (UID: \"641bfafc-e012-4233-b772-bf8757e9ccf7\") " pod="openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m" Jul 08 19:37:48.805973 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:48.805980 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/641bfafc-e012-4233-b772-bf8757e9ccf7-istio-envoy\") pod \"e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m\" (UID: \"641bfafc-e012-4233-b772-bf8757e9ccf7\") " pod="openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m" Jul 08 19:37:48.806223 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:48.806000 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/641bfafc-e012-4233-b772-bf8757e9ccf7-credential-socket\") pod \"e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m\" (UID: \"641bfafc-e012-4233-b772-bf8757e9ccf7\") " pod="openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m" Jul 08 19:37:48.806223 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:48.806116 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/641bfafc-e012-4233-b772-bf8757e9ccf7-istio-podinfo\") pod \"e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m\" (UID: \"641bfafc-e012-4233-b772-bf8757e9ccf7\") " pod="openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m" Jul 08 19:37:48.806223 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:48.806158 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9qmj\" (UniqueName: \"kubernetes.io/projected/641bfafc-e012-4233-b772-bf8757e9ccf7-kube-api-access-s9qmj\") pod \"e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m\" (UID: \"641bfafc-e012-4233-b772-bf8757e9ccf7\") " pod="openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m" Jul 08 19:37:48.806223 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:48.806190 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/641bfafc-e012-4233-b772-bf8757e9ccf7-istio-token\") pod \"e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m\" (UID: \"641bfafc-e012-4233-b772-bf8757e9ccf7\") " pod="openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m" Jul 08 19:37:48.806223 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:48.806221 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/641bfafc-e012-4233-b772-bf8757e9ccf7-workload-certs\") pod \"e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m\" (UID: \"641bfafc-e012-4233-b772-bf8757e9ccf7\") " pod="openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m" Jul 08 19:37:48.806516 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:48.806260 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/641bfafc-e012-4233-b772-bf8757e9ccf7-istiod-ca-cert\") pod \"e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m\" (UID: \"641bfafc-e012-4233-b772-bf8757e9ccf7\") " pod="openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m" Jul 08 19:37:48.806516 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:48.806291 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/641bfafc-e012-4233-b772-bf8757e9ccf7-workload-socket\") pod \"e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m\" (UID: \"641bfafc-e012-4233-b772-bf8757e9ccf7\") " pod="openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m" Jul 08 19:37:48.806516 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:48.806424 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/641bfafc-e012-4233-b772-bf8757e9ccf7-credential-socket\") pod \"e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m\" (UID: \"641bfafc-e012-4233-b772-bf8757e9ccf7\") " pod="openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m" Jul 08 19:37:48.806673 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:48.806514 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/641bfafc-e012-4233-b772-bf8757e9ccf7-istio-data\") pod \"e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m\" (UID: \"641bfafc-e012-4233-b772-bf8757e9ccf7\") " pod="openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m" Jul 08 19:37:48.806745 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:48.806725 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/641bfafc-e012-4233-b772-bf8757e9ccf7-workload-certs\") pod \"e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m\" (UID: \"641bfafc-e012-4233-b772-bf8757e9ccf7\") " pod="openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m" Jul 08 19:37:48.806967 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:48.806952 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/641bfafc-e012-4233-b772-bf8757e9ccf7-workload-socket\") pod \"e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m\" (UID: \"641bfafc-e012-4233-b772-bf8757e9ccf7\") " pod="openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m" Jul 08 19:37:48.807333 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:48.807311 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/641bfafc-e012-4233-b772-bf8757e9ccf7-istiod-ca-cert\") pod \"e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m\" (UID: \"641bfafc-e012-4233-b772-bf8757e9ccf7\") " pod="openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m" Jul 08 19:37:48.808720 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:48.808686 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/641bfafc-e012-4233-b772-bf8757e9ccf7-istio-envoy\") pod \"e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m\" (UID: \"641bfafc-e012-4233-b772-bf8757e9ccf7\") " pod="openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m" Jul 08 19:37:48.808837 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:48.808738 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/641bfafc-e012-4233-b772-bf8757e9ccf7-istio-podinfo\") pod \"e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m\" (UID: \"641bfafc-e012-4233-b772-bf8757e9ccf7\") " pod="openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m" Jul 08 19:37:48.815592 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:48.815565 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/641bfafc-e012-4233-b772-bf8757e9ccf7-istio-token\") pod \"e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m\" (UID: \"641bfafc-e012-4233-b772-bf8757e9ccf7\") " pod="openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m" Jul 08 19:37:48.815702 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:48.815606 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9qmj\" (UniqueName: \"kubernetes.io/projected/641bfafc-e012-4233-b772-bf8757e9ccf7-kube-api-access-s9qmj\") pod \"e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m\" (UID: \"641bfafc-e012-4233-b772-bf8757e9ccf7\") " pod="openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m" Jul 08 19:37:48.895485 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:48.895396 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m" Jul 08 19:37:49.037590 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:49.037565 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m"] Jul 08 19:37:49.040237 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:37:49.040209 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod641bfafc_e012_4233_b772_bf8757e9ccf7.slice/crio-107fa085d901a31d9890e886762b4327fddabc64fb47b0d52c5cf851d513b04c WatchSource:0}: Error finding container 107fa085d901a31d9890e886762b4327fddabc64fb47b0d52c5cf851d513b04c: Status 404 returned error can't find the container with id 107fa085d901a31d9890e886762b4327fddabc64fb47b0d52c5cf851d513b04c Jul 08 19:37:49.042535 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:49.042497 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:37:49.042604 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:49.042588 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:37:49.042647 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:49.042636 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:37:49.267406 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:49.267365 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m" event={"ID":"641bfafc-e012-4233-b772-bf8757e9ccf7","Type":"ContainerStarted","Data":"c1357c466be997e42453079a53b15a63ea8e23bc98ed8e512eadb6440674920f"} Jul 08 19:37:49.267639 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:49.267422 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m" event={"ID":"641bfafc-e012-4233-b772-bf8757e9ccf7","Type":"ContainerStarted","Data":"107fa085d901a31d9890e886762b4327fddabc64fb47b0d52c5cf851d513b04c"} Jul 08 19:37:49.304824 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:49.304760 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m" podStartSLOduration=1.304738382 podStartE2EDuration="1.304738382s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:37:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:37:49.304624105 +0000 UTC m=+2034.583711726" watchObservedRunningTime="2026-07-08 19:37:49.304738382 +0000 UTC m=+2034.583826004" Jul 08 19:37:49.895919 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:49.895886 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m" Jul 08 19:37:50.900778 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:50.900751 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m" Jul 08 19:37:51.276180 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:51.276142 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m" Jul 08 19:37:51.277266 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:51.277242 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m" Jul 08 19:37:52.939233 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:52.939207 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8" Jul 08 19:37:52.942257 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:52.942239 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b57a2e6a-e2f3-422b-8fb7-c4c6f409845c-workload-socket\") pod \"b57a2e6a-e2f3-422b-8fb7-c4c6f409845c\" (UID: \"b57a2e6a-e2f3-422b-8fb7-c4c6f409845c\") " Jul 08 19:37:52.942375 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:52.942277 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b57a2e6a-e2f3-422b-8fb7-c4c6f409845c-istio-podinfo\") pod \"b57a2e6a-e2f3-422b-8fb7-c4c6f409845c\" (UID: \"b57a2e6a-e2f3-422b-8fb7-c4c6f409845c\") " Jul 08 19:37:52.942375 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:52.942315 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b57a2e6a-e2f3-422b-8fb7-c4c6f409845c-istio-data\") pod \"b57a2e6a-e2f3-422b-8fb7-c4c6f409845c\" (UID: \"b57a2e6a-e2f3-422b-8fb7-c4c6f409845c\") " Jul 08 19:37:52.942375 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:52.942366 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b57a2e6a-e2f3-422b-8fb7-c4c6f409845c-workload-certs\") pod \"b57a2e6a-e2f3-422b-8fb7-c4c6f409845c\" (UID: \"b57a2e6a-e2f3-422b-8fb7-c4c6f409845c\") " Jul 08 19:37:52.942515 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:52.942388 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b57a2e6a-e2f3-422b-8fb7-c4c6f409845c-istio-envoy\") pod \"b57a2e6a-e2f3-422b-8fb7-c4c6f409845c\" (UID: \"b57a2e6a-e2f3-422b-8fb7-c4c6f409845c\") " Jul 08 19:37:52.942515 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:52.942413 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j25pg\" (UniqueName: \"kubernetes.io/projected/b57a2e6a-e2f3-422b-8fb7-c4c6f409845c-kube-api-access-j25pg\") pod \"b57a2e6a-e2f3-422b-8fb7-c4c6f409845c\" (UID: \"b57a2e6a-e2f3-422b-8fb7-c4c6f409845c\") " Jul 08 19:37:52.942515 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:52.942496 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b57a2e6a-e2f3-422b-8fb7-c4c6f409845c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b57a2e6a-e2f3-422b-8fb7-c4c6f409845c" (UID: "b57a2e6a-e2f3-422b-8fb7-c4c6f409845c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:37:52.942645 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:52.942519 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b57a2e6a-e2f3-422b-8fb7-c4c6f409845c-istiod-ca-cert\") pod \"b57a2e6a-e2f3-422b-8fb7-c4c6f409845c\" (UID: \"b57a2e6a-e2f3-422b-8fb7-c4c6f409845c\") " Jul 08 19:37:52.942645 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:52.942553 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b57a2e6a-e2f3-422b-8fb7-c4c6f409845c-credential-socket\") pod \"b57a2e6a-e2f3-422b-8fb7-c4c6f409845c\" (UID: \"b57a2e6a-e2f3-422b-8fb7-c4c6f409845c\") " Jul 08 19:37:52.942645 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:52.942576 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b57a2e6a-e2f3-422b-8fb7-c4c6f409845c-istio-token\") pod \"b57a2e6a-e2f3-422b-8fb7-c4c6f409845c\" (UID: \"b57a2e6a-e2f3-422b-8fb7-c4c6f409845c\") " Jul 08 19:37:52.942863 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:52.942835 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b57a2e6a-e2f3-422b-8fb7-c4c6f409845c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b57a2e6a-e2f3-422b-8fb7-c4c6f409845c" (UID: "b57a2e6a-e2f3-422b-8fb7-c4c6f409845c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:37:52.942931 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:52.942894 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b57a2e6a-e2f3-422b-8fb7-c4c6f409845c-workload-socket\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:37:52.942991 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:52.942961 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b57a2e6a-e2f3-422b-8fb7-c4c6f409845c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b57a2e6a-e2f3-422b-8fb7-c4c6f409845c" (UID: "b57a2e6a-e2f3-422b-8fb7-c4c6f409845c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:37:52.942991 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:52.942972 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b57a2e6a-e2f3-422b-8fb7-c4c6f409845c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b57a2e6a-e2f3-422b-8fb7-c4c6f409845c" (UID: "b57a2e6a-e2f3-422b-8fb7-c4c6f409845c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:37:52.943369 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:52.943339 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b57a2e6a-e2f3-422b-8fb7-c4c6f409845c-istio-data" (OuterVolumeSpecName: "istio-data") pod "b57a2e6a-e2f3-422b-8fb7-c4c6f409845c" (UID: "b57a2e6a-e2f3-422b-8fb7-c4c6f409845c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:37:52.944669 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:52.944643 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b57a2e6a-e2f3-422b-8fb7-c4c6f409845c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b57a2e6a-e2f3-422b-8fb7-c4c6f409845c" (UID: "b57a2e6a-e2f3-422b-8fb7-c4c6f409845c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 19:37:52.944669 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:52.944657 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b57a2e6a-e2f3-422b-8fb7-c4c6f409845c-kube-api-access-j25pg" (OuterVolumeSpecName: "kube-api-access-j25pg") pod "b57a2e6a-e2f3-422b-8fb7-c4c6f409845c" (UID: "b57a2e6a-e2f3-422b-8fb7-c4c6f409845c"). InnerVolumeSpecName "kube-api-access-j25pg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:37:52.944787 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:52.944709 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b57a2e6a-e2f3-422b-8fb7-c4c6f409845c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b57a2e6a-e2f3-422b-8fb7-c4c6f409845c" (UID: "b57a2e6a-e2f3-422b-8fb7-c4c6f409845c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:37:52.945125 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:52.945109 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b57a2e6a-e2f3-422b-8fb7-c4c6f409845c-istio-token" (OuterVolumeSpecName: "istio-token") pod "b57a2e6a-e2f3-422b-8fb7-c4c6f409845c" (UID: "b57a2e6a-e2f3-422b-8fb7-c4c6f409845c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:37:53.044294 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:53.044262 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b57a2e6a-e2f3-422b-8fb7-c4c6f409845c-istiod-ca-cert\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:37:53.044294 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:53.044293 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b57a2e6a-e2f3-422b-8fb7-c4c6f409845c-credential-socket\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:37:53.044500 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:53.044305 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b57a2e6a-e2f3-422b-8fb7-c4c6f409845c-istio-token\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:37:53.044500 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:53.044314 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b57a2e6a-e2f3-422b-8fb7-c4c6f409845c-istio-podinfo\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:37:53.044500 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:53.044322 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b57a2e6a-e2f3-422b-8fb7-c4c6f409845c-istio-data\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:37:53.044500 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:53.044330 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b57a2e6a-e2f3-422b-8fb7-c4c6f409845c-workload-certs\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:37:53.044500 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:53.044337 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b57a2e6a-e2f3-422b-8fb7-c4c6f409845c-istio-envoy\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:37:53.044500 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:53.044345 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j25pg\" (UniqueName: \"kubernetes.io/projected/b57a2e6a-e2f3-422b-8fb7-c4c6f409845c-kube-api-access-j25pg\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:37:53.286791 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:53.286696 2583 generic.go:356] "Generic (PLEG): container finished" podID="b57a2e6a-e2f3-422b-8fb7-c4c6f409845c" containerID="e30d8ff681a04c561574a4c69168553488a454317639582aef28f8c2630e854b" exitCode=0 Jul 08 19:37:53.286791 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:53.286744 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8" Jul 08 19:37:53.286977 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:53.286786 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8" event={"ID":"b57a2e6a-e2f3-422b-8fb7-c4c6f409845c","Type":"ContainerDied","Data":"e30d8ff681a04c561574a4c69168553488a454317639582aef28f8c2630e854b"} Jul 08 19:37:53.286977 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:53.286838 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8" event={"ID":"b57a2e6a-e2f3-422b-8fb7-c4c6f409845c","Type":"ContainerDied","Data":"4639402140e8eee8d81b0194893bfdaa3fc8721cc883d20567c9c3f01cdb69f9"} Jul 08 19:37:53.286977 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:53.286860 2583 scope.go:117] "RemoveContainer" containerID="e30d8ff681a04c561574a4c69168553488a454317639582aef28f8c2630e854b" Jul 08 19:37:53.295503 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:53.295453 2583 scope.go:117] "RemoveContainer" containerID="321ce26d41120044e4281e4e86a1edb2015ebef1eb8b1ac73cd869a12f049e5e" Jul 08 19:37:53.303680 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:53.303662 2583 scope.go:117] "RemoveContainer" containerID="e30d8ff681a04c561574a4c69168553488a454317639582aef28f8c2630e854b" Jul 08 19:37:53.303977 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:37:53.303948 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e30d8ff681a04c561574a4c69168553488a454317639582aef28f8c2630e854b\": container with ID starting with e30d8ff681a04c561574a4c69168553488a454317639582aef28f8c2630e854b not found: ID does not exist" containerID="e30d8ff681a04c561574a4c69168553488a454317639582aef28f8c2630e854b" Jul 08 19:37:53.304064 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:53.303985 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e30d8ff681a04c561574a4c69168553488a454317639582aef28f8c2630e854b"} err="failed to get container status \"e30d8ff681a04c561574a4c69168553488a454317639582aef28f8c2630e854b\": rpc error: code = NotFound desc = could not find container \"e30d8ff681a04c561574a4c69168553488a454317639582aef28f8c2630e854b\": container with ID starting with e30d8ff681a04c561574a4c69168553488a454317639582aef28f8c2630e854b not found: ID does not exist" Jul 08 19:37:53.304064 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:53.304007 2583 scope.go:117] "RemoveContainer" containerID="321ce26d41120044e4281e4e86a1edb2015ebef1eb8b1ac73cd869a12f049e5e" Jul 08 19:37:53.304259 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:37:53.304238 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"321ce26d41120044e4281e4e86a1edb2015ebef1eb8b1ac73cd869a12f049e5e\": container with ID starting with 321ce26d41120044e4281e4e86a1edb2015ebef1eb8b1ac73cd869a12f049e5e not found: ID does not exist" containerID="321ce26d41120044e4281e4e86a1edb2015ebef1eb8b1ac73cd869a12f049e5e" Jul 08 19:37:53.304316 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:53.304261 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"321ce26d41120044e4281e4e86a1edb2015ebef1eb8b1ac73cd869a12f049e5e"} err="failed to get container status \"321ce26d41120044e4281e4e86a1edb2015ebef1eb8b1ac73cd869a12f049e5e\": rpc error: code = NotFound desc = could not find container \"321ce26d41120044e4281e4e86a1edb2015ebef1eb8b1ac73cd869a12f049e5e\": container with ID starting with 321ce26d41120044e4281e4e86a1edb2015ebef1eb8b1ac73cd869a12f049e5e not found: ID does not exist" Jul 08 19:37:53.305297 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:53.305274 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8"] Jul 08 19:37:53.310717 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:53.310696 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-1709ef-openshift-default-869966b4cf-g86q8"] Jul 08 19:37:55.175358 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:55.175319 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b57a2e6a-e2f3-422b-8fb7-c4c6f409845c" path="/var/lib/kubelet/pods/b57a2e6a-e2f3-422b-8fb7-c4c6f409845c/volumes" Jul 08 19:37:56.520908 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:56.520875 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s"] Jul 08 19:37:56.521282 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:56.521230 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b57a2e6a-e2f3-422b-8fb7-c4c6f409845c" containerName="istio-proxy" Jul 08 19:37:56.521282 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:56.521242 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b57a2e6a-e2f3-422b-8fb7-c4c6f409845c" containerName="istio-proxy" Jul 08 19:37:56.521282 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:56.521261 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="034f96e9-baa5-4c2e-b011-04791ce8d7ed" containerName="istio-proxy" Jul 08 19:37:56.521282 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:56.521266 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="034f96e9-baa5-4c2e-b011-04791ce8d7ed" containerName="istio-proxy" Jul 08 19:37:56.521420 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:56.521321 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="034f96e9-baa5-4c2e-b011-04791ce8d7ed" containerName="istio-proxy" Jul 08 19:37:56.521420 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:56.521328 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="b57a2e6a-e2f3-422b-8fb7-c4c6f409845c" containerName="istio-proxy" Jul 08 19:37:56.521420 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:56.521335 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="b57a2e6a-e2f3-422b-8fb7-c4c6f409845c" containerName="istio-proxy" Jul 08 19:37:56.525966 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:56.525945 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s" Jul 08 19:37:56.529157 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:56.529137 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-39e11a-openshift-default-dockercfg-kk97j\"" Jul 08 19:37:56.540469 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:56.540424 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s"] Jul 08 19:37:56.576626 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:56.576584 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5682b31f-cd3d-4fd7-8660-4c768f8d8c93-istio-podinfo\") pod \"e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s\" (UID: \"5682b31f-cd3d-4fd7-8660-4c768f8d8c93\") " pod="openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s" Jul 08 19:37:56.576795 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:56.576637 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5682b31f-cd3d-4fd7-8660-4c768f8d8c93-workload-certs\") pod \"e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s\" (UID: \"5682b31f-cd3d-4fd7-8660-4c768f8d8c93\") " pod="openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s" Jul 08 19:37:56.576795 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:56.576660 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5682b31f-cd3d-4fd7-8660-4c768f8d8c93-istio-data\") pod \"e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s\" (UID: \"5682b31f-cd3d-4fd7-8660-4c768f8d8c93\") " pod="openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s" Jul 08 19:37:56.576795 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:56.576680 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5682b31f-cd3d-4fd7-8660-4c768f8d8c93-istiod-ca-cert\") pod \"e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s\" (UID: \"5682b31f-cd3d-4fd7-8660-4c768f8d8c93\") " pod="openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s" Jul 08 19:37:56.576795 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:56.576714 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5682b31f-cd3d-4fd7-8660-4c768f8d8c93-credential-socket\") pod \"e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s\" (UID: \"5682b31f-cd3d-4fd7-8660-4c768f8d8c93\") " pod="openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s" Jul 08 19:37:56.576795 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:56.576748 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5682b31f-cd3d-4fd7-8660-4c768f8d8c93-istio-envoy\") pod \"e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s\" (UID: \"5682b31f-cd3d-4fd7-8660-4c768f8d8c93\") " pod="openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s" Jul 08 19:37:56.576795 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:56.576786 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnnv2\" (UniqueName: \"kubernetes.io/projected/5682b31f-cd3d-4fd7-8660-4c768f8d8c93-kube-api-access-gnnv2\") pod \"e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s\" (UID: \"5682b31f-cd3d-4fd7-8660-4c768f8d8c93\") " pod="openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s" Jul 08 19:37:56.577088 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:56.576812 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5682b31f-cd3d-4fd7-8660-4c768f8d8c93-workload-socket\") pod \"e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s\" (UID: \"5682b31f-cd3d-4fd7-8660-4c768f8d8c93\") " pod="openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s" Jul 08 19:37:56.577088 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:56.576843 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5682b31f-cd3d-4fd7-8660-4c768f8d8c93-istio-token\") pod \"e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s\" (UID: \"5682b31f-cd3d-4fd7-8660-4c768f8d8c93\") " pod="openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s" Jul 08 19:37:56.678027 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:56.677992 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnnv2\" (UniqueName: \"kubernetes.io/projected/5682b31f-cd3d-4fd7-8660-4c768f8d8c93-kube-api-access-gnnv2\") pod \"e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s\" (UID: \"5682b31f-cd3d-4fd7-8660-4c768f8d8c93\") " pod="openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s" Jul 08 19:37:56.678027 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:56.678032 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5682b31f-cd3d-4fd7-8660-4c768f8d8c93-workload-socket\") pod \"e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s\" (UID: \"5682b31f-cd3d-4fd7-8660-4c768f8d8c93\") " pod="openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s" Jul 08 19:37:56.678244 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:56.678062 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5682b31f-cd3d-4fd7-8660-4c768f8d8c93-istio-token\") pod \"e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s\" (UID: \"5682b31f-cd3d-4fd7-8660-4c768f8d8c93\") " pod="openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s" Jul 08 19:37:56.678244 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:56.678110 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5682b31f-cd3d-4fd7-8660-4c768f8d8c93-istio-podinfo\") pod \"e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s\" (UID: \"5682b31f-cd3d-4fd7-8660-4c768f8d8c93\") " pod="openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s" Jul 08 19:37:56.678244 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:56.678149 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5682b31f-cd3d-4fd7-8660-4c768f8d8c93-workload-certs\") pod \"e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s\" (UID: \"5682b31f-cd3d-4fd7-8660-4c768f8d8c93\") " pod="openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s" Jul 08 19:37:56.678244 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:56.678174 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5682b31f-cd3d-4fd7-8660-4c768f8d8c93-istio-data\") pod \"e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s\" (UID: \"5682b31f-cd3d-4fd7-8660-4c768f8d8c93\") " pod="openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s" Jul 08 19:37:56.678465 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:56.678324 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5682b31f-cd3d-4fd7-8660-4c768f8d8c93-istiod-ca-cert\") pod \"e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s\" (UID: \"5682b31f-cd3d-4fd7-8660-4c768f8d8c93\") " pod="openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s" Jul 08 19:37:56.678465 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:56.678395 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5682b31f-cd3d-4fd7-8660-4c768f8d8c93-credential-socket\") pod \"e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s\" (UID: \"5682b31f-cd3d-4fd7-8660-4c768f8d8c93\") " pod="openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s" Jul 08 19:37:56.678465 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:56.678452 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5682b31f-cd3d-4fd7-8660-4c768f8d8c93-istio-envoy\") pod \"e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s\" (UID: \"5682b31f-cd3d-4fd7-8660-4c768f8d8c93\") " pod="openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s" Jul 08 19:37:56.678619 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:56.678514 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5682b31f-cd3d-4fd7-8660-4c768f8d8c93-istio-data\") pod \"e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s\" (UID: \"5682b31f-cd3d-4fd7-8660-4c768f8d8c93\") " pod="openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s" Jul 08 19:37:56.678619 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:56.678514 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5682b31f-cd3d-4fd7-8660-4c768f8d8c93-workload-socket\") pod \"e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s\" (UID: \"5682b31f-cd3d-4fd7-8660-4c768f8d8c93\") " pod="openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s" Jul 08 19:37:56.678619 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:56.678595 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5682b31f-cd3d-4fd7-8660-4c768f8d8c93-workload-certs\") pod \"e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s\" (UID: \"5682b31f-cd3d-4fd7-8660-4c768f8d8c93\") " pod="openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s" Jul 08 19:37:56.678777 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:56.678754 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5682b31f-cd3d-4fd7-8660-4c768f8d8c93-credential-socket\") pod \"e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s\" (UID: \"5682b31f-cd3d-4fd7-8660-4c768f8d8c93\") " pod="openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s" Jul 08 19:37:56.679022 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:56.678999 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5682b31f-cd3d-4fd7-8660-4c768f8d8c93-istiod-ca-cert\") pod \"e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s\" (UID: \"5682b31f-cd3d-4fd7-8660-4c768f8d8c93\") " pod="openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s" Jul 08 19:37:56.680743 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:56.680725 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5682b31f-cd3d-4fd7-8660-4c768f8d8c93-istio-envoy\") pod \"e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s\" (UID: \"5682b31f-cd3d-4fd7-8660-4c768f8d8c93\") " pod="openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s" Jul 08 19:37:56.680809 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:56.680793 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5682b31f-cd3d-4fd7-8660-4c768f8d8c93-istio-podinfo\") pod \"e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s\" (UID: \"5682b31f-cd3d-4fd7-8660-4c768f8d8c93\") " pod="openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s" Jul 08 19:37:56.703816 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:56.703782 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5682b31f-cd3d-4fd7-8660-4c768f8d8c93-istio-token\") pod \"e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s\" (UID: \"5682b31f-cd3d-4fd7-8660-4c768f8d8c93\") " pod="openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s" Jul 08 19:37:56.704245 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:56.704220 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnnv2\" (UniqueName: \"kubernetes.io/projected/5682b31f-cd3d-4fd7-8660-4c768f8d8c93-kube-api-access-gnnv2\") pod \"e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s\" (UID: \"5682b31f-cd3d-4fd7-8660-4c768f8d8c93\") " pod="openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s" Jul 08 19:37:56.837975 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:56.837882 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s" Jul 08 19:37:56.987264 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:56.987240 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s"] Jul 08 19:37:56.989004 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:37:56.988977 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5682b31f_cd3d_4fd7_8660_4c768f8d8c93.slice/crio-84f489133aa93196c6aab112d9fc46330e84517a419ec5c98e978b73abb4fd16 WatchSource:0}: Error finding container 84f489133aa93196c6aab112d9fc46330e84517a419ec5c98e978b73abb4fd16: Status 404 returned error can't find the container with id 84f489133aa93196c6aab112d9fc46330e84517a419ec5c98e978b73abb4fd16 Jul 08 19:37:56.991161 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:56.991130 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:37:56.991247 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:56.991193 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:37:56.991247 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:56.991223 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:37:57.306839 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:57.306798 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s" event={"ID":"5682b31f-cd3d-4fd7-8660-4c768f8d8c93","Type":"ContainerStarted","Data":"690881ddc715837e3e8ed615ca068ebe941b0f0ae899a45792465f73b3df49c4"} Jul 08 19:37:57.307005 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:57.306869 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s" event={"ID":"5682b31f-cd3d-4fd7-8660-4c768f8d8c93","Type":"ContainerStarted","Data":"84f489133aa93196c6aab112d9fc46330e84517a419ec5c98e978b73abb4fd16"} Jul 08 19:37:57.332735 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:57.332681 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s" podStartSLOduration=1.332663454 podStartE2EDuration="1.332663454s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:37:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:37:57.330769376 +0000 UTC m=+2042.609857068" watchObservedRunningTime="2026-07-08 19:37:57.332663454 +0000 UTC m=+2042.611751074" Jul 08 19:37:57.838236 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:57.838197 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s" Jul 08 19:37:58.844294 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:58.844258 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s" Jul 08 19:37:59.317222 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:59.317190 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s" Jul 08 19:37:59.318313 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:37:59.318292 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s" Jul 08 19:38:06.706756 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:06.705612 2583 patch_prober.go:28] interesting pod/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": read tcp 10.133.0.2:46408->10.133.0.50:15021: read: connection reset by peer" start-of-body= Jul 08 19:38:06.706756 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:06.705707 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m" podUID="641bfafc-e012-4233-b772-bf8757e9ccf7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": read tcp 10.133.0.2:46408->10.133.0.50:15021: read: connection reset by peer" Jul 08 19:38:07.357957 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:07.357922 2583 generic.go:356] "Generic (PLEG): container finished" podID="641bfafc-e012-4233-b772-bf8757e9ccf7" containerID="c1357c466be997e42453079a53b15a63ea8e23bc98ed8e512eadb6440674920f" exitCode=0 Jul 08 19:38:07.358175 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:07.358004 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m" event={"ID":"641bfafc-e012-4233-b772-bf8757e9ccf7","Type":"ContainerDied","Data":"c1357c466be997e42453079a53b15a63ea8e23bc98ed8e512eadb6440674920f"} Jul 08 19:38:07.358508 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:07.358476 2583 scope.go:117] "RemoveContainer" containerID="c1357c466be997e42453079a53b15a63ea8e23bc98ed8e512eadb6440674920f" Jul 08 19:38:07.359588 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:07.359558 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:38:07.359660 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:07.359643 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:38:07.359698 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:07.359686 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:38:08.368316 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:08.368277 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m" event={"ID":"641bfafc-e012-4233-b772-bf8757e9ccf7","Type":"ContainerStarted","Data":"4c59d8e787db29eed14f1bc2515b63ac65d3b3e891a15d5745d0a97274b8d86e"} Jul 08 19:38:08.896460 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:08.896394 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m" Jul 08 19:38:08.897070 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:08.897033 2583 patch_prober.go:28] interesting pod/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 08 19:38:08.897219 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:08.897107 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m" podUID="641bfafc-e012-4233-b772-bf8757e9ccf7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 08 19:38:09.896361 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:09.896322 2583 patch_prober.go:28] interesting pod/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 08 19:38:09.897020 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:09.896985 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m" podUID="641bfafc-e012-4233-b772-bf8757e9ccf7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 08 19:38:10.896769 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:10.896720 2583 patch_prober.go:28] interesting pod/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 08 19:38:10.897324 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:10.896810 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m" podUID="641bfafc-e012-4233-b772-bf8757e9ccf7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 08 19:38:11.897905 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:11.897864 2583 patch_prober.go:28] interesting pod/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 08 19:38:11.898643 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:11.898606 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m" podUID="641bfafc-e012-4233-b772-bf8757e9ccf7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 08 19:38:12.900046 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:12.900014 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m" Jul 08 19:38:12.900531 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:12.900300 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m" Jul 08 19:38:12.901168 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:12.901150 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m" Jul 08 19:38:15.360631 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:15.360587 2583 patch_prober.go:28] interesting pod/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 08 19:38:15.361005 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:15.360667 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s" podUID="5682b31f-cd3d-4fd7-8660-4c768f8d8c93" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 08 19:38:16.426134 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:16.426101 2583 generic.go:356] "Generic (PLEG): container finished" podID="5682b31f-cd3d-4fd7-8660-4c768f8d8c93" containerID="690881ddc715837e3e8ed615ca068ebe941b0f0ae899a45792465f73b3df49c4" exitCode=0 Jul 08 19:38:16.426572 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:16.426151 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s" event={"ID":"5682b31f-cd3d-4fd7-8660-4c768f8d8c93","Type":"ContainerDied","Data":"690881ddc715837e3e8ed615ca068ebe941b0f0ae899a45792465f73b3df49c4"} Jul 08 19:38:16.426572 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:16.426561 2583 scope.go:117] "RemoveContainer" containerID="690881ddc715837e3e8ed615ca068ebe941b0f0ae899a45792465f73b3df49c4" Jul 08 19:38:16.427656 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:16.427621 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:38:16.427739 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:16.427718 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:38:16.427777 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:16.427761 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:38:17.432183 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:17.432150 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s" event={"ID":"5682b31f-cd3d-4fd7-8660-4c768f8d8c93","Type":"ContainerStarted","Data":"32228d287f3108f88bf248943c26aa46194e5a9ac53b15b215b187658f735546"} Jul 08 19:38:17.838741 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:17.838656 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s" Jul 08 19:38:17.838895 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:17.838866 2583 patch_prober.go:28] interesting pod/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 08 19:38:17.838947 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:17.838926 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s" podUID="5682b31f-cd3d-4fd7-8660-4c768f8d8c93" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 08 19:38:18.636311 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:18.636281 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m"] Jul 08 19:38:18.636708 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:18.636484 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m" podUID="641bfafc-e012-4233-b772-bf8757e9ccf7" containerName="istio-proxy" containerID="cri-o://4c59d8e787db29eed14f1bc2515b63ac65d3b3e891a15d5745d0a97274b8d86e" gracePeriod=30 Jul 08 19:38:18.838678 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:18.838641 2583 patch_prober.go:28] interesting pod/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 08 19:38:18.838857 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:18.838712 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s" podUID="5682b31f-cd3d-4fd7-8660-4c768f8d8c93" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 08 19:38:19.839308 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:19.839271 2583 patch_prober.go:28] interesting pod/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 08 19:38:19.839770 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:19.839329 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s" podUID="5682b31f-cd3d-4fd7-8660-4c768f8d8c93" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 08 19:38:20.838685 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:20.838649 2583 patch_prober.go:28] interesting pod/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 08 19:38:20.838887 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:20.838707 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s" podUID="5682b31f-cd3d-4fd7-8660-4c768f8d8c93" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 08 19:38:21.842317 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:21.842284 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s" Jul 08 19:38:21.842748 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:21.842673 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s" Jul 08 19:38:21.843404 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:21.843386 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s" Jul 08 19:38:23.898622 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:23.898560 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m" Jul 08 19:38:24.038037 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:24.038003 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/641bfafc-e012-4233-b772-bf8757e9ccf7-credential-socket\") pod \"641bfafc-e012-4233-b772-bf8757e9ccf7\" (UID: \"641bfafc-e012-4233-b772-bf8757e9ccf7\") " Jul 08 19:38:24.038240 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:24.038069 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/641bfafc-e012-4233-b772-bf8757e9ccf7-workload-socket\") pod \"641bfafc-e012-4233-b772-bf8757e9ccf7\" (UID: \"641bfafc-e012-4233-b772-bf8757e9ccf7\") " Jul 08 19:38:24.038240 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:24.038104 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/641bfafc-e012-4233-b772-bf8757e9ccf7-istiod-ca-cert\") pod \"641bfafc-e012-4233-b772-bf8757e9ccf7\" (UID: \"641bfafc-e012-4233-b772-bf8757e9ccf7\") " Jul 08 19:38:24.038240 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:24.038137 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/641bfafc-e012-4233-b772-bf8757e9ccf7-istio-envoy\") pod \"641bfafc-e012-4233-b772-bf8757e9ccf7\" (UID: \"641bfafc-e012-4233-b772-bf8757e9ccf7\") " Jul 08 19:38:24.038240 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:24.038156 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/641bfafc-e012-4233-b772-bf8757e9ccf7-istio-podinfo\") pod \"641bfafc-e012-4233-b772-bf8757e9ccf7\" (UID: \"641bfafc-e012-4233-b772-bf8757e9ccf7\") " Jul 08 19:38:24.038240 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:24.038172 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/641bfafc-e012-4233-b772-bf8757e9ccf7-workload-certs\") pod \"641bfafc-e012-4233-b772-bf8757e9ccf7\" (UID: \"641bfafc-e012-4233-b772-bf8757e9ccf7\") " Jul 08 19:38:24.038240 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:24.038202 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/641bfafc-e012-4233-b772-bf8757e9ccf7-istio-token\") pod \"641bfafc-e012-4233-b772-bf8757e9ccf7\" (UID: \"641bfafc-e012-4233-b772-bf8757e9ccf7\") " Jul 08 19:38:24.038661 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:24.038243 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/641bfafc-e012-4233-b772-bf8757e9ccf7-istio-data\") pod \"641bfafc-e012-4233-b772-bf8757e9ccf7\" (UID: \"641bfafc-e012-4233-b772-bf8757e9ccf7\") " Jul 08 19:38:24.038661 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:24.038295 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s9qmj\" (UniqueName: \"kubernetes.io/projected/641bfafc-e012-4233-b772-bf8757e9ccf7-kube-api-access-s9qmj\") pod \"641bfafc-e012-4233-b772-bf8757e9ccf7\" (UID: \"641bfafc-e012-4233-b772-bf8757e9ccf7\") " Jul 08 19:38:24.038661 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:24.038410 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/641bfafc-e012-4233-b772-bf8757e9ccf7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "641bfafc-e012-4233-b772-bf8757e9ccf7" (UID: "641bfafc-e012-4233-b772-bf8757e9ccf7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:38:24.038661 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:24.038461 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/641bfafc-e012-4233-b772-bf8757e9ccf7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "641bfafc-e012-4233-b772-bf8757e9ccf7" (UID: "641bfafc-e012-4233-b772-bf8757e9ccf7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:38:24.038831 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:24.038691 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/641bfafc-e012-4233-b772-bf8757e9ccf7-workload-socket\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:38:24.038831 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:24.038713 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/641bfafc-e012-4233-b772-bf8757e9ccf7-credential-socket\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:38:24.038831 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:24.038612 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/641bfafc-e012-4233-b772-bf8757e9ccf7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "641bfafc-e012-4233-b772-bf8757e9ccf7" (UID: "641bfafc-e012-4233-b772-bf8757e9ccf7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:38:24.038831 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:24.038750 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/641bfafc-e012-4233-b772-bf8757e9ccf7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "641bfafc-e012-4233-b772-bf8757e9ccf7" (UID: "641bfafc-e012-4233-b772-bf8757e9ccf7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:38:24.039255 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:24.039236 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/641bfafc-e012-4233-b772-bf8757e9ccf7-istio-data" (OuterVolumeSpecName: "istio-data") pod "641bfafc-e012-4233-b772-bf8757e9ccf7" (UID: "641bfafc-e012-4233-b772-bf8757e9ccf7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:38:24.041227 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:24.041202 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/641bfafc-e012-4233-b772-bf8757e9ccf7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "641bfafc-e012-4233-b772-bf8757e9ccf7" (UID: "641bfafc-e012-4233-b772-bf8757e9ccf7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:38:24.041338 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:24.041224 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/641bfafc-e012-4233-b772-bf8757e9ccf7-istio-token" (OuterVolumeSpecName: "istio-token") pod "641bfafc-e012-4233-b772-bf8757e9ccf7" (UID: "641bfafc-e012-4233-b772-bf8757e9ccf7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:38:24.041338 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:24.041254 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/641bfafc-e012-4233-b772-bf8757e9ccf7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "641bfafc-e012-4233-b772-bf8757e9ccf7" (UID: "641bfafc-e012-4233-b772-bf8757e9ccf7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 19:38:24.041338 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:24.041268 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/641bfafc-e012-4233-b772-bf8757e9ccf7-kube-api-access-s9qmj" (OuterVolumeSpecName: "kube-api-access-s9qmj") pod "641bfafc-e012-4233-b772-bf8757e9ccf7" (UID: "641bfafc-e012-4233-b772-bf8757e9ccf7"). InnerVolumeSpecName "kube-api-access-s9qmj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:38:24.140096 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:24.140046 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/641bfafc-e012-4233-b772-bf8757e9ccf7-istio-podinfo\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:38:24.140096 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:24.140090 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/641bfafc-e012-4233-b772-bf8757e9ccf7-workload-certs\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:38:24.140096 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:24.140099 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/641bfafc-e012-4233-b772-bf8757e9ccf7-istio-token\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:38:24.140096 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:24.140107 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/641bfafc-e012-4233-b772-bf8757e9ccf7-istio-data\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:38:24.140096 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:24.140115 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s9qmj\" (UniqueName: \"kubernetes.io/projected/641bfafc-e012-4233-b772-bf8757e9ccf7-kube-api-access-s9qmj\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:38:24.140398 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:24.140125 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/641bfafc-e012-4233-b772-bf8757e9ccf7-istiod-ca-cert\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:38:24.140398 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:24.140133 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/641bfafc-e012-4233-b772-bf8757e9ccf7-istio-envoy\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:38:24.469509 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:24.469477 2583 generic.go:356] "Generic (PLEG): container finished" podID="641bfafc-e012-4233-b772-bf8757e9ccf7" containerID="4c59d8e787db29eed14f1bc2515b63ac65d3b3e891a15d5745d0a97274b8d86e" exitCode=0 Jul 08 19:38:24.469688 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:24.469541 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m" Jul 08 19:38:24.469688 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:24.469557 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m" event={"ID":"641bfafc-e012-4233-b772-bf8757e9ccf7","Type":"ContainerDied","Data":"4c59d8e787db29eed14f1bc2515b63ac65d3b3e891a15d5745d0a97274b8d86e"} Jul 08 19:38:24.469688 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:24.469619 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m" event={"ID":"641bfafc-e012-4233-b772-bf8757e9ccf7","Type":"ContainerDied","Data":"107fa085d901a31d9890e886762b4327fddabc64fb47b0d52c5cf851d513b04c"} Jul 08 19:38:24.469688 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:24.469645 2583 scope.go:117] "RemoveContainer" containerID="4c59d8e787db29eed14f1bc2515b63ac65d3b3e891a15d5745d0a97274b8d86e" Jul 08 19:38:24.481060 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:24.481036 2583 scope.go:117] "RemoveContainer" containerID="c1357c466be997e42453079a53b15a63ea8e23bc98ed8e512eadb6440674920f" Jul 08 19:38:24.489322 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:24.489307 2583 scope.go:117] "RemoveContainer" containerID="4c59d8e787db29eed14f1bc2515b63ac65d3b3e891a15d5745d0a97274b8d86e" Jul 08 19:38:24.489600 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:38:24.489582 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c59d8e787db29eed14f1bc2515b63ac65d3b3e891a15d5745d0a97274b8d86e\": container with ID starting with 4c59d8e787db29eed14f1bc2515b63ac65d3b3e891a15d5745d0a97274b8d86e not found: ID does not exist" containerID="4c59d8e787db29eed14f1bc2515b63ac65d3b3e891a15d5745d0a97274b8d86e" Jul 08 19:38:24.489649 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:24.489608 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c59d8e787db29eed14f1bc2515b63ac65d3b3e891a15d5745d0a97274b8d86e"} err="failed to get container status \"4c59d8e787db29eed14f1bc2515b63ac65d3b3e891a15d5745d0a97274b8d86e\": rpc error: code = NotFound desc = could not find container \"4c59d8e787db29eed14f1bc2515b63ac65d3b3e891a15d5745d0a97274b8d86e\": container with ID starting with 4c59d8e787db29eed14f1bc2515b63ac65d3b3e891a15d5745d0a97274b8d86e not found: ID does not exist" Jul 08 19:38:24.489649 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:24.489624 2583 scope.go:117] "RemoveContainer" containerID="c1357c466be997e42453079a53b15a63ea8e23bc98ed8e512eadb6440674920f" Jul 08 19:38:24.489894 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:38:24.489876 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1357c466be997e42453079a53b15a63ea8e23bc98ed8e512eadb6440674920f\": container with ID starting with c1357c466be997e42453079a53b15a63ea8e23bc98ed8e512eadb6440674920f not found: ID does not exist" containerID="c1357c466be997e42453079a53b15a63ea8e23bc98ed8e512eadb6440674920f" Jul 08 19:38:24.489946 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:24.489899 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1357c466be997e42453079a53b15a63ea8e23bc98ed8e512eadb6440674920f"} err="failed to get container status \"c1357c466be997e42453079a53b15a63ea8e23bc98ed8e512eadb6440674920f\": rpc error: code = NotFound desc = could not find container \"c1357c466be997e42453079a53b15a63ea8e23bc98ed8e512eadb6440674920f\": container with ID starting with c1357c466be997e42453079a53b15a63ea8e23bc98ed8e512eadb6440674920f not found: ID does not exist" Jul 08 19:38:24.498703 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:24.498677 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m"] Jul 08 19:38:24.501982 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:24.501961 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-5f8b33-openshift-default-69894f77d5-ggk8m"] Jul 08 19:38:25.174221 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:25.174179 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="641bfafc-e012-4233-b772-bf8757e9ccf7" path="/var/lib/kubelet/pods/641bfafc-e012-4233-b772-bf8757e9ccf7/volumes" Jul 08 19:38:25.841900 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:25.841865 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s"] Jul 08 19:38:25.842196 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:25.842143 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s" podUID="5682b31f-cd3d-4fd7-8660-4c768f8d8c93" containerName="istio-proxy" containerID="cri-o://32228d287f3108f88bf248943c26aa46194e5a9ac53b15b215b187658f735546" gracePeriod=30 Jul 08 19:38:26.740565 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:26.740536 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs"] Jul 08 19:38:26.740964 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:26.740890 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b57a2e6a-e2f3-422b-8fb7-c4c6f409845c" containerName="istio-proxy" Jul 08 19:38:26.740964 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:26.740902 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b57a2e6a-e2f3-422b-8fb7-c4c6f409845c" containerName="istio-proxy" Jul 08 19:38:26.740964 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:26.740910 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="641bfafc-e012-4233-b772-bf8757e9ccf7" containerName="istio-proxy" Jul 08 19:38:26.740964 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:26.740916 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="641bfafc-e012-4233-b772-bf8757e9ccf7" containerName="istio-proxy" Jul 08 19:38:26.740964 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:26.740924 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="641bfafc-e012-4233-b772-bf8757e9ccf7" containerName="istio-proxy" Jul 08 19:38:26.740964 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:26.740929 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="641bfafc-e012-4233-b772-bf8757e9ccf7" containerName="istio-proxy" Jul 08 19:38:26.741176 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:26.740994 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="641bfafc-e012-4233-b772-bf8757e9ccf7" containerName="istio-proxy" Jul 08 19:38:26.741176 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:26.741002 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="641bfafc-e012-4233-b772-bf8757e9ccf7" containerName="istio-proxy" Jul 08 19:38:26.745833 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:26.745813 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs" Jul 08 19:38:26.748578 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:26.748554 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-d86987-openshift-default-dockercfg-q5mqr\"" Jul 08 19:38:26.759083 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:26.759055 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs"] Jul 08 19:38:26.863490 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:26.863447 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4a6ecf9-6c35-403a-b85f-28666300a263-istio-envoy\") pod \"e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs\" (UID: \"b4a6ecf9-6c35-403a-b85f-28666300a263\") " pod="openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs" Jul 08 19:38:26.863701 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:26.863499 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4a6ecf9-6c35-403a-b85f-28666300a263-workload-socket\") pod \"e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs\" (UID: \"b4a6ecf9-6c35-403a-b85f-28666300a263\") " pod="openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs" Jul 08 19:38:26.863701 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:26.863549 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4a6ecf9-6c35-403a-b85f-28666300a263-istio-token\") pod \"e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs\" (UID: \"b4a6ecf9-6c35-403a-b85f-28666300a263\") " pod="openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs" Jul 08 19:38:26.863701 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:26.863582 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-db7jf\" (UniqueName: \"kubernetes.io/projected/b4a6ecf9-6c35-403a-b85f-28666300a263-kube-api-access-db7jf\") pod \"e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs\" (UID: \"b4a6ecf9-6c35-403a-b85f-28666300a263\") " pod="openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs" Jul 08 19:38:26.863701 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:26.863637 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4a6ecf9-6c35-403a-b85f-28666300a263-workload-certs\") pod \"e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs\" (UID: \"b4a6ecf9-6c35-403a-b85f-28666300a263\") " pod="openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs" Jul 08 19:38:26.863701 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:26.863667 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4a6ecf9-6c35-403a-b85f-28666300a263-istiod-ca-cert\") pod \"e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs\" (UID: \"b4a6ecf9-6c35-403a-b85f-28666300a263\") " pod="openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs" Jul 08 19:38:26.863911 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:26.863749 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4a6ecf9-6c35-403a-b85f-28666300a263-credential-socket\") pod \"e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs\" (UID: \"b4a6ecf9-6c35-403a-b85f-28666300a263\") " pod="openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs" Jul 08 19:38:26.863911 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:26.863788 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4a6ecf9-6c35-403a-b85f-28666300a263-istio-data\") pod \"e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs\" (UID: \"b4a6ecf9-6c35-403a-b85f-28666300a263\") " pod="openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs" Jul 08 19:38:26.863911 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:26.863805 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4a6ecf9-6c35-403a-b85f-28666300a263-istio-podinfo\") pod \"e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs\" (UID: \"b4a6ecf9-6c35-403a-b85f-28666300a263\") " pod="openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs" Jul 08 19:38:26.964766 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:26.964726 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-db7jf\" (UniqueName: \"kubernetes.io/projected/b4a6ecf9-6c35-403a-b85f-28666300a263-kube-api-access-db7jf\") pod \"e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs\" (UID: \"b4a6ecf9-6c35-403a-b85f-28666300a263\") " pod="openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs" Jul 08 19:38:26.964766 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:26.964776 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4a6ecf9-6c35-403a-b85f-28666300a263-workload-certs\") pod \"e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs\" (UID: \"b4a6ecf9-6c35-403a-b85f-28666300a263\") " pod="openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs" Jul 08 19:38:26.965033 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:26.964843 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4a6ecf9-6c35-403a-b85f-28666300a263-istiod-ca-cert\") pod \"e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs\" (UID: \"b4a6ecf9-6c35-403a-b85f-28666300a263\") " pod="openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs" Jul 08 19:38:26.965033 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:26.964929 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4a6ecf9-6c35-403a-b85f-28666300a263-credential-socket\") pod \"e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs\" (UID: \"b4a6ecf9-6c35-403a-b85f-28666300a263\") " pod="openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs" Jul 08 19:38:26.965033 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:26.964976 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4a6ecf9-6c35-403a-b85f-28666300a263-istio-data\") pod \"e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs\" (UID: \"b4a6ecf9-6c35-403a-b85f-28666300a263\") " pod="openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs" Jul 08 19:38:26.965033 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:26.965001 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4a6ecf9-6c35-403a-b85f-28666300a263-istio-podinfo\") pod \"e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs\" (UID: \"b4a6ecf9-6c35-403a-b85f-28666300a263\") " pod="openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs" Jul 08 19:38:26.965245 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:26.965038 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4a6ecf9-6c35-403a-b85f-28666300a263-istio-envoy\") pod \"e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs\" (UID: \"b4a6ecf9-6c35-403a-b85f-28666300a263\") " pod="openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs" Jul 08 19:38:26.965245 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:26.965069 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4a6ecf9-6c35-403a-b85f-28666300a263-workload-socket\") pod \"e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs\" (UID: \"b4a6ecf9-6c35-403a-b85f-28666300a263\") " pod="openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs" Jul 08 19:38:26.965245 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:26.965146 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4a6ecf9-6c35-403a-b85f-28666300a263-istio-token\") pod \"e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs\" (UID: \"b4a6ecf9-6c35-403a-b85f-28666300a263\") " pod="openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs" Jul 08 19:38:26.965508 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:26.965486 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4a6ecf9-6c35-403a-b85f-28666300a263-workload-socket\") pod \"e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs\" (UID: \"b4a6ecf9-6c35-403a-b85f-28666300a263\") " pod="openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs" Jul 08 19:38:26.965577 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:26.965505 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4a6ecf9-6c35-403a-b85f-28666300a263-workload-certs\") pod \"e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs\" (UID: \"b4a6ecf9-6c35-403a-b85f-28666300a263\") " pod="openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs" Jul 08 19:38:26.965707 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:26.965683 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4a6ecf9-6c35-403a-b85f-28666300a263-credential-socket\") pod \"e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs\" (UID: \"b4a6ecf9-6c35-403a-b85f-28666300a263\") " pod="openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs" Jul 08 19:38:26.965766 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:26.965705 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4a6ecf9-6c35-403a-b85f-28666300a263-istio-data\") pod \"e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs\" (UID: \"b4a6ecf9-6c35-403a-b85f-28666300a263\") " pod="openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs" Jul 08 19:38:26.966280 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:26.966256 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4a6ecf9-6c35-403a-b85f-28666300a263-istiod-ca-cert\") pod \"e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs\" (UID: \"b4a6ecf9-6c35-403a-b85f-28666300a263\") " pod="openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs" Jul 08 19:38:26.967902 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:26.967867 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4a6ecf9-6c35-403a-b85f-28666300a263-istio-envoy\") pod \"e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs\" (UID: \"b4a6ecf9-6c35-403a-b85f-28666300a263\") " pod="openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs" Jul 08 19:38:26.968203 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:26.968178 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4a6ecf9-6c35-403a-b85f-28666300a263-istio-podinfo\") pod \"e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs\" (UID: \"b4a6ecf9-6c35-403a-b85f-28666300a263\") " pod="openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs" Jul 08 19:38:26.974733 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:26.974708 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4a6ecf9-6c35-403a-b85f-28666300a263-istio-token\") pod \"e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs\" (UID: \"b4a6ecf9-6c35-403a-b85f-28666300a263\") " pod="openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs" Jul 08 19:38:26.976071 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:26.976046 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-db7jf\" (UniqueName: \"kubernetes.io/projected/b4a6ecf9-6c35-403a-b85f-28666300a263-kube-api-access-db7jf\") pod \"e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs\" (UID: \"b4a6ecf9-6c35-403a-b85f-28666300a263\") " pod="openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs" Jul 08 19:38:27.060008 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:27.059923 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs" Jul 08 19:38:27.217326 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:27.217301 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs"] Jul 08 19:38:27.219858 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:38:27.219834 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4a6ecf9_6c35_403a_b85f_28666300a263.slice/crio-6d0e5307d8874777b55bffb551e19fc1ec2b211b51836f1486151476d32f5423 WatchSource:0}: Error finding container 6d0e5307d8874777b55bffb551e19fc1ec2b211b51836f1486151476d32f5423: Status 404 returned error can't find the container with id 6d0e5307d8874777b55bffb551e19fc1ec2b211b51836f1486151476d32f5423 Jul 08 19:38:27.222395 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:27.222361 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:38:27.222523 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:27.222456 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:38:27.222523 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:27.222495 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:38:27.486449 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:27.486396 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs" event={"ID":"b4a6ecf9-6c35-403a-b85f-28666300a263","Type":"ContainerStarted","Data":"8d3df0e6719c7c3deafb7fbba129f972def2b470cf980dabac0b8704e8f46c32"} Jul 08 19:38:27.486661 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:27.486470 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs" event={"ID":"b4a6ecf9-6c35-403a-b85f-28666300a263","Type":"ContainerStarted","Data":"6d0e5307d8874777b55bffb551e19fc1ec2b211b51836f1486151476d32f5423"} Jul 08 19:38:27.512669 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:27.512610 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs" podStartSLOduration=1.512594826 podStartE2EDuration="1.512594826s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:38:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:38:27.510199561 +0000 UTC m=+2072.789287180" watchObservedRunningTime="2026-07-08 19:38:27.512594826 +0000 UTC m=+2072.791682446" Jul 08 19:38:28.060387 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:28.060352 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs" Jul 08 19:38:28.065108 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:28.065083 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs" Jul 08 19:38:28.491268 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:28.491238 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs" Jul 08 19:38:28.492391 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:28.492375 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs" Jul 08 19:38:31.097872 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:31.097844 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s" Jul 08 19:38:31.206557 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:31.206528 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5682b31f-cd3d-4fd7-8660-4c768f8d8c93-istio-podinfo\") pod \"5682b31f-cd3d-4fd7-8660-4c768f8d8c93\" (UID: \"5682b31f-cd3d-4fd7-8660-4c768f8d8c93\") " Jul 08 19:38:31.206752 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:31.206562 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gnnv2\" (UniqueName: \"kubernetes.io/projected/5682b31f-cd3d-4fd7-8660-4c768f8d8c93-kube-api-access-gnnv2\") pod \"5682b31f-cd3d-4fd7-8660-4c768f8d8c93\" (UID: \"5682b31f-cd3d-4fd7-8660-4c768f8d8c93\") " Jul 08 19:38:31.206752 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:31.206579 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5682b31f-cd3d-4fd7-8660-4c768f8d8c93-istio-token\") pod \"5682b31f-cd3d-4fd7-8660-4c768f8d8c93\" (UID: \"5682b31f-cd3d-4fd7-8660-4c768f8d8c93\") " Jul 08 19:38:31.206752 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:31.206620 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5682b31f-cd3d-4fd7-8660-4c768f8d8c93-credential-socket\") pod \"5682b31f-cd3d-4fd7-8660-4c768f8d8c93\" (UID: \"5682b31f-cd3d-4fd7-8660-4c768f8d8c93\") " Jul 08 19:38:31.206752 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:31.206659 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5682b31f-cd3d-4fd7-8660-4c768f8d8c93-workload-certs\") pod \"5682b31f-cd3d-4fd7-8660-4c768f8d8c93\" (UID: \"5682b31f-cd3d-4fd7-8660-4c768f8d8c93\") " Jul 08 19:38:31.206752 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:31.206687 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5682b31f-cd3d-4fd7-8660-4c768f8d8c93-istiod-ca-cert\") pod \"5682b31f-cd3d-4fd7-8660-4c768f8d8c93\" (UID: \"5682b31f-cd3d-4fd7-8660-4c768f8d8c93\") " Jul 08 19:38:31.206752 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:31.206745 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5682b31f-cd3d-4fd7-8660-4c768f8d8c93-istio-envoy\") pod \"5682b31f-cd3d-4fd7-8660-4c768f8d8c93\" (UID: \"5682b31f-cd3d-4fd7-8660-4c768f8d8c93\") " Jul 08 19:38:31.207070 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:31.206772 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5682b31f-cd3d-4fd7-8660-4c768f8d8c93-workload-socket\") pod \"5682b31f-cd3d-4fd7-8660-4c768f8d8c93\" (UID: \"5682b31f-cd3d-4fd7-8660-4c768f8d8c93\") " Jul 08 19:38:31.207070 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:31.206808 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5682b31f-cd3d-4fd7-8660-4c768f8d8c93-istio-data\") pod \"5682b31f-cd3d-4fd7-8660-4c768f8d8c93\" (UID: \"5682b31f-cd3d-4fd7-8660-4c768f8d8c93\") " Jul 08 19:38:31.207070 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:31.207009 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5682b31f-cd3d-4fd7-8660-4c768f8d8c93-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5682b31f-cd3d-4fd7-8660-4c768f8d8c93" (UID: "5682b31f-cd3d-4fd7-8660-4c768f8d8c93"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:38:31.207214 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:31.207145 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5682b31f-cd3d-4fd7-8660-4c768f8d8c93-credential-socket\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:38:31.207387 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:31.207300 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5682b31f-cd3d-4fd7-8660-4c768f8d8c93-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5682b31f-cd3d-4fd7-8660-4c768f8d8c93" (UID: "5682b31f-cd3d-4fd7-8660-4c768f8d8c93"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:38:31.207387 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:31.207280 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5682b31f-cd3d-4fd7-8660-4c768f8d8c93-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5682b31f-cd3d-4fd7-8660-4c768f8d8c93" (UID: "5682b31f-cd3d-4fd7-8660-4c768f8d8c93"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:38:31.207595 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:31.207521 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5682b31f-cd3d-4fd7-8660-4c768f8d8c93-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5682b31f-cd3d-4fd7-8660-4c768f8d8c93" (UID: "5682b31f-cd3d-4fd7-8660-4c768f8d8c93"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:38:31.208350 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:31.208320 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5682b31f-cd3d-4fd7-8660-4c768f8d8c93-istio-data" (OuterVolumeSpecName: "istio-data") pod "5682b31f-cd3d-4fd7-8660-4c768f8d8c93" (UID: "5682b31f-cd3d-4fd7-8660-4c768f8d8c93"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:38:31.209064 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:31.209045 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5682b31f-cd3d-4fd7-8660-4c768f8d8c93-istio-token" (OuterVolumeSpecName: "istio-token") pod "5682b31f-cd3d-4fd7-8660-4c768f8d8c93" (UID: "5682b31f-cd3d-4fd7-8660-4c768f8d8c93"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:38:31.209182 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:31.209042 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5682b31f-cd3d-4fd7-8660-4c768f8d8c93-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5682b31f-cd3d-4fd7-8660-4c768f8d8c93" (UID: "5682b31f-cd3d-4fd7-8660-4c768f8d8c93"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 19:38:31.209423 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:31.209400 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5682b31f-cd3d-4fd7-8660-4c768f8d8c93-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5682b31f-cd3d-4fd7-8660-4c768f8d8c93" (UID: "5682b31f-cd3d-4fd7-8660-4c768f8d8c93"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:38:31.209516 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:31.209488 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5682b31f-cd3d-4fd7-8660-4c768f8d8c93-kube-api-access-gnnv2" (OuterVolumeSpecName: "kube-api-access-gnnv2") pod "5682b31f-cd3d-4fd7-8660-4c768f8d8c93" (UID: "5682b31f-cd3d-4fd7-8660-4c768f8d8c93"). InnerVolumeSpecName "kube-api-access-gnnv2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:38:31.308336 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:31.308288 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5682b31f-cd3d-4fd7-8660-4c768f8d8c93-istio-podinfo\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:38:31.308336 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:31.308331 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gnnv2\" (UniqueName: \"kubernetes.io/projected/5682b31f-cd3d-4fd7-8660-4c768f8d8c93-kube-api-access-gnnv2\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:38:31.308336 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:31.308341 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5682b31f-cd3d-4fd7-8660-4c768f8d8c93-istio-token\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:38:31.308336 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:31.308350 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5682b31f-cd3d-4fd7-8660-4c768f8d8c93-workload-certs\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:38:31.308336 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:31.308358 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5682b31f-cd3d-4fd7-8660-4c768f8d8c93-istiod-ca-cert\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:38:31.308676 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:31.308365 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5682b31f-cd3d-4fd7-8660-4c768f8d8c93-istio-envoy\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:38:31.308676 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:31.308375 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5682b31f-cd3d-4fd7-8660-4c768f8d8c93-workload-socket\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:38:31.308676 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:31.308384 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5682b31f-cd3d-4fd7-8660-4c768f8d8c93-istio-data\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:38:31.508225 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:31.508131 2583 generic.go:356] "Generic (PLEG): container finished" podID="5682b31f-cd3d-4fd7-8660-4c768f8d8c93" containerID="32228d287f3108f88bf248943c26aa46194e5a9ac53b15b215b187658f735546" exitCode=0 Jul 08 19:38:31.508225 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:31.508182 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s" Jul 08 19:38:31.508225 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:31.508207 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s" event={"ID":"5682b31f-cd3d-4fd7-8660-4c768f8d8c93","Type":"ContainerDied","Data":"32228d287f3108f88bf248943c26aa46194e5a9ac53b15b215b187658f735546"} Jul 08 19:38:31.508478 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:31.508261 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s" event={"ID":"5682b31f-cd3d-4fd7-8660-4c768f8d8c93","Type":"ContainerDied","Data":"84f489133aa93196c6aab112d9fc46330e84517a419ec5c98e978b73abb4fd16"} Jul 08 19:38:31.508478 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:31.508283 2583 scope.go:117] "RemoveContainer" containerID="32228d287f3108f88bf248943c26aa46194e5a9ac53b15b215b187658f735546" Jul 08 19:38:31.517721 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:31.517706 2583 scope.go:117] "RemoveContainer" containerID="690881ddc715837e3e8ed615ca068ebe941b0f0ae899a45792465f73b3df49c4" Jul 08 19:38:31.525549 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:31.525531 2583 scope.go:117] "RemoveContainer" containerID="32228d287f3108f88bf248943c26aa46194e5a9ac53b15b215b187658f735546" Jul 08 19:38:31.525792 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:38:31.525773 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32228d287f3108f88bf248943c26aa46194e5a9ac53b15b215b187658f735546\": container with ID starting with 32228d287f3108f88bf248943c26aa46194e5a9ac53b15b215b187658f735546 not found: ID does not exist" containerID="32228d287f3108f88bf248943c26aa46194e5a9ac53b15b215b187658f735546" Jul 08 19:38:31.525837 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:31.525803 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32228d287f3108f88bf248943c26aa46194e5a9ac53b15b215b187658f735546"} err="failed to get container status \"32228d287f3108f88bf248943c26aa46194e5a9ac53b15b215b187658f735546\": rpc error: code = NotFound desc = could not find container \"32228d287f3108f88bf248943c26aa46194e5a9ac53b15b215b187658f735546\": container with ID starting with 32228d287f3108f88bf248943c26aa46194e5a9ac53b15b215b187658f735546 not found: ID does not exist" Jul 08 19:38:31.525837 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:31.525824 2583 scope.go:117] "RemoveContainer" containerID="690881ddc715837e3e8ed615ca068ebe941b0f0ae899a45792465f73b3df49c4" Jul 08 19:38:31.526049 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:38:31.526035 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"690881ddc715837e3e8ed615ca068ebe941b0f0ae899a45792465f73b3df49c4\": container with ID starting with 690881ddc715837e3e8ed615ca068ebe941b0f0ae899a45792465f73b3df49c4 not found: ID does not exist" containerID="690881ddc715837e3e8ed615ca068ebe941b0f0ae899a45792465f73b3df49c4" Jul 08 19:38:31.526099 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:31.526051 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"690881ddc715837e3e8ed615ca068ebe941b0f0ae899a45792465f73b3df49c4"} err="failed to get container status \"690881ddc715837e3e8ed615ca068ebe941b0f0ae899a45792465f73b3df49c4\": rpc error: code = NotFound desc = could not find container \"690881ddc715837e3e8ed615ca068ebe941b0f0ae899a45792465f73b3df49c4\": container with ID starting with 690881ddc715837e3e8ed615ca068ebe941b0f0ae899a45792465f73b3df49c4 not found: ID does not exist" Jul 08 19:38:31.530878 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:31.530858 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s"] Jul 08 19:38:31.535369 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:31.535349 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-39e11a-openshift-default-86f656944f-rqv7s"] Jul 08 19:38:33.174617 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:33.174582 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5682b31f-cd3d-4fd7-8660-4c768f8d8c93" path="/var/lib/kubelet/pods/5682b31f-cd3d-4fd7-8660-4c768f8d8c93/volumes" Jul 08 19:38:34.565633 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:34.565605 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt"] Jul 08 19:38:34.566033 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:34.565956 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5682b31f-cd3d-4fd7-8660-4c768f8d8c93" containerName="istio-proxy" Jul 08 19:38:34.566033 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:34.565967 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5682b31f-cd3d-4fd7-8660-4c768f8d8c93" containerName="istio-proxy" Jul 08 19:38:34.566033 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:34.565976 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5682b31f-cd3d-4fd7-8660-4c768f8d8c93" containerName="istio-proxy" Jul 08 19:38:34.566033 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:34.565982 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5682b31f-cd3d-4fd7-8660-4c768f8d8c93" containerName="istio-proxy" Jul 08 19:38:34.566218 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:34.566037 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="5682b31f-cd3d-4fd7-8660-4c768f8d8c93" containerName="istio-proxy" Jul 08 19:38:34.566218 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:34.566047 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="5682b31f-cd3d-4fd7-8660-4c768f8d8c93" containerName="istio-proxy" Jul 08 19:38:34.570570 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:34.570545 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt" Jul 08 19:38:34.573254 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:34.573235 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-4a9cb4-openshift-default-dockercfg-s7fdj\"" Jul 08 19:38:34.585645 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:34.585622 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt"] Jul 08 19:38:34.737253 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:34.737209 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1273c7c-f18f-4459-8cec-a5770acff4b5-istio-token\") pod \"e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt\" (UID: \"a1273c7c-f18f-4459-8cec-a5770acff4b5\") " pod="openshift-ingress/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt" Jul 08 19:38:34.737487 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:34.737265 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1273c7c-f18f-4459-8cec-a5770acff4b5-workload-socket\") pod \"e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt\" (UID: \"a1273c7c-f18f-4459-8cec-a5770acff4b5\") " pod="openshift-ingress/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt" Jul 08 19:38:34.737487 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:34.737343 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1273c7c-f18f-4459-8cec-a5770acff4b5-workload-certs\") pod \"e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt\" (UID: \"a1273c7c-f18f-4459-8cec-a5770acff4b5\") " pod="openshift-ingress/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt" Jul 08 19:38:34.737487 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:34.737417 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1273c7c-f18f-4459-8cec-a5770acff4b5-istiod-ca-cert\") pod \"e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt\" (UID: \"a1273c7c-f18f-4459-8cec-a5770acff4b5\") " pod="openshift-ingress/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt" Jul 08 19:38:34.737641 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:34.737485 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1273c7c-f18f-4459-8cec-a5770acff4b5-credential-socket\") pod \"e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt\" (UID: \"a1273c7c-f18f-4459-8cec-a5770acff4b5\") " pod="openshift-ingress/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt" Jul 08 19:38:34.737641 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:34.737544 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ng92h\" (UniqueName: \"kubernetes.io/projected/a1273c7c-f18f-4459-8cec-a5770acff4b5-kube-api-access-ng92h\") pod \"e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt\" (UID: \"a1273c7c-f18f-4459-8cec-a5770acff4b5\") " pod="openshift-ingress/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt" Jul 08 19:38:34.737641 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:34.737571 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1273c7c-f18f-4459-8cec-a5770acff4b5-istio-envoy\") pod \"e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt\" (UID: \"a1273c7c-f18f-4459-8cec-a5770acff4b5\") " pod="openshift-ingress/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt" Jul 08 19:38:34.737641 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:34.737590 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1273c7c-f18f-4459-8cec-a5770acff4b5-istio-data\") pod \"e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt\" (UID: \"a1273c7c-f18f-4459-8cec-a5770acff4b5\") " pod="openshift-ingress/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt" Jul 08 19:38:34.737641 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:34.737608 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1273c7c-f18f-4459-8cec-a5770acff4b5-istio-podinfo\") pod \"e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt\" (UID: \"a1273c7c-f18f-4459-8cec-a5770acff4b5\") " pod="openshift-ingress/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt" Jul 08 19:38:34.838871 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:34.838758 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1273c7c-f18f-4459-8cec-a5770acff4b5-istiod-ca-cert\") pod \"e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt\" (UID: \"a1273c7c-f18f-4459-8cec-a5770acff4b5\") " pod="openshift-ingress/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt" Jul 08 19:38:34.838871 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:34.838822 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1273c7c-f18f-4459-8cec-a5770acff4b5-credential-socket\") pod \"e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt\" (UID: \"a1273c7c-f18f-4459-8cec-a5770acff4b5\") " pod="openshift-ingress/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt" Jul 08 19:38:34.838871 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:34.838854 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ng92h\" (UniqueName: \"kubernetes.io/projected/a1273c7c-f18f-4459-8cec-a5770acff4b5-kube-api-access-ng92h\") pod \"e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt\" (UID: \"a1273c7c-f18f-4459-8cec-a5770acff4b5\") " pod="openshift-ingress/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt" Jul 08 19:38:34.838871 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:34.838871 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1273c7c-f18f-4459-8cec-a5770acff4b5-istio-envoy\") pod \"e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt\" (UID: \"a1273c7c-f18f-4459-8cec-a5770acff4b5\") " pod="openshift-ingress/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt" Jul 08 19:38:34.839213 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:34.838888 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1273c7c-f18f-4459-8cec-a5770acff4b5-istio-data\") pod \"e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt\" (UID: \"a1273c7c-f18f-4459-8cec-a5770acff4b5\") " pod="openshift-ingress/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt" Jul 08 19:38:34.839213 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:34.838907 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1273c7c-f18f-4459-8cec-a5770acff4b5-istio-podinfo\") pod \"e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt\" (UID: \"a1273c7c-f18f-4459-8cec-a5770acff4b5\") " pod="openshift-ingress/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt" Jul 08 19:38:34.839213 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:34.838934 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1273c7c-f18f-4459-8cec-a5770acff4b5-istio-token\") pod \"e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt\" (UID: \"a1273c7c-f18f-4459-8cec-a5770acff4b5\") " pod="openshift-ingress/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt" Jul 08 19:38:34.839213 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:34.838969 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1273c7c-f18f-4459-8cec-a5770acff4b5-workload-socket\") pod \"e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt\" (UID: \"a1273c7c-f18f-4459-8cec-a5770acff4b5\") " pod="openshift-ingress/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt" Jul 08 19:38:34.839213 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:34.839005 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1273c7c-f18f-4459-8cec-a5770acff4b5-workload-certs\") pod \"e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt\" (UID: \"a1273c7c-f18f-4459-8cec-a5770acff4b5\") " pod="openshift-ingress/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt" Jul 08 19:38:34.839507 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:34.839478 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1273c7c-f18f-4459-8cec-a5770acff4b5-workload-socket\") pod \"e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt\" (UID: \"a1273c7c-f18f-4459-8cec-a5770acff4b5\") " pod="openshift-ingress/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt" Jul 08 19:38:34.839571 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:34.839493 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1273c7c-f18f-4459-8cec-a5770acff4b5-workload-certs\") pod \"e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt\" (UID: \"a1273c7c-f18f-4459-8cec-a5770acff4b5\") " pod="openshift-ingress/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt" Jul 08 19:38:34.839623 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:34.839598 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1273c7c-f18f-4459-8cec-a5770acff4b5-credential-socket\") pod \"e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt\" (UID: \"a1273c7c-f18f-4459-8cec-a5770acff4b5\") " pod="openshift-ingress/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt" Jul 08 19:38:34.839672 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:34.839623 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1273c7c-f18f-4459-8cec-a5770acff4b5-istiod-ca-cert\") pod \"e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt\" (UID: \"a1273c7c-f18f-4459-8cec-a5770acff4b5\") " pod="openshift-ingress/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt" Jul 08 19:38:34.839672 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:34.839639 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1273c7c-f18f-4459-8cec-a5770acff4b5-istio-data\") pod \"e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt\" (UID: \"a1273c7c-f18f-4459-8cec-a5770acff4b5\") " pod="openshift-ingress/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt" Jul 08 19:38:34.841779 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:34.841750 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1273c7c-f18f-4459-8cec-a5770acff4b5-istio-envoy\") pod \"e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt\" (UID: \"a1273c7c-f18f-4459-8cec-a5770acff4b5\") " pod="openshift-ingress/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt" Jul 08 19:38:34.842044 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:34.842024 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1273c7c-f18f-4459-8cec-a5770acff4b5-istio-podinfo\") pod \"e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt\" (UID: \"a1273c7c-f18f-4459-8cec-a5770acff4b5\") " pod="openshift-ingress/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt" Jul 08 19:38:34.852296 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:34.852265 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ng92h\" (UniqueName: \"kubernetes.io/projected/a1273c7c-f18f-4459-8cec-a5770acff4b5-kube-api-access-ng92h\") pod \"e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt\" (UID: \"a1273c7c-f18f-4459-8cec-a5770acff4b5\") " pod="openshift-ingress/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt" Jul 08 19:38:34.852737 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:34.852717 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1273c7c-f18f-4459-8cec-a5770acff4b5-istio-token\") pod \"e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt\" (UID: \"a1273c7c-f18f-4459-8cec-a5770acff4b5\") " pod="openshift-ingress/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt" Jul 08 19:38:34.880308 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:34.880271 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt" Jul 08 19:38:35.027323 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:35.027291 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt"] Jul 08 19:38:35.030534 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:38:35.030504 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1273c7c_f18f_4459_8cec_a5770acff4b5.slice/crio-1392de99899d0c8d8ae9e1ac7ed41b17141ff0d45e227f221603195669302805 WatchSource:0}: Error finding container 1392de99899d0c8d8ae9e1ac7ed41b17141ff0d45e227f221603195669302805: Status 404 returned error can't find the container with id 1392de99899d0c8d8ae9e1ac7ed41b17141ff0d45e227f221603195669302805 Jul 08 19:38:35.032633 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:35.032601 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:38:35.032726 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:35.032668 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:38:35.032726 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:35.032695 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:38:35.529023 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:35.528984 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt" event={"ID":"a1273c7c-f18f-4459-8cec-a5770acff4b5","Type":"ContainerStarted","Data":"357ccd2c7eaa29ec5f7c73034e146d23e59a9beceeb747dee59a4ad215f1513c"} Jul 08 19:38:35.529207 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:35.529038 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt" event={"ID":"a1273c7c-f18f-4459-8cec-a5770acff4b5","Type":"ContainerStarted","Data":"1392de99899d0c8d8ae9e1ac7ed41b17141ff0d45e227f221603195669302805"} Jul 08 19:38:35.569315 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:35.569268 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt" podStartSLOduration=1.569251411 podStartE2EDuration="1.569251411s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:38:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:38:35.567637628 +0000 UTC m=+2080.846725250" watchObservedRunningTime="2026-07-08 19:38:35.569251411 +0000 UTC m=+2080.848339035" Jul 08 19:38:35.880905 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:35.880788 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt" Jul 08 19:38:36.886097 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:36.886070 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt" Jul 08 19:38:37.540246 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:37.540212 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt" Jul 08 19:38:37.541134 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:37.541115 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt" Jul 08 19:38:43.492932 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:43.492883 2583 patch_prober.go:28] interesting pod/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 08 19:38:43.493391 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:43.492971 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs" podUID="b4a6ecf9-6c35-403a-b85f-28666300a263" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 08 19:38:43.577321 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:43.577286 2583 generic.go:356] "Generic (PLEG): container finished" podID="b4a6ecf9-6c35-403a-b85f-28666300a263" containerID="8d3df0e6719c7c3deafb7fbba129f972def2b470cf980dabac0b8704e8f46c32" exitCode=0 Jul 08 19:38:43.577553 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:43.577366 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs" event={"ID":"b4a6ecf9-6c35-403a-b85f-28666300a263","Type":"ContainerDied","Data":"8d3df0e6719c7c3deafb7fbba129f972def2b470cf980dabac0b8704e8f46c32"} Jul 08 19:38:43.577854 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:43.577838 2583 scope.go:117] "RemoveContainer" containerID="8d3df0e6719c7c3deafb7fbba129f972def2b470cf980dabac0b8704e8f46c32" Jul 08 19:38:43.578957 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:43.578925 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:38:43.579069 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:43.579011 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:38:43.579069 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:43.579058 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:38:44.588446 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:44.587502 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs" event={"ID":"b4a6ecf9-6c35-403a-b85f-28666300a263","Type":"ContainerStarted","Data":"cddbec3767f1680a339e01b49fbd7424f387783ed86a973030f424e6dced08d8"} Jul 08 19:38:45.060912 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:45.060880 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs" Jul 08 19:38:45.061265 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:45.061244 2583 patch_prober.go:28] interesting pod/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 08 19:38:45.061416 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:45.061390 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs" podUID="b4a6ecf9-6c35-403a-b85f-28666300a263" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 08 19:38:46.061223 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:46.061182 2583 patch_prober.go:28] interesting pod/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 08 19:38:46.061867 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:46.061832 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs" podUID="b4a6ecf9-6c35-403a-b85f-28666300a263" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 08 19:38:47.061809 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:47.061769 2583 patch_prober.go:28] interesting pod/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 08 19:38:47.062916 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:47.062484 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs" podUID="b4a6ecf9-6c35-403a-b85f-28666300a263" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 08 19:38:48.060634 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:48.060593 2583 patch_prober.go:28] interesting pod/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 08 19:38:48.060948 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:48.060915 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs" podUID="b4a6ecf9-6c35-403a-b85f-28666300a263" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 08 19:38:49.064804 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:49.064767 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs" Jul 08 19:38:49.065319 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:49.065296 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs" Jul 08 19:38:49.065989 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:49.065965 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs" Jul 08 19:38:50.620802 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:50.620716 2583 generic.go:356] "Generic (PLEG): container finished" podID="a1273c7c-f18f-4459-8cec-a5770acff4b5" containerID="357ccd2c7eaa29ec5f7c73034e146d23e59a9beceeb747dee59a4ad215f1513c" exitCode=0 Jul 08 19:38:50.620802 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:50.620789 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt" event={"ID":"a1273c7c-f18f-4459-8cec-a5770acff4b5","Type":"ContainerDied","Data":"357ccd2c7eaa29ec5f7c73034e146d23e59a9beceeb747dee59a4ad215f1513c"} Jul 08 19:38:50.621372 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:50.621354 2583 scope.go:117] "RemoveContainer" containerID="357ccd2c7eaa29ec5f7c73034e146d23e59a9beceeb747dee59a4ad215f1513c" Jul 08 19:38:50.622309 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:50.622278 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:38:50.622404 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:50.622385 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:38:50.622484 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:50.622456 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:38:51.626562 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:51.626525 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt" event={"ID":"a1273c7c-f18f-4459-8cec-a5770acff4b5","Type":"ContainerStarted","Data":"8e5aec4252da57700394711db4da537839c51480baa446d9b38ad70d510fb3df"} Jul 08 19:38:51.881482 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:51.881359 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt" Jul 08 19:38:51.882658 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:51.882634 2583 patch_prober.go:28] interesting pod/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 08 19:38:51.882720 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:51.882700 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt" podUID="a1273c7c-f18f-4459-8cec-a5770acff4b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 08 19:38:52.540607 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:52.540577 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt" Jul 08 19:38:52.880663 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:52.880573 2583 patch_prober.go:28] interesting pod/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 08 19:38:52.880663 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:52.880634 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt" podUID="a1273c7c-f18f-4459-8cec-a5770acff4b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 08 19:38:53.881695 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:53.881662 2583 patch_prober.go:28] interesting pod/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 08 19:38:53.882096 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:53.881718 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt" podUID="a1273c7c-f18f-4459-8cec-a5770acff4b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 08 19:38:54.881677 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:54.881627 2583 patch_prober.go:28] interesting pod/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 08 19:38:54.881862 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:54.881698 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt" podUID="a1273c7c-f18f-4459-8cec-a5770acff4b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 08 19:38:55.884157 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:55.884117 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt" Jul 08 19:38:55.885174 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:38:55.885158 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt" Jul 08 19:39:22.107050 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:39:22.107006 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-4a9cb4/auth-test-model-4a9cb4-kserve-78dbd6c755-mkjfx"] Jul 08 19:39:22.112208 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:39:22.112182 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4a9cb4/auth-test-model-4a9cb4-kserve-78dbd6c755-mkjfx" Jul 08 19:39:22.115248 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:39:22.115220 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-4a9cb4/auth-test-model-4a9cb4-kserve-78dbd6c755-mkjfx"] Jul 08 19:39:22.115988 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:39:22.115870 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-4a9cb4\"/\"kube-root-ca.crt\"" Jul 08 19:39:22.116219 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:39:22.116199 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-4a9cb4\"/\"openshift-service-ca.crt\"" Jul 08 19:39:22.116444 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:39:22.116230 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-4a9cb4\"/\"auth-test-model-4a9cb4-kserve-self-signed-certs\"" Jul 08 19:39:22.116599 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:39:22.116253 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-4a9cb4\"/\"default-dockercfg-kcgp9\"" Jul 08 19:39:22.203167 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:39:22.203127 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a357283c-3904-4eca-ba02-09775a896ce8-dshm\") pod \"auth-test-model-4a9cb4-kserve-78dbd6c755-mkjfx\" (UID: \"a357283c-3904-4eca-ba02-09775a896ce8\") " pod="ai-tenant-e2e-shared-b-4a9cb4/auth-test-model-4a9cb4-kserve-78dbd6c755-mkjfx" Jul 08 19:39:22.203167 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:39:22.203174 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a357283c-3904-4eca-ba02-09775a896ce8-tls-certs\") pod \"auth-test-model-4a9cb4-kserve-78dbd6c755-mkjfx\" (UID: \"a357283c-3904-4eca-ba02-09775a896ce8\") " pod="ai-tenant-e2e-shared-b-4a9cb4/auth-test-model-4a9cb4-kserve-78dbd6c755-mkjfx" Jul 08 19:39:22.203389 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:39:22.203302 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q97dt\" (UniqueName: \"kubernetes.io/projected/a357283c-3904-4eca-ba02-09775a896ce8-kube-api-access-q97dt\") pod \"auth-test-model-4a9cb4-kserve-78dbd6c755-mkjfx\" (UID: \"a357283c-3904-4eca-ba02-09775a896ce8\") " pod="ai-tenant-e2e-shared-b-4a9cb4/auth-test-model-4a9cb4-kserve-78dbd6c755-mkjfx" Jul 08 19:39:22.203389 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:39:22.203349 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a357283c-3904-4eca-ba02-09775a896ce8-tmp-dir\") pod \"auth-test-model-4a9cb4-kserve-78dbd6c755-mkjfx\" (UID: \"a357283c-3904-4eca-ba02-09775a896ce8\") " pod="ai-tenant-e2e-shared-b-4a9cb4/auth-test-model-4a9cb4-kserve-78dbd6c755-mkjfx" Jul 08 19:39:22.203501 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:39:22.203403 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a357283c-3904-4eca-ba02-09775a896ce8-model-cache\") pod \"auth-test-model-4a9cb4-kserve-78dbd6c755-mkjfx\" (UID: \"a357283c-3904-4eca-ba02-09775a896ce8\") " pod="ai-tenant-e2e-shared-b-4a9cb4/auth-test-model-4a9cb4-kserve-78dbd6c755-mkjfx" Jul 08 19:39:22.203501 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:39:22.203458 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a357283c-3904-4eca-ba02-09775a896ce8-home\") pod \"auth-test-model-4a9cb4-kserve-78dbd6c755-mkjfx\" (UID: \"a357283c-3904-4eca-ba02-09775a896ce8\") " pod="ai-tenant-e2e-shared-b-4a9cb4/auth-test-model-4a9cb4-kserve-78dbd6c755-mkjfx" Jul 08 19:39:22.304796 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:39:22.304758 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q97dt\" (UniqueName: \"kubernetes.io/projected/a357283c-3904-4eca-ba02-09775a896ce8-kube-api-access-q97dt\") pod \"auth-test-model-4a9cb4-kserve-78dbd6c755-mkjfx\" (UID: \"a357283c-3904-4eca-ba02-09775a896ce8\") " pod="ai-tenant-e2e-shared-b-4a9cb4/auth-test-model-4a9cb4-kserve-78dbd6c755-mkjfx" Jul 08 19:39:22.304796 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:39:22.304800 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a357283c-3904-4eca-ba02-09775a896ce8-tmp-dir\") pod \"auth-test-model-4a9cb4-kserve-78dbd6c755-mkjfx\" (UID: \"a357283c-3904-4eca-ba02-09775a896ce8\") " pod="ai-tenant-e2e-shared-b-4a9cb4/auth-test-model-4a9cb4-kserve-78dbd6c755-mkjfx" Jul 08 19:39:22.305081 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:39:22.304832 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a357283c-3904-4eca-ba02-09775a896ce8-model-cache\") pod \"auth-test-model-4a9cb4-kserve-78dbd6c755-mkjfx\" (UID: \"a357283c-3904-4eca-ba02-09775a896ce8\") " pod="ai-tenant-e2e-shared-b-4a9cb4/auth-test-model-4a9cb4-kserve-78dbd6c755-mkjfx" Jul 08 19:39:22.305081 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:39:22.304850 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a357283c-3904-4eca-ba02-09775a896ce8-home\") pod \"auth-test-model-4a9cb4-kserve-78dbd6c755-mkjfx\" (UID: \"a357283c-3904-4eca-ba02-09775a896ce8\") " pod="ai-tenant-e2e-shared-b-4a9cb4/auth-test-model-4a9cb4-kserve-78dbd6c755-mkjfx" Jul 08 19:39:22.305081 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:39:22.304882 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a357283c-3904-4eca-ba02-09775a896ce8-dshm\") pod \"auth-test-model-4a9cb4-kserve-78dbd6c755-mkjfx\" (UID: \"a357283c-3904-4eca-ba02-09775a896ce8\") " pod="ai-tenant-e2e-shared-b-4a9cb4/auth-test-model-4a9cb4-kserve-78dbd6c755-mkjfx" Jul 08 19:39:22.305081 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:39:22.304917 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a357283c-3904-4eca-ba02-09775a896ce8-tls-certs\") pod \"auth-test-model-4a9cb4-kserve-78dbd6c755-mkjfx\" (UID: \"a357283c-3904-4eca-ba02-09775a896ce8\") " pod="ai-tenant-e2e-shared-b-4a9cb4/auth-test-model-4a9cb4-kserve-78dbd6c755-mkjfx" Jul 08 19:39:22.305341 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:39:22.305319 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a357283c-3904-4eca-ba02-09775a896ce8-tmp-dir\") pod \"auth-test-model-4a9cb4-kserve-78dbd6c755-mkjfx\" (UID: \"a357283c-3904-4eca-ba02-09775a896ce8\") " pod="ai-tenant-e2e-shared-b-4a9cb4/auth-test-model-4a9cb4-kserve-78dbd6c755-mkjfx" Jul 08 19:39:22.305407 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:39:22.305361 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a357283c-3904-4eca-ba02-09775a896ce8-model-cache\") pod \"auth-test-model-4a9cb4-kserve-78dbd6c755-mkjfx\" (UID: \"a357283c-3904-4eca-ba02-09775a896ce8\") " pod="ai-tenant-e2e-shared-b-4a9cb4/auth-test-model-4a9cb4-kserve-78dbd6c755-mkjfx" Jul 08 19:39:22.305407 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:39:22.305371 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a357283c-3904-4eca-ba02-09775a896ce8-home\") pod \"auth-test-model-4a9cb4-kserve-78dbd6c755-mkjfx\" (UID: \"a357283c-3904-4eca-ba02-09775a896ce8\") " pod="ai-tenant-e2e-shared-b-4a9cb4/auth-test-model-4a9cb4-kserve-78dbd6c755-mkjfx" Jul 08 19:39:22.307401 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:39:22.307367 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a357283c-3904-4eca-ba02-09775a896ce8-dshm\") pod \"auth-test-model-4a9cb4-kserve-78dbd6c755-mkjfx\" (UID: \"a357283c-3904-4eca-ba02-09775a896ce8\") " pod="ai-tenant-e2e-shared-b-4a9cb4/auth-test-model-4a9cb4-kserve-78dbd6c755-mkjfx" Jul 08 19:39:22.307549 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:39:22.307526 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a357283c-3904-4eca-ba02-09775a896ce8-tls-certs\") pod \"auth-test-model-4a9cb4-kserve-78dbd6c755-mkjfx\" (UID: \"a357283c-3904-4eca-ba02-09775a896ce8\") " pod="ai-tenant-e2e-shared-b-4a9cb4/auth-test-model-4a9cb4-kserve-78dbd6c755-mkjfx" Jul 08 19:39:22.312923 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:39:22.312900 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q97dt\" (UniqueName: \"kubernetes.io/projected/a357283c-3904-4eca-ba02-09775a896ce8-kube-api-access-q97dt\") pod \"auth-test-model-4a9cb4-kserve-78dbd6c755-mkjfx\" (UID: \"a357283c-3904-4eca-ba02-09775a896ce8\") " pod="ai-tenant-e2e-shared-b-4a9cb4/auth-test-model-4a9cb4-kserve-78dbd6c755-mkjfx" Jul 08 19:39:22.423957 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:39:22.423920 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4a9cb4/auth-test-model-4a9cb4-kserve-78dbd6c755-mkjfx" Jul 08 19:39:22.571089 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:39:22.571059 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-4a9cb4/auth-test-model-4a9cb4-kserve-78dbd6c755-mkjfx"] Jul 08 19:39:22.572735 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:39:22.572702 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda357283c_3904_4eca_ba02_09775a896ce8.slice/crio-0c2aa2904631a7a601f4ec75c734b78f21acda40cd3062bc84f0d5ef1182b118 WatchSource:0}: Error finding container 0c2aa2904631a7a601f4ec75c734b78f21acda40cd3062bc84f0d5ef1182b118: Status 404 returned error can't find the container with id 0c2aa2904631a7a601f4ec75c734b78f21acda40cd3062bc84f0d5ef1182b118 Jul 08 19:39:22.779716 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:39:22.779628 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4a9cb4/auth-test-model-4a9cb4-kserve-78dbd6c755-mkjfx" event={"ID":"a357283c-3904-4eca-ba02-09775a896ce8","Type":"ContainerStarted","Data":"cc9282d7fce8f6ae3e4620ddfbbcb491a59a202316e90b23c561e8be24768d1d"} Jul 08 19:39:22.779716 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:39:22.779671 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4a9cb4/auth-test-model-4a9cb4-kserve-78dbd6c755-mkjfx" event={"ID":"a357283c-3904-4eca-ba02-09775a896ce8","Type":"ContainerStarted","Data":"0c2aa2904631a7a601f4ec75c734b78f21acda40cd3062bc84f0d5ef1182b118"} Jul 08 19:39:22.797023 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:39:22.796970 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-4a9cb4/auth-test-model-4a9cb4-kserve-78dbd6c755-mkjfx" podStartSLOduration=0.796955048 podStartE2EDuration="796.955048ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:39:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:39:22.796141603 +0000 UTC m=+2128.075229222" watchObservedRunningTime="2026-07-08 19:39:22.796955048 +0000 UTC m=+2128.076042668" Jul 08 19:39:32.424209 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:39:32.424168 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-4a9cb4/auth-test-model-4a9cb4-kserve-78dbd6c755-mkjfx" Jul 08 19:39:32.424657 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:39:32.424267 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-4a9cb4/auth-test-model-4a9cb4-kserve-78dbd6c755-mkjfx" Jul 08 19:39:32.436674 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:39:32.436644 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-4a9cb4/auth-test-model-4a9cb4-kserve-78dbd6c755-mkjfx" Jul 08 19:39:32.838689 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:39:32.838603 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-4a9cb4/auth-test-model-4a9cb4-kserve-78dbd6c755-mkjfx" Jul 08 19:41:25.917069 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:41:25.916970 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-4a9cb4/auth-test-model-4a9cb4-kserve-78dbd6c755-mkjfx"] Jul 08 19:41:40.937996 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:41:40.937952 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-4a9cb4/auth-test-model-4a9cb4-kserve-78dbd6c755-mkjfx" podUID="a357283c-3904-4eca-ba02-09775a896ce8" containerName="main" containerID="cri-o://cc9282d7fce8f6ae3e4620ddfbbcb491a59a202316e90b23c561e8be24768d1d" gracePeriod=45 Jul 08 19:41:41.198558 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:41:41.198494 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4a9cb4/auth-test-model-4a9cb4-kserve-78dbd6c755-mkjfx" Jul 08 19:41:41.328651 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:41:41.328605 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a357283c-3904-4eca-ba02-09775a896ce8-tls-certs\") pod \"a357283c-3904-4eca-ba02-09775a896ce8\" (UID: \"a357283c-3904-4eca-ba02-09775a896ce8\") " Jul 08 19:41:41.328651 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:41:41.328660 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a357283c-3904-4eca-ba02-09775a896ce8-dshm\") pod \"a357283c-3904-4eca-ba02-09775a896ce8\" (UID: \"a357283c-3904-4eca-ba02-09775a896ce8\") " Jul 08 19:41:41.328651 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:41:41.328684 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a357283c-3904-4eca-ba02-09775a896ce8-tmp-dir\") pod \"a357283c-3904-4eca-ba02-09775a896ce8\" (UID: \"a357283c-3904-4eca-ba02-09775a896ce8\") " Jul 08 19:41:41.328990 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:41:41.328705 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a357283c-3904-4eca-ba02-09775a896ce8-home\") pod \"a357283c-3904-4eca-ba02-09775a896ce8\" (UID: \"a357283c-3904-4eca-ba02-09775a896ce8\") " Jul 08 19:41:41.328990 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:41:41.328763 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q97dt\" (UniqueName: \"kubernetes.io/projected/a357283c-3904-4eca-ba02-09775a896ce8-kube-api-access-q97dt\") pod \"a357283c-3904-4eca-ba02-09775a896ce8\" (UID: \"a357283c-3904-4eca-ba02-09775a896ce8\") " Jul 08 19:41:41.328990 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:41:41.328792 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a357283c-3904-4eca-ba02-09775a896ce8-model-cache\") pod \"a357283c-3904-4eca-ba02-09775a896ce8\" (UID: \"a357283c-3904-4eca-ba02-09775a896ce8\") " Jul 08 19:41:41.329139 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:41:41.328983 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a357283c-3904-4eca-ba02-09775a896ce8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a357283c-3904-4eca-ba02-09775a896ce8" (UID: "a357283c-3904-4eca-ba02-09775a896ce8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:41:41.329139 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:41:41.329058 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a357283c-3904-4eca-ba02-09775a896ce8-home" (OuterVolumeSpecName: "home") pod "a357283c-3904-4eca-ba02-09775a896ce8" (UID: "a357283c-3904-4eca-ba02-09775a896ce8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:41:41.329244 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:41:41.329162 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a357283c-3904-4eca-ba02-09775a896ce8-model-cache" (OuterVolumeSpecName: "model-cache") pod "a357283c-3904-4eca-ba02-09775a896ce8" (UID: "a357283c-3904-4eca-ba02-09775a896ce8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:41:41.331141 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:41:41.331105 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a357283c-3904-4eca-ba02-09775a896ce8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a357283c-3904-4eca-ba02-09775a896ce8" (UID: "a357283c-3904-4eca-ba02-09775a896ce8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:41:41.331256 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:41:41.331169 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a357283c-3904-4eca-ba02-09775a896ce8-kube-api-access-q97dt" (OuterVolumeSpecName: "kube-api-access-q97dt") pod "a357283c-3904-4eca-ba02-09775a896ce8" (UID: "a357283c-3904-4eca-ba02-09775a896ce8"). InnerVolumeSpecName "kube-api-access-q97dt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:41:41.331256 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:41:41.331182 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a357283c-3904-4eca-ba02-09775a896ce8-dshm" (OuterVolumeSpecName: "dshm") pod "a357283c-3904-4eca-ba02-09775a896ce8" (UID: "a357283c-3904-4eca-ba02-09775a896ce8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:41:41.430405 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:41:41.430368 2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a357283c-3904-4eca-ba02-09775a896ce8-tls-certs\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:41:41.430405 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:41:41.430399 2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a357283c-3904-4eca-ba02-09775a896ce8-dshm\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:41:41.430405 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:41:41.430407 2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a357283c-3904-4eca-ba02-09775a896ce8-tmp-dir\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:41:41.430405 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:41:41.430415 2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a357283c-3904-4eca-ba02-09775a896ce8-home\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:41:41.430695 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:41:41.430423 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q97dt\" (UniqueName: \"kubernetes.io/projected/a357283c-3904-4eca-ba02-09775a896ce8-kube-api-access-q97dt\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:41:41.430695 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:41:41.430464 2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a357283c-3904-4eca-ba02-09775a896ce8-model-cache\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:41:41.453408 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:41:41.453311 2583 generic.go:356] "Generic (PLEG): container finished" podID="a357283c-3904-4eca-ba02-09775a896ce8" containerID="cc9282d7fce8f6ae3e4620ddfbbcb491a59a202316e90b23c561e8be24768d1d" exitCode=0 Jul 08 19:41:41.453408 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:41:41.453379 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4a9cb4/auth-test-model-4a9cb4-kserve-78dbd6c755-mkjfx" event={"ID":"a357283c-3904-4eca-ba02-09775a896ce8","Type":"ContainerDied","Data":"cc9282d7fce8f6ae3e4620ddfbbcb491a59a202316e90b23c561e8be24768d1d"} Jul 08 19:41:41.453408 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:41:41.453390 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-4a9cb4/auth-test-model-4a9cb4-kserve-78dbd6c755-mkjfx" Jul 08 19:41:41.453408 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:41:41.453411 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-4a9cb4/auth-test-model-4a9cb4-kserve-78dbd6c755-mkjfx" event={"ID":"a357283c-3904-4eca-ba02-09775a896ce8","Type":"ContainerDied","Data":"0c2aa2904631a7a601f4ec75c734b78f21acda40cd3062bc84f0d5ef1182b118"} Jul 08 19:41:41.453782 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:41:41.453427 2583 scope.go:117] "RemoveContainer" containerID="cc9282d7fce8f6ae3e4620ddfbbcb491a59a202316e90b23c561e8be24768d1d" Jul 08 19:41:41.463369 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:41:41.463349 2583 scope.go:117] "RemoveContainer" containerID="cc9282d7fce8f6ae3e4620ddfbbcb491a59a202316e90b23c561e8be24768d1d" Jul 08 19:41:41.463699 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:41:41.463679 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc9282d7fce8f6ae3e4620ddfbbcb491a59a202316e90b23c561e8be24768d1d\": container with ID starting with cc9282d7fce8f6ae3e4620ddfbbcb491a59a202316e90b23c561e8be24768d1d not found: ID does not exist" containerID="cc9282d7fce8f6ae3e4620ddfbbcb491a59a202316e90b23c561e8be24768d1d" Jul 08 19:41:41.463796 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:41:41.463707 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc9282d7fce8f6ae3e4620ddfbbcb491a59a202316e90b23c561e8be24768d1d"} err="failed to get container status \"cc9282d7fce8f6ae3e4620ddfbbcb491a59a202316e90b23c561e8be24768d1d\": rpc error: code = NotFound desc = could not find container \"cc9282d7fce8f6ae3e4620ddfbbcb491a59a202316e90b23c561e8be24768d1d\": container with ID starting with cc9282d7fce8f6ae3e4620ddfbbcb491a59a202316e90b23c561e8be24768d1d not found: ID does not exist" Jul 08 19:41:41.476455 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:41:41.476397 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-4a9cb4/auth-test-model-4a9cb4-kserve-78dbd6c755-mkjfx"] Jul 08 19:41:41.478348 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:41:41.478323 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-4a9cb4/auth-test-model-4a9cb4-kserve-78dbd6c755-mkjfx"] Jul 08 19:41:43.175067 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:41:43.175031 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a357283c-3904-4eca-ba02-09775a896ce8" path="/var/lib/kubelet/pods/a357283c-3904-4eca-ba02-09775a896ce8/volumes" Jul 08 19:42:59.413495 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:42:59.413410 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-d86987/rate-test-model-d86987-kserve-76c9cb5759-4ms44"] Jul 08 19:42:59.413937 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:42:59.413766 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a357283c-3904-4eca-ba02-09775a896ce8" containerName="main" Jul 08 19:42:59.413937 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:42:59.413777 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a357283c-3904-4eca-ba02-09775a896ce8" containerName="main" Jul 08 19:42:59.413937 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:42:59.413862 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="a357283c-3904-4eca-ba02-09775a896ce8" containerName="main" Jul 08 19:42:59.416813 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:42:59.416795 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d86987/rate-test-model-d86987-kserve-76c9cb5759-4ms44" Jul 08 19:42:59.419683 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:42:59.419656 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-d86987\"/\"kube-root-ca.crt\"" Jul 08 19:42:59.419832 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:42:59.419712 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-d86987\"/\"rate-test-model-d86987-kserve-self-signed-certs\"" Jul 08 19:42:59.420496 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:42:59.420477 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-d86987\"/\"openshift-service-ca.crt\"" Jul 08 19:42:59.420557 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:42:59.420504 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-d86987\"/\"default-dockercfg-pbqfh\"" Jul 08 19:42:59.426514 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:42:59.426490 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-d86987/rate-test-model-d86987-kserve-76c9cb5759-4ms44"] Jul 08 19:42:59.445587 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:42:59.445559 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2180fb32-89e4-4b70-9605-52dcef09eb6f-dshm\") pod \"rate-test-model-d86987-kserve-76c9cb5759-4ms44\" (UID: \"2180fb32-89e4-4b70-9605-52dcef09eb6f\") " pod="ai-tenant-e2e-shared-a-d86987/rate-test-model-d86987-kserve-76c9cb5759-4ms44" Jul 08 19:42:59.445750 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:42:59.445616 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2180fb32-89e4-4b70-9605-52dcef09eb6f-tls-certs\") pod \"rate-test-model-d86987-kserve-76c9cb5759-4ms44\" (UID: \"2180fb32-89e4-4b70-9605-52dcef09eb6f\") " pod="ai-tenant-e2e-shared-a-d86987/rate-test-model-d86987-kserve-76c9cb5759-4ms44" Jul 08 19:42:59.445750 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:42:59.445635 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2180fb32-89e4-4b70-9605-52dcef09eb6f-tmp-dir\") pod \"rate-test-model-d86987-kserve-76c9cb5759-4ms44\" (UID: \"2180fb32-89e4-4b70-9605-52dcef09eb6f\") " pod="ai-tenant-e2e-shared-a-d86987/rate-test-model-d86987-kserve-76c9cb5759-4ms44" Jul 08 19:42:59.445750 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:42:59.445695 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2180fb32-89e4-4b70-9605-52dcef09eb6f-model-cache\") pod \"rate-test-model-d86987-kserve-76c9cb5759-4ms44\" (UID: \"2180fb32-89e4-4b70-9605-52dcef09eb6f\") " pod="ai-tenant-e2e-shared-a-d86987/rate-test-model-d86987-kserve-76c9cb5759-4ms44" Jul 08 19:42:59.445859 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:42:59.445763 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2180fb32-89e4-4b70-9605-52dcef09eb6f-home\") pod \"rate-test-model-d86987-kserve-76c9cb5759-4ms44\" (UID: \"2180fb32-89e4-4b70-9605-52dcef09eb6f\") " pod="ai-tenant-e2e-shared-a-d86987/rate-test-model-d86987-kserve-76c9cb5759-4ms44" Jul 08 19:42:59.445859 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:42:59.445785 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhvg7\" (UniqueName: \"kubernetes.io/projected/2180fb32-89e4-4b70-9605-52dcef09eb6f-kube-api-access-lhvg7\") pod \"rate-test-model-d86987-kserve-76c9cb5759-4ms44\" (UID: \"2180fb32-89e4-4b70-9605-52dcef09eb6f\") " pod="ai-tenant-e2e-shared-a-d86987/rate-test-model-d86987-kserve-76c9cb5759-4ms44" Jul 08 19:42:59.547182 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:42:59.547137 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2180fb32-89e4-4b70-9605-52dcef09eb6f-dshm\") pod \"rate-test-model-d86987-kserve-76c9cb5759-4ms44\" (UID: \"2180fb32-89e4-4b70-9605-52dcef09eb6f\") " pod="ai-tenant-e2e-shared-a-d86987/rate-test-model-d86987-kserve-76c9cb5759-4ms44" Jul 08 19:42:59.547371 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:42:59.547238 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2180fb32-89e4-4b70-9605-52dcef09eb6f-tls-certs\") pod \"rate-test-model-d86987-kserve-76c9cb5759-4ms44\" (UID: \"2180fb32-89e4-4b70-9605-52dcef09eb6f\") " pod="ai-tenant-e2e-shared-a-d86987/rate-test-model-d86987-kserve-76c9cb5759-4ms44" Jul 08 19:42:59.547371 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:42:59.547266 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2180fb32-89e4-4b70-9605-52dcef09eb6f-tmp-dir\") pod \"rate-test-model-d86987-kserve-76c9cb5759-4ms44\" (UID: \"2180fb32-89e4-4b70-9605-52dcef09eb6f\") " pod="ai-tenant-e2e-shared-a-d86987/rate-test-model-d86987-kserve-76c9cb5759-4ms44" Jul 08 19:42:59.547371 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:42:59.547304 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2180fb32-89e4-4b70-9605-52dcef09eb6f-model-cache\") pod \"rate-test-model-d86987-kserve-76c9cb5759-4ms44\" (UID: \"2180fb32-89e4-4b70-9605-52dcef09eb6f\") " pod="ai-tenant-e2e-shared-a-d86987/rate-test-model-d86987-kserve-76c9cb5759-4ms44" Jul 08 19:42:59.547371 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:42:59.547365 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2180fb32-89e4-4b70-9605-52dcef09eb6f-home\") pod \"rate-test-model-d86987-kserve-76c9cb5759-4ms44\" (UID: \"2180fb32-89e4-4b70-9605-52dcef09eb6f\") " pod="ai-tenant-e2e-shared-a-d86987/rate-test-model-d86987-kserve-76c9cb5759-4ms44" Jul 08 19:42:59.547655 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:42:59.547389 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhvg7\" (UniqueName: \"kubernetes.io/projected/2180fb32-89e4-4b70-9605-52dcef09eb6f-kube-api-access-lhvg7\") pod \"rate-test-model-d86987-kserve-76c9cb5759-4ms44\" (UID: \"2180fb32-89e4-4b70-9605-52dcef09eb6f\") " pod="ai-tenant-e2e-shared-a-d86987/rate-test-model-d86987-kserve-76c9cb5759-4ms44" Jul 08 19:42:59.548201 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:42:59.548174 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2180fb32-89e4-4b70-9605-52dcef09eb6f-model-cache\") pod \"rate-test-model-d86987-kserve-76c9cb5759-4ms44\" (UID: \"2180fb32-89e4-4b70-9605-52dcef09eb6f\") " pod="ai-tenant-e2e-shared-a-d86987/rate-test-model-d86987-kserve-76c9cb5759-4ms44" Jul 08 19:42:59.548356 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:42:59.548229 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2180fb32-89e4-4b70-9605-52dcef09eb6f-home\") pod \"rate-test-model-d86987-kserve-76c9cb5759-4ms44\" (UID: \"2180fb32-89e4-4b70-9605-52dcef09eb6f\") " pod="ai-tenant-e2e-shared-a-d86987/rate-test-model-d86987-kserve-76c9cb5759-4ms44" Jul 08 19:42:59.548505 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:42:59.548480 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2180fb32-89e4-4b70-9605-52dcef09eb6f-tmp-dir\") pod \"rate-test-model-d86987-kserve-76c9cb5759-4ms44\" (UID: \"2180fb32-89e4-4b70-9605-52dcef09eb6f\") " pod="ai-tenant-e2e-shared-a-d86987/rate-test-model-d86987-kserve-76c9cb5759-4ms44" Jul 08 19:42:59.550559 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:42:59.550536 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2180fb32-89e4-4b70-9605-52dcef09eb6f-tls-certs\") pod \"rate-test-model-d86987-kserve-76c9cb5759-4ms44\" (UID: \"2180fb32-89e4-4b70-9605-52dcef09eb6f\") " pod="ai-tenant-e2e-shared-a-d86987/rate-test-model-d86987-kserve-76c9cb5759-4ms44" Jul 08 19:42:59.550825 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:42:59.550802 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2180fb32-89e4-4b70-9605-52dcef09eb6f-dshm\") pod \"rate-test-model-d86987-kserve-76c9cb5759-4ms44\" (UID: \"2180fb32-89e4-4b70-9605-52dcef09eb6f\") " pod="ai-tenant-e2e-shared-a-d86987/rate-test-model-d86987-kserve-76c9cb5759-4ms44" Jul 08 19:42:59.555200 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:42:59.555179 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhvg7\" (UniqueName: \"kubernetes.io/projected/2180fb32-89e4-4b70-9605-52dcef09eb6f-kube-api-access-lhvg7\") pod \"rate-test-model-d86987-kserve-76c9cb5759-4ms44\" (UID: \"2180fb32-89e4-4b70-9605-52dcef09eb6f\") " pod="ai-tenant-e2e-shared-a-d86987/rate-test-model-d86987-kserve-76c9cb5759-4ms44" Jul 08 19:42:59.727467 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:42:59.727330 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d86987/rate-test-model-d86987-kserve-76c9cb5759-4ms44" Jul 08 19:42:59.866121 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:42:59.866094 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-d86987/rate-test-model-d86987-kserve-76c9cb5759-4ms44"] Jul 08 19:42:59.867847 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:42:59.867816 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2180fb32_89e4_4b70_9605_52dcef09eb6f.slice/crio-30fcc42481fba33a1137a6ce2e9849c79d57ef77ad47d242c4bbab54077860d1 WatchSource:0}: Error finding container 30fcc42481fba33a1137a6ce2e9849c79d57ef77ad47d242c4bbab54077860d1: Status 404 returned error can't find the container with id 30fcc42481fba33a1137a6ce2e9849c79d57ef77ad47d242c4bbab54077860d1 Jul 08 19:43:00.827600 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:43:00.827562 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d86987/rate-test-model-d86987-kserve-76c9cb5759-4ms44" event={"ID":"2180fb32-89e4-4b70-9605-52dcef09eb6f","Type":"ContainerStarted","Data":"a2e52742d92c30a241f82631ff567e6ab66b3634361ec986a7f1be5a9dbbcac5"} Jul 08 19:43:00.827600 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:43:00.827608 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d86987/rate-test-model-d86987-kserve-76c9cb5759-4ms44" event={"ID":"2180fb32-89e4-4b70-9605-52dcef09eb6f","Type":"ContainerStarted","Data":"30fcc42481fba33a1137a6ce2e9849c79d57ef77ad47d242c4bbab54077860d1"} Jul 08 19:43:00.846003 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:43:00.845948 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-d86987/rate-test-model-d86987-kserve-76c9cb5759-4ms44" podStartSLOduration=1.84593295 podStartE2EDuration="1.84593295s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:42:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:43:00.843812265 +0000 UTC m=+2346.122899908" watchObservedRunningTime="2026-07-08 19:43:00.84593295 +0000 UTC m=+2346.125020569" Jul 08 19:43:09.728454 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:43:09.728385 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-d86987/rate-test-model-d86987-kserve-76c9cb5759-4ms44" Jul 08 19:43:09.729195 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:43:09.728467 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-d86987/rate-test-model-d86987-kserve-76c9cb5759-4ms44" Jul 08 19:43:09.741043 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:43:09.741013 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-d86987/rate-test-model-d86987-kserve-76c9cb5759-4ms44" Jul 08 19:43:09.894987 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:43:09.894956 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-d86987/rate-test-model-d86987-kserve-76c9cb5759-4ms44" Jul 08 19:44:31.036615 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:44:31.036393 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-d86987/rate-test-model-d86987-kserve-76c9cb5759-4ms44"] Jul 08 19:44:46.061175 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:44:46.061107 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-d86987/rate-test-model-d86987-kserve-76c9cb5759-4ms44" podUID="2180fb32-89e4-4b70-9605-52dcef09eb6f" containerName="main" containerID="cri-o://a2e52742d92c30a241f82631ff567e6ab66b3634361ec986a7f1be5a9dbbcac5" gracePeriod=45 Jul 08 19:44:46.314167 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:44:46.314103 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d86987/rate-test-model-d86987-kserve-76c9cb5759-4ms44" Jul 08 19:44:46.333234 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:44:46.333181 2583 generic.go:356] "Generic (PLEG): container finished" podID="2180fb32-89e4-4b70-9605-52dcef09eb6f" containerID="a2e52742d92c30a241f82631ff567e6ab66b3634361ec986a7f1be5a9dbbcac5" exitCode=0 Jul 08 19:44:46.333421 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:44:46.333240 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d86987/rate-test-model-d86987-kserve-76c9cb5759-4ms44" Jul 08 19:44:46.333421 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:44:46.333319 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d86987/rate-test-model-d86987-kserve-76c9cb5759-4ms44" event={"ID":"2180fb32-89e4-4b70-9605-52dcef09eb6f","Type":"ContainerDied","Data":"a2e52742d92c30a241f82631ff567e6ab66b3634361ec986a7f1be5a9dbbcac5"} Jul 08 19:44:46.333421 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:44:46.333370 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d86987/rate-test-model-d86987-kserve-76c9cb5759-4ms44" event={"ID":"2180fb32-89e4-4b70-9605-52dcef09eb6f","Type":"ContainerDied","Data":"30fcc42481fba33a1137a6ce2e9849c79d57ef77ad47d242c4bbab54077860d1"} Jul 08 19:44:46.333421 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:44:46.333392 2583 scope.go:117] "RemoveContainer" containerID="a2e52742d92c30a241f82631ff567e6ab66b3634361ec986a7f1be5a9dbbcac5" Jul 08 19:44:46.343285 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:44:46.343259 2583 scope.go:117] "RemoveContainer" containerID="a2e52742d92c30a241f82631ff567e6ab66b3634361ec986a7f1be5a9dbbcac5" Jul 08 19:44:46.343640 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:44:46.343616 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2e52742d92c30a241f82631ff567e6ab66b3634361ec986a7f1be5a9dbbcac5\": container with ID starting with a2e52742d92c30a241f82631ff567e6ab66b3634361ec986a7f1be5a9dbbcac5 not found: ID does not exist" containerID="a2e52742d92c30a241f82631ff567e6ab66b3634361ec986a7f1be5a9dbbcac5" Jul 08 19:44:46.343711 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:44:46.343653 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2e52742d92c30a241f82631ff567e6ab66b3634361ec986a7f1be5a9dbbcac5"} err="failed to get container status \"a2e52742d92c30a241f82631ff567e6ab66b3634361ec986a7f1be5a9dbbcac5\": rpc error: code = NotFound desc = could not find container \"a2e52742d92c30a241f82631ff567e6ab66b3634361ec986a7f1be5a9dbbcac5\": container with ID starting with a2e52742d92c30a241f82631ff567e6ab66b3634361ec986a7f1be5a9dbbcac5 not found: ID does not exist" Jul 08 19:44:46.431179 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:44:46.431144 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2180fb32-89e4-4b70-9605-52dcef09eb6f-model-cache\") pod \"2180fb32-89e4-4b70-9605-52dcef09eb6f\" (UID: \"2180fb32-89e4-4b70-9605-52dcef09eb6f\") " Jul 08 19:44:46.431390 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:44:46.431211 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lhvg7\" (UniqueName: \"kubernetes.io/projected/2180fb32-89e4-4b70-9605-52dcef09eb6f-kube-api-access-lhvg7\") pod \"2180fb32-89e4-4b70-9605-52dcef09eb6f\" (UID: \"2180fb32-89e4-4b70-9605-52dcef09eb6f\") " Jul 08 19:44:46.431390 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:44:46.431280 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2180fb32-89e4-4b70-9605-52dcef09eb6f-home\") pod \"2180fb32-89e4-4b70-9605-52dcef09eb6f\" (UID: \"2180fb32-89e4-4b70-9605-52dcef09eb6f\") " Jul 08 19:44:46.431390 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:44:46.431306 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2180fb32-89e4-4b70-9605-52dcef09eb6f-tmp-dir\") pod \"2180fb32-89e4-4b70-9605-52dcef09eb6f\" (UID: \"2180fb32-89e4-4b70-9605-52dcef09eb6f\") " Jul 08 19:44:46.431390 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:44:46.431330 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2180fb32-89e4-4b70-9605-52dcef09eb6f-dshm\") pod \"2180fb32-89e4-4b70-9605-52dcef09eb6f\" (UID: \"2180fb32-89e4-4b70-9605-52dcef09eb6f\") " Jul 08 19:44:46.431653 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:44:46.431410 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2180fb32-89e4-4b70-9605-52dcef09eb6f-tls-certs\") pod \"2180fb32-89e4-4b70-9605-52dcef09eb6f\" (UID: \"2180fb32-89e4-4b70-9605-52dcef09eb6f\") " Jul 08 19:44:46.431653 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:44:46.431490 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2180fb32-89e4-4b70-9605-52dcef09eb6f-model-cache" (OuterVolumeSpecName: "model-cache") pod "2180fb32-89e4-4b70-9605-52dcef09eb6f" (UID: "2180fb32-89e4-4b70-9605-52dcef09eb6f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:44:46.431653 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:44:46.431612 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2180fb32-89e4-4b70-9605-52dcef09eb6f-home" (OuterVolumeSpecName: "home") pod "2180fb32-89e4-4b70-9605-52dcef09eb6f" (UID: "2180fb32-89e4-4b70-9605-52dcef09eb6f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:44:46.431653 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:44:46.431624 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2180fb32-89e4-4b70-9605-52dcef09eb6f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2180fb32-89e4-4b70-9605-52dcef09eb6f" (UID: "2180fb32-89e4-4b70-9605-52dcef09eb6f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:44:46.431856 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:44:46.431752 2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2180fb32-89e4-4b70-9605-52dcef09eb6f-home\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:44:46.431856 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:44:46.431767 2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2180fb32-89e4-4b70-9605-52dcef09eb6f-tmp-dir\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:44:46.431856 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:44:46.431780 2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2180fb32-89e4-4b70-9605-52dcef09eb6f-model-cache\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:44:46.433716 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:44:46.433684 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2180fb32-89e4-4b70-9605-52dcef09eb6f-kube-api-access-lhvg7" (OuterVolumeSpecName: "kube-api-access-lhvg7") pod "2180fb32-89e4-4b70-9605-52dcef09eb6f" (UID: "2180fb32-89e4-4b70-9605-52dcef09eb6f"). InnerVolumeSpecName "kube-api-access-lhvg7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:44:46.433822 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:44:46.433731 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2180fb32-89e4-4b70-9605-52dcef09eb6f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2180fb32-89e4-4b70-9605-52dcef09eb6f" (UID: "2180fb32-89e4-4b70-9605-52dcef09eb6f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:44:46.433822 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:44:46.433743 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2180fb32-89e4-4b70-9605-52dcef09eb6f-dshm" (OuterVolumeSpecName: "dshm") pod "2180fb32-89e4-4b70-9605-52dcef09eb6f" (UID: "2180fb32-89e4-4b70-9605-52dcef09eb6f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:44:46.532389 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:44:46.532355 2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2180fb32-89e4-4b70-9605-52dcef09eb6f-dshm\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:44:46.532389 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:44:46.532388 2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2180fb32-89e4-4b70-9605-52dcef09eb6f-tls-certs\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:44:46.532645 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:44:46.532403 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lhvg7\" (UniqueName: \"kubernetes.io/projected/2180fb32-89e4-4b70-9605-52dcef09eb6f-kube-api-access-lhvg7\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:44:46.654751 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:44:46.654720 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-d86987/rate-test-model-d86987-kserve-76c9cb5759-4ms44"] Jul 08 19:44:46.658452 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:44:46.658409 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-d86987/rate-test-model-d86987-kserve-76c9cb5759-4ms44"] Jul 08 19:44:47.174635 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:44:47.174602 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2180fb32-89e4-4b70-9605-52dcef09eb6f" path="/var/lib/kubelet/pods/2180fb32-89e4-4b70-9605-52dcef09eb6f/volumes" Jul 08 19:45:19.268003 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:19.267964 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs"] Jul 08 19:45:19.268581 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:19.268222 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs" podUID="b4a6ecf9-6c35-403a-b85f-28666300a263" containerName="istio-proxy" containerID="cri-o://cddbec3767f1680a339e01b49fbd7424f387783ed86a973030f424e6dced08d8" gracePeriod=30 Jul 08 19:45:24.517675 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:24.517648 2583 generic.go:356] "Generic (PLEG): container finished" podID="b4a6ecf9-6c35-403a-b85f-28666300a263" containerID="cddbec3767f1680a339e01b49fbd7424f387783ed86a973030f424e6dced08d8" exitCode=0 Jul 08 19:45:24.518023 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:24.517719 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs" event={"ID":"b4a6ecf9-6c35-403a-b85f-28666300a263","Type":"ContainerDied","Data":"cddbec3767f1680a339e01b49fbd7424f387783ed86a973030f424e6dced08d8"} Jul 08 19:45:24.518023 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:24.517784 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs" event={"ID":"b4a6ecf9-6c35-403a-b85f-28666300a263","Type":"ContainerDied","Data":"6d0e5307d8874777b55bffb551e19fc1ec2b211b51836f1486151476d32f5423"} Jul 08 19:45:24.518023 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:24.517806 2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6d0e5307d8874777b55bffb551e19fc1ec2b211b51836f1486151476d32f5423" Jul 08 19:45:24.518023 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:24.517820 2583 scope.go:117] "RemoveContainer" containerID="8d3df0e6719c7c3deafb7fbba129f972def2b470cf980dabac0b8704e8f46c32" Jul 08 19:45:24.533015 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:24.532995 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs" Jul 08 19:45:24.585084 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:24.585042 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4a6ecf9-6c35-403a-b85f-28666300a263-istiod-ca-cert\") pod \"b4a6ecf9-6c35-403a-b85f-28666300a263\" (UID: \"b4a6ecf9-6c35-403a-b85f-28666300a263\") " Jul 08 19:45:24.585084 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:24.585091 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4a6ecf9-6c35-403a-b85f-28666300a263-istio-podinfo\") pod \"b4a6ecf9-6c35-403a-b85f-28666300a263\" (UID: \"b4a6ecf9-6c35-403a-b85f-28666300a263\") " Jul 08 19:45:24.585330 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:24.585112 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4a6ecf9-6c35-403a-b85f-28666300a263-istio-data\") pod \"b4a6ecf9-6c35-403a-b85f-28666300a263\" (UID: \"b4a6ecf9-6c35-403a-b85f-28666300a263\") " Jul 08 19:45:24.585330 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:24.585152 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4a6ecf9-6c35-403a-b85f-28666300a263-workload-certs\") pod \"b4a6ecf9-6c35-403a-b85f-28666300a263\" (UID: \"b4a6ecf9-6c35-403a-b85f-28666300a263\") " Jul 08 19:45:24.585330 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:24.585183 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-db7jf\" (UniqueName: \"kubernetes.io/projected/b4a6ecf9-6c35-403a-b85f-28666300a263-kube-api-access-db7jf\") pod \"b4a6ecf9-6c35-403a-b85f-28666300a263\" (UID: \"b4a6ecf9-6c35-403a-b85f-28666300a263\") " Jul 08 19:45:24.585330 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:24.585243 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4a6ecf9-6c35-403a-b85f-28666300a263-workload-socket\") pod \"b4a6ecf9-6c35-403a-b85f-28666300a263\" (UID: \"b4a6ecf9-6c35-403a-b85f-28666300a263\") " Jul 08 19:45:24.585330 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:24.585269 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4a6ecf9-6c35-403a-b85f-28666300a263-istio-token\") pod \"b4a6ecf9-6c35-403a-b85f-28666300a263\" (UID: \"b4a6ecf9-6c35-403a-b85f-28666300a263\") " Jul 08 19:45:24.585330 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:24.585289 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4a6ecf9-6c35-403a-b85f-28666300a263-credential-socket\") pod \"b4a6ecf9-6c35-403a-b85f-28666300a263\" (UID: \"b4a6ecf9-6c35-403a-b85f-28666300a263\") " Jul 08 19:45:24.585330 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:24.585317 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4a6ecf9-6c35-403a-b85f-28666300a263-istio-envoy\") pod \"b4a6ecf9-6c35-403a-b85f-28666300a263\" (UID: \"b4a6ecf9-6c35-403a-b85f-28666300a263\") " Jul 08 19:45:24.585707 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:24.585548 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4a6ecf9-6c35-403a-b85f-28666300a263-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b4a6ecf9-6c35-403a-b85f-28666300a263" (UID: "b4a6ecf9-6c35-403a-b85f-28666300a263"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:45:24.585707 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:24.585632 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4a6ecf9-6c35-403a-b85f-28666300a263-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b4a6ecf9-6c35-403a-b85f-28666300a263" (UID: "b4a6ecf9-6c35-403a-b85f-28666300a263"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:45:24.585831 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:24.585798 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4a6ecf9-6c35-403a-b85f-28666300a263-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b4a6ecf9-6c35-403a-b85f-28666300a263" (UID: "b4a6ecf9-6c35-403a-b85f-28666300a263"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:45:24.585997 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:24.585962 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4a6ecf9-6c35-403a-b85f-28666300a263-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b4a6ecf9-6c35-403a-b85f-28666300a263" (UID: "b4a6ecf9-6c35-403a-b85f-28666300a263"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:45:24.586240 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:24.586063 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4a6ecf9-6c35-403a-b85f-28666300a263-istio-data" (OuterVolumeSpecName: "istio-data") pod "b4a6ecf9-6c35-403a-b85f-28666300a263" (UID: "b4a6ecf9-6c35-403a-b85f-28666300a263"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:45:24.587512 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:24.587489 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b4a6ecf9-6c35-403a-b85f-28666300a263-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b4a6ecf9-6c35-403a-b85f-28666300a263" (UID: "b4a6ecf9-6c35-403a-b85f-28666300a263"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 19:45:24.587608 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:24.587588 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4a6ecf9-6c35-403a-b85f-28666300a263-istio-token" (OuterVolumeSpecName: "istio-token") pod "b4a6ecf9-6c35-403a-b85f-28666300a263" (UID: "b4a6ecf9-6c35-403a-b85f-28666300a263"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:45:24.588124 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:24.588099 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4a6ecf9-6c35-403a-b85f-28666300a263-kube-api-access-db7jf" (OuterVolumeSpecName: "kube-api-access-db7jf") pod "b4a6ecf9-6c35-403a-b85f-28666300a263" (UID: "b4a6ecf9-6c35-403a-b85f-28666300a263"). InnerVolumeSpecName "kube-api-access-db7jf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:45:24.588224 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:24.588127 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4a6ecf9-6c35-403a-b85f-28666300a263-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b4a6ecf9-6c35-403a-b85f-28666300a263" (UID: "b4a6ecf9-6c35-403a-b85f-28666300a263"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:45:24.687003 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:24.686961 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4a6ecf9-6c35-403a-b85f-28666300a263-workload-socket\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:45:24.687003 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:24.686993 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4a6ecf9-6c35-403a-b85f-28666300a263-istio-token\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:45:24.687003 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:24.687007 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4a6ecf9-6c35-403a-b85f-28666300a263-credential-socket\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:45:24.687263 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:24.687018 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4a6ecf9-6c35-403a-b85f-28666300a263-istio-envoy\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:45:24.687263 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:24.687031 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4a6ecf9-6c35-403a-b85f-28666300a263-istiod-ca-cert\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:45:24.687263 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:24.687042 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4a6ecf9-6c35-403a-b85f-28666300a263-istio-podinfo\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:45:24.687263 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:24.687052 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4a6ecf9-6c35-403a-b85f-28666300a263-istio-data\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:45:24.687263 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:24.687062 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4a6ecf9-6c35-403a-b85f-28666300a263-workload-certs\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:45:24.687263 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:24.687073 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-db7jf\" (UniqueName: \"kubernetes.io/projected/b4a6ecf9-6c35-403a-b85f-28666300a263-kube-api-access-db7jf\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:45:25.524841 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:25.524761 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs" Jul 08 19:45:25.559255 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:25.559219 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs"] Jul 08 19:45:25.563104 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:25.563077 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-d86987-openshift-default-66fdf5cd87-rcgzs"] Jul 08 19:45:26.206757 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:26.206712 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt"] Jul 08 19:45:26.207111 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:26.207056 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt" podUID="a1273c7c-f18f-4459-8cec-a5770acff4b5" containerName="istio-proxy" containerID="cri-o://8e5aec4252da57700394711db4da537839c51480baa446d9b38ad70d510fb3df" gracePeriod=30 Jul 08 19:45:26.880835 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:26.880804 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-664b8dd44f-ffm98_65eee363-b056-4e91-8a78-dc9d141ad46c/authorino/0.log" Jul 08 19:45:27.175277 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:27.175242 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4a6ecf9-6c35-403a-b85f-28666300a263" path="/var/lib/kubelet/pods/b4a6ecf9-6c35-403a-b85f-28666300a263/volumes" Jul 08 19:45:31.449345 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:31.449320 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-xx469_7ac398a1-e561-4a15-9217-4ae93da132f9/manager/0.log" Jul 08 19:45:31.461421 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:31.461396 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt" Jul 08 19:45:31.550035 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:31.549935 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1273c7c-f18f-4459-8cec-a5770acff4b5-istio-token\") pod \"a1273c7c-f18f-4459-8cec-a5770acff4b5\" (UID: \"a1273c7c-f18f-4459-8cec-a5770acff4b5\") " Jul 08 19:45:31.550035 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:31.550002 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1273c7c-f18f-4459-8cec-a5770acff4b5-credential-socket\") pod \"a1273c7c-f18f-4459-8cec-a5770acff4b5\" (UID: \"a1273c7c-f18f-4459-8cec-a5770acff4b5\") " Jul 08 19:45:31.550035 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:31.550035 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1273c7c-f18f-4459-8cec-a5770acff4b5-workload-socket\") pod \"a1273c7c-f18f-4459-8cec-a5770acff4b5\" (UID: \"a1273c7c-f18f-4459-8cec-a5770acff4b5\") " Jul 08 19:45:31.550322 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:31.550059 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ng92h\" (UniqueName: \"kubernetes.io/projected/a1273c7c-f18f-4459-8cec-a5770acff4b5-kube-api-access-ng92h\") pod \"a1273c7c-f18f-4459-8cec-a5770acff4b5\" (UID: \"a1273c7c-f18f-4459-8cec-a5770acff4b5\") " Jul 08 19:45:31.550322 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:31.550085 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1273c7c-f18f-4459-8cec-a5770acff4b5-istio-data\") pod \"a1273c7c-f18f-4459-8cec-a5770acff4b5\" (UID: \"a1273c7c-f18f-4459-8cec-a5770acff4b5\") " Jul 08 19:45:31.550322 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:31.550124 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1273c7c-f18f-4459-8cec-a5770acff4b5-istiod-ca-cert\") pod \"a1273c7c-f18f-4459-8cec-a5770acff4b5\" (UID: \"a1273c7c-f18f-4459-8cec-a5770acff4b5\") " Jul 08 19:45:31.550322 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:31.550146 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1273c7c-f18f-4459-8cec-a5770acff4b5-istio-envoy\") pod \"a1273c7c-f18f-4459-8cec-a5770acff4b5\" (UID: \"a1273c7c-f18f-4459-8cec-a5770acff4b5\") " Jul 08 19:45:31.550322 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:31.550188 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1273c7c-f18f-4459-8cec-a5770acff4b5-istio-podinfo\") pod \"a1273c7c-f18f-4459-8cec-a5770acff4b5\" (UID: \"a1273c7c-f18f-4459-8cec-a5770acff4b5\") " Jul 08 19:45:31.550322 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:31.550260 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1273c7c-f18f-4459-8cec-a5770acff4b5-workload-certs\") pod \"a1273c7c-f18f-4459-8cec-a5770acff4b5\" (UID: \"a1273c7c-f18f-4459-8cec-a5770acff4b5\") " Jul 08 19:45:31.550668 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:31.550386 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1273c7c-f18f-4459-8cec-a5770acff4b5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a1273c7c-f18f-4459-8cec-a5770acff4b5" (UID: "a1273c7c-f18f-4459-8cec-a5770acff4b5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:45:31.550668 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:31.550543 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1273c7c-f18f-4459-8cec-a5770acff4b5-workload-socket\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:45:31.550668 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:31.550625 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1273c7c-f18f-4459-8cec-a5770acff4b5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a1273c7c-f18f-4459-8cec-a5770acff4b5" (UID: "a1273c7c-f18f-4459-8cec-a5770acff4b5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:45:31.550988 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:31.550876 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1273c7c-f18f-4459-8cec-a5770acff4b5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a1273c7c-f18f-4459-8cec-a5770acff4b5" (UID: "a1273c7c-f18f-4459-8cec-a5770acff4b5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:45:31.550988 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:31.550887 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1273c7c-f18f-4459-8cec-a5770acff4b5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a1273c7c-f18f-4459-8cec-a5770acff4b5" (UID: "a1273c7c-f18f-4459-8cec-a5770acff4b5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:45:31.551316 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:31.551272 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1273c7c-f18f-4459-8cec-a5770acff4b5-istio-data" (OuterVolumeSpecName: "istio-data") pod "a1273c7c-f18f-4459-8cec-a5770acff4b5" (UID: "a1273c7c-f18f-4459-8cec-a5770acff4b5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:45:31.552556 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:31.552528 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1273c7c-f18f-4459-8cec-a5770acff4b5-istio-token" (OuterVolumeSpecName: "istio-token") pod "a1273c7c-f18f-4459-8cec-a5770acff4b5" (UID: "a1273c7c-f18f-4459-8cec-a5770acff4b5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:45:31.552951 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:31.552916 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1273c7c-f18f-4459-8cec-a5770acff4b5-kube-api-access-ng92h" (OuterVolumeSpecName: "kube-api-access-ng92h") pod "a1273c7c-f18f-4459-8cec-a5770acff4b5" (UID: "a1273c7c-f18f-4459-8cec-a5770acff4b5"). InnerVolumeSpecName "kube-api-access-ng92h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:45:31.553144 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:31.553122 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1273c7c-f18f-4459-8cec-a5770acff4b5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a1273c7c-f18f-4459-8cec-a5770acff4b5" (UID: "a1273c7c-f18f-4459-8cec-a5770acff4b5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:45:31.553261 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:31.553183 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a1273c7c-f18f-4459-8cec-a5770acff4b5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a1273c7c-f18f-4459-8cec-a5770acff4b5" (UID: "a1273c7c-f18f-4459-8cec-a5770acff4b5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 19:45:31.559886 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:31.559859 2583 generic.go:356] "Generic (PLEG): container finished" podID="a1273c7c-f18f-4459-8cec-a5770acff4b5" containerID="8e5aec4252da57700394711db4da537839c51480baa446d9b38ad70d510fb3df" exitCode=0 Jul 08 19:45:31.559985 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:31.559910 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt" event={"ID":"a1273c7c-f18f-4459-8cec-a5770acff4b5","Type":"ContainerDied","Data":"8e5aec4252da57700394711db4da537839c51480baa446d9b38ad70d510fb3df"} Jul 08 19:45:31.559985 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:31.559952 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt" Jul 08 19:45:31.559985 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:31.559973 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt" event={"ID":"a1273c7c-f18f-4459-8cec-a5770acff4b5","Type":"ContainerDied","Data":"1392de99899d0c8d8ae9e1ac7ed41b17141ff0d45e227f221603195669302805"} Jul 08 19:45:31.560136 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:31.560004 2583 scope.go:117] "RemoveContainer" containerID="8e5aec4252da57700394711db4da537839c51480baa446d9b38ad70d510fb3df" Jul 08 19:45:31.569921 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:31.569902 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-sjflv_5d6c1088-bfef-4914-8c2e-d444040c89aa/manager/0.log" Jul 08 19:45:31.573662 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:31.573647 2583 scope.go:117] "RemoveContainer" containerID="357ccd2c7eaa29ec5f7c73034e146d23e59a9beceeb747dee59a4ad215f1513c" Jul 08 19:45:31.581445 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:31.581409 2583 scope.go:117] "RemoveContainer" containerID="8e5aec4252da57700394711db4da537839c51480baa446d9b38ad70d510fb3df" Jul 08 19:45:31.581733 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:45:31.581711 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e5aec4252da57700394711db4da537839c51480baa446d9b38ad70d510fb3df\": container with ID starting with 8e5aec4252da57700394711db4da537839c51480baa446d9b38ad70d510fb3df not found: ID does not exist" containerID="8e5aec4252da57700394711db4da537839c51480baa446d9b38ad70d510fb3df" Jul 08 19:45:31.581801 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:31.581749 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e5aec4252da57700394711db4da537839c51480baa446d9b38ad70d510fb3df"} err="failed to get container status \"8e5aec4252da57700394711db4da537839c51480baa446d9b38ad70d510fb3df\": rpc error: code = NotFound desc = could not find container \"8e5aec4252da57700394711db4da537839c51480baa446d9b38ad70d510fb3df\": container with ID starting with 8e5aec4252da57700394711db4da537839c51480baa446d9b38ad70d510fb3df not found: ID does not exist" Jul 08 19:45:31.581801 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:31.581770 2583 scope.go:117] "RemoveContainer" containerID="357ccd2c7eaa29ec5f7c73034e146d23e59a9beceeb747dee59a4ad215f1513c" Jul 08 19:45:31.581999 ip-10-0-137-244 kubenswrapper[2583]: E0708 19:45:31.581982 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"357ccd2c7eaa29ec5f7c73034e146d23e59a9beceeb747dee59a4ad215f1513c\": container with ID starting with 357ccd2c7eaa29ec5f7c73034e146d23e59a9beceeb747dee59a4ad215f1513c not found: ID does not exist" containerID="357ccd2c7eaa29ec5f7c73034e146d23e59a9beceeb747dee59a4ad215f1513c" Jul 08 19:45:31.582037 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:31.582005 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"357ccd2c7eaa29ec5f7c73034e146d23e59a9beceeb747dee59a4ad215f1513c"} err="failed to get container status \"357ccd2c7eaa29ec5f7c73034e146d23e59a9beceeb747dee59a4ad215f1513c\": rpc error: code = NotFound desc = could not find container \"357ccd2c7eaa29ec5f7c73034e146d23e59a9beceeb747dee59a4ad215f1513c\": container with ID starting with 357ccd2c7eaa29ec5f7c73034e146d23e59a9beceeb747dee59a4ad215f1513c not found: ID does not exist" Jul 08 19:45:31.585273 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:31.585250 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt"] Jul 08 19:45:31.588929 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:31.588909 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-4a9cb4-openshift-default-78867866c9-bwvjt"] Jul 08 19:45:31.651373 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:31.651330 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1273c7c-f18f-4459-8cec-a5770acff4b5-istiod-ca-cert\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:45:31.651373 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:31.651375 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1273c7c-f18f-4459-8cec-a5770acff4b5-istio-envoy\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:45:31.651609 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:31.651388 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1273c7c-f18f-4459-8cec-a5770acff4b5-istio-podinfo\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:45:31.651609 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:31.651401 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1273c7c-f18f-4459-8cec-a5770acff4b5-workload-certs\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:45:31.651609 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:31.651417 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1273c7c-f18f-4459-8cec-a5770acff4b5-istio-token\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:45:31.651609 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:31.651458 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1273c7c-f18f-4459-8cec-a5770acff4b5-credential-socket\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:45:31.651609 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:31.651474 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ng92h\" (UniqueName: \"kubernetes.io/projected/a1273c7c-f18f-4459-8cec-a5770acff4b5-kube-api-access-ng92h\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:45:31.651609 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:31.651487 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1273c7c-f18f-4459-8cec-a5770acff4b5-istio-data\") on node \"ip-10-0-137-244.ec2.internal\" DevicePath \"\"" Jul 08 19:45:31.808879 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:31.808767 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-t4v2b_23b0aacd-a6a4-49a0-8881-77974f6465b1/server/0.log" Jul 08 19:45:31.930258 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:31.930221 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-qzg9x_4e80abce-1fd7-43ca-942a-d23acb4b1f87/manager/0.log" Jul 08 19:45:32.177597 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:32.177573 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-994cc9f69-7vjn8_0c940ac7-574d-41be-b59b-45dfd4db0dba/copy-manifests/0.log" Jul 08 19:45:32.183457 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:32.183419 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-994cc9f69-7vjn8_0c940ac7-574d-41be-b59b-45dfd4db0dba/manager/0.log" Jul 08 19:45:33.175219 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:33.175181 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1273c7c-f18f-4459-8cec-a5770acff4b5" path="/var/lib/kubelet/pods/a1273c7c-f18f-4459-8cec-a5770acff4b5/volumes" Jul 08 19:45:33.619974 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:33.619869 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-664b8dd44f-ffm98_65eee363-b056-4e91-8a78-dc9d141ad46c/authorino/0.log" Jul 08 19:45:33.748644 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:33.748592 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-lk5nn_e1bee84b-23f5-434f-b00b-82d65e2035e5/manager/0.log" Jul 08 19:45:34.093953 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:34.093920 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-xwj24_761c0231-6742-402c-9c83-a0d45f2d8fff/registry-server/0.log" Jul 08 19:45:34.210286 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:34.210246 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-d48fl_8649d14a-1909-4ce9-a380-d35fa273a605/manager/0.log" Jul 08 19:45:34.437671 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:34.437645 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-tn9qn_80f18c7b-fa4a-4389-a961-f92e015fd1f6/manager/0.log" Jul 08 19:45:34.785837 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:34.785765 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf44g28w_cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a/istio-proxy/0.log" Jul 08 19:45:35.023107 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:35.023077 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-685cdb786f-4pcjp_02ef5a7c-9326-42c8-9933-c2e53a151ff2/kube-auth-proxy/0.log" Jul 08 19:45:35.246133 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:35.246093 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-7w2g7_5a885949-6c60-41c0-a333-7e9cdb926d69/istio-proxy/1.log" Jul 08 19:45:36.077568 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:36.077540 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-bnhkc_5a031dfc-2a95-4ee0-91df-e2cb55cc167e/main/0.log" Jul 08 19:45:36.418426 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:36.418398 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-blpg4_50133b47-8d89-4405-b67e-a951717e8710/main/0.log" Jul 08 19:45:40.416664 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:40.416626 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-fqj4d/must-gather-r6mzr"] Jul 08 19:45:40.417062 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:40.417026 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1273c7c-f18f-4459-8cec-a5770acff4b5" containerName="istio-proxy" Jul 08 19:45:40.417062 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:40.417040 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1273c7c-f18f-4459-8cec-a5770acff4b5" containerName="istio-proxy" Jul 08 19:45:40.417062 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:40.417055 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4a6ecf9-6c35-403a-b85f-28666300a263" containerName="istio-proxy" Jul 08 19:45:40.417062 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:40.417060 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4a6ecf9-6c35-403a-b85f-28666300a263" containerName="istio-proxy" Jul 08 19:45:40.417188 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:40.417069 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1273c7c-f18f-4459-8cec-a5770acff4b5" containerName="istio-proxy" Jul 08 19:45:40.417188 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:40.417075 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1273c7c-f18f-4459-8cec-a5770acff4b5" containerName="istio-proxy" Jul 08 19:45:40.417188 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:40.417086 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4a6ecf9-6c35-403a-b85f-28666300a263" containerName="istio-proxy" Jul 08 19:45:40.417188 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:40.417090 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4a6ecf9-6c35-403a-b85f-28666300a263" containerName="istio-proxy" Jul 08 19:45:40.417188 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:40.417108 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2180fb32-89e4-4b70-9605-52dcef09eb6f" containerName="main" Jul 08 19:45:40.417188 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:40.417115 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="2180fb32-89e4-4b70-9605-52dcef09eb6f" containerName="main" Jul 08 19:45:40.417188 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:40.417166 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4a6ecf9-6c35-403a-b85f-28666300a263" containerName="istio-proxy" Jul 08 19:45:40.417188 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:40.417176 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1273c7c-f18f-4459-8cec-a5770acff4b5" containerName="istio-proxy" Jul 08 19:45:40.417188 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:40.417185 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1273c7c-f18f-4459-8cec-a5770acff4b5" containerName="istio-proxy" Jul 08 19:45:40.417472 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:40.417195 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="2180fb32-89e4-4b70-9605-52dcef09eb6f" containerName="main" Jul 08 19:45:40.417472 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:40.417311 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4a6ecf9-6c35-403a-b85f-28666300a263" containerName="istio-proxy" Jul 08 19:45:40.421706 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:40.421687 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fqj4d/must-gather-r6mzr" Jul 08 19:45:40.424542 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:40.424521 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fqj4d\"/\"openshift-service-ca.crt\"" Jul 08 19:45:40.424648 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:40.424560 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-fqj4d\"/\"default-dockercfg-m444m\"" Jul 08 19:45:40.424708 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:40.424657 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fqj4d\"/\"kube-root-ca.crt\"" Jul 08 19:45:40.430815 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:40.430793 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fqj4d/must-gather-r6mzr"] Jul 08 19:45:40.535089 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:40.535053 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/6614845b-be66-4ac8-9d03-95758ec8763b-must-gather-output\") pod \"must-gather-r6mzr\" (UID: \"6614845b-be66-4ac8-9d03-95758ec8763b\") " pod="openshift-must-gather-fqj4d/must-gather-r6mzr" Jul 08 19:45:40.535269 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:40.535139 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtj4w\" (UniqueName: \"kubernetes.io/projected/6614845b-be66-4ac8-9d03-95758ec8763b-kube-api-access-qtj4w\") pod \"must-gather-r6mzr\" (UID: \"6614845b-be66-4ac8-9d03-95758ec8763b\") " pod="openshift-must-gather-fqj4d/must-gather-r6mzr" Jul 08 19:45:40.636516 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:40.636478 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtj4w\" (UniqueName: \"kubernetes.io/projected/6614845b-be66-4ac8-9d03-95758ec8763b-kube-api-access-qtj4w\") pod \"must-gather-r6mzr\" (UID: \"6614845b-be66-4ac8-9d03-95758ec8763b\") " pod="openshift-must-gather-fqj4d/must-gather-r6mzr" Jul 08 19:45:40.636700 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:40.636531 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/6614845b-be66-4ac8-9d03-95758ec8763b-must-gather-output\") pod \"must-gather-r6mzr\" (UID: \"6614845b-be66-4ac8-9d03-95758ec8763b\") " pod="openshift-must-gather-fqj4d/must-gather-r6mzr" Jul 08 19:45:40.636865 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:40.636848 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/6614845b-be66-4ac8-9d03-95758ec8763b-must-gather-output\") pod \"must-gather-r6mzr\" (UID: \"6614845b-be66-4ac8-9d03-95758ec8763b\") " pod="openshift-must-gather-fqj4d/must-gather-r6mzr" Jul 08 19:45:40.645233 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:40.645208 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtj4w\" (UniqueName: \"kubernetes.io/projected/6614845b-be66-4ac8-9d03-95758ec8763b-kube-api-access-qtj4w\") pod \"must-gather-r6mzr\" (UID: \"6614845b-be66-4ac8-9d03-95758ec8763b\") " pod="openshift-must-gather-fqj4d/must-gather-r6mzr" Jul 08 19:45:40.731656 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:40.731556 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fqj4d/must-gather-r6mzr" Jul 08 19:45:40.868299 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:40.868263 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fqj4d/must-gather-r6mzr"] Jul 08 19:45:40.871739 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:45:40.871713 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6614845b_be66_4ac8_9d03_95758ec8763b.slice/crio-9ad9eb898f783e8b33ded532e1232e28ebb8a1995ceb5eb809aaf74b75c8e693 WatchSource:0}: Error finding container 9ad9eb898f783e8b33ded532e1232e28ebb8a1995ceb5eb809aaf74b75c8e693: Status 404 returned error can't find the container with id 9ad9eb898f783e8b33ded532e1232e28ebb8a1995ceb5eb809aaf74b75c8e693 Jul 08 19:45:40.873622 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:40.873598 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 19:45:41.609478 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:41.609414 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fqj4d/must-gather-r6mzr" event={"ID":"6614845b-be66-4ac8-9d03-95758ec8763b","Type":"ContainerStarted","Data":"9ad9eb898f783e8b33ded532e1232e28ebb8a1995ceb5eb809aaf74b75c8e693"} Jul 08 19:45:42.618407 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:42.618367 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fqj4d/must-gather-r6mzr" event={"ID":"6614845b-be66-4ac8-9d03-95758ec8763b","Type":"ContainerStarted","Data":"409e677b550eed02e389779061a5a515358ab488d68ad85759c54a11bd0d390e"} Jul 08 19:45:42.618906 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:42.618421 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fqj4d/must-gather-r6mzr" event={"ID":"6614845b-be66-4ac8-9d03-95758ec8763b","Type":"ContainerStarted","Data":"73769bde5b6074860edfa32d46b1f09f3cf5ddc5d7a440b9d8e5c82b95c6e158"} Jul 08 19:45:42.634985 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:42.634923 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-fqj4d/must-gather-r6mzr" podStartSLOduration=1.94136306 podStartE2EDuration="2.63490116s" totalImagesPullingTime="693.5381ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:45:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:45:42.633372908 +0000 UTC m=+2507.912460530" watchObservedRunningTime="2026-07-08 19:45:42.63490116 +0000 UTC m=+2507.913988781" Jul 08 19:45:43.203144 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:43.203106 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-qx98w_9a902ad1-6161-490d-afb2-006966897633/global-pull-secret-syncer/0.log" Jul 08 19:45:43.338203 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:43.338166 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-n8prt_1033c4ea-5be7-4c7c-9c6c-0bbb663fad60/konnectivity-agent/0.log" Jul 08 19:45:43.430878 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:43.430851 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-244.ec2.internal_09c7821b3b1a8a9c6dd9709ff325da43/haproxy/0.log" Jul 08 19:45:49.027314 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:49.027217 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-664b8dd44f-ffm98_65eee363-b056-4e91-8a78-dc9d141ad46c/authorino/0.log" Jul 08 19:45:49.060202 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:49.060167 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-lk5nn_e1bee84b-23f5-434f-b00b-82d65e2035e5/manager/0.log" Jul 08 19:45:49.176818 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:49.176776 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-xwj24_761c0231-6742-402c-9c83-a0d45f2d8fff/registry-server/0.log" Jul 08 19:45:49.341694 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:49.341603 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-d48fl_8649d14a-1909-4ce9-a380-d35fa273a605/manager/0.log" Jul 08 19:45:49.441618 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:49.441585 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-tn9qn_80f18c7b-fa4a-4389-a961-f92e015fd1f6/manager/0.log" Jul 08 19:45:51.625305 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:51.625259 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vj8cc_54b8b171-50ea-4a73-bc26-5208007a513a/node-exporter/0.log" Jul 08 19:45:51.656751 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:51.656717 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vj8cc_54b8b171-50ea-4a73-bc26-5208007a513a/kube-rbac-proxy/0.log" Jul 08 19:45:51.680176 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:51.680146 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vj8cc_54b8b171-50ea-4a73-bc26-5208007a513a/init-textfile/0.log" Jul 08 19:45:53.100161 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:53.100122 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-fqj4d/perf-node-gather-daemonset-mk5nk"] Jul 08 19:45:53.107577 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:53.107544 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fqj4d/perf-node-gather-daemonset-mk5nk" Jul 08 19:45:53.114540 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:53.113800 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fqj4d/perf-node-gather-daemonset-mk5nk"] Jul 08 19:45:53.270595 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:53.270562 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7dcf2c03-4ca0-449a-9237-8241c6978149-proc\") pod \"perf-node-gather-daemonset-mk5nk\" (UID: \"7dcf2c03-4ca0-449a-9237-8241c6978149\") " pod="openshift-must-gather-fqj4d/perf-node-gather-daemonset-mk5nk" Jul 08 19:45:53.270784 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:53.270680 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7dcf2c03-4ca0-449a-9237-8241c6978149-sys\") pod \"perf-node-gather-daemonset-mk5nk\" (UID: \"7dcf2c03-4ca0-449a-9237-8241c6978149\") " pod="openshift-must-gather-fqj4d/perf-node-gather-daemonset-mk5nk" Jul 08 19:45:53.270784 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:53.270724 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7dcf2c03-4ca0-449a-9237-8241c6978149-podres\") pod \"perf-node-gather-daemonset-mk5nk\" (UID: \"7dcf2c03-4ca0-449a-9237-8241c6978149\") " pod="openshift-must-gather-fqj4d/perf-node-gather-daemonset-mk5nk" Jul 08 19:45:53.270899 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:53.270787 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7dcf2c03-4ca0-449a-9237-8241c6978149-lib-modules\") pod \"perf-node-gather-daemonset-mk5nk\" (UID: \"7dcf2c03-4ca0-449a-9237-8241c6978149\") " pod="openshift-must-gather-fqj4d/perf-node-gather-daemonset-mk5nk" Jul 08 19:45:53.270899 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:53.270861 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7247h\" (UniqueName: \"kubernetes.io/projected/7dcf2c03-4ca0-449a-9237-8241c6978149-kube-api-access-7247h\") pod \"perf-node-gather-daemonset-mk5nk\" (UID: \"7dcf2c03-4ca0-449a-9237-8241c6978149\") " pod="openshift-must-gather-fqj4d/perf-node-gather-daemonset-mk5nk" Jul 08 19:45:53.372001 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:53.371900 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7dcf2c03-4ca0-449a-9237-8241c6978149-lib-modules\") pod \"perf-node-gather-daemonset-mk5nk\" (UID: \"7dcf2c03-4ca0-449a-9237-8241c6978149\") " pod="openshift-must-gather-fqj4d/perf-node-gather-daemonset-mk5nk" Jul 08 19:45:53.372001 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:53.371973 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7247h\" (UniqueName: \"kubernetes.io/projected/7dcf2c03-4ca0-449a-9237-8241c6978149-kube-api-access-7247h\") pod \"perf-node-gather-daemonset-mk5nk\" (UID: \"7dcf2c03-4ca0-449a-9237-8241c6978149\") " pod="openshift-must-gather-fqj4d/perf-node-gather-daemonset-mk5nk" Jul 08 19:45:53.372239 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:53.372021 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7dcf2c03-4ca0-449a-9237-8241c6978149-proc\") pod \"perf-node-gather-daemonset-mk5nk\" (UID: \"7dcf2c03-4ca0-449a-9237-8241c6978149\") " pod="openshift-must-gather-fqj4d/perf-node-gather-daemonset-mk5nk" Jul 08 19:45:53.372239 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:53.372082 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7dcf2c03-4ca0-449a-9237-8241c6978149-sys\") pod \"perf-node-gather-daemonset-mk5nk\" (UID: \"7dcf2c03-4ca0-449a-9237-8241c6978149\") " pod="openshift-must-gather-fqj4d/perf-node-gather-daemonset-mk5nk" Jul 08 19:45:53.372239 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:53.372107 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7dcf2c03-4ca0-449a-9237-8241c6978149-podres\") pod \"perf-node-gather-daemonset-mk5nk\" (UID: \"7dcf2c03-4ca0-449a-9237-8241c6978149\") " pod="openshift-must-gather-fqj4d/perf-node-gather-daemonset-mk5nk" Jul 08 19:45:53.372389 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:53.372288 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7dcf2c03-4ca0-449a-9237-8241c6978149-podres\") pod \"perf-node-gather-daemonset-mk5nk\" (UID: \"7dcf2c03-4ca0-449a-9237-8241c6978149\") " pod="openshift-must-gather-fqj4d/perf-node-gather-daemonset-mk5nk" Jul 08 19:45:53.372546 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:53.372388 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7dcf2c03-4ca0-449a-9237-8241c6978149-lib-modules\") pod \"perf-node-gather-daemonset-mk5nk\" (UID: \"7dcf2c03-4ca0-449a-9237-8241c6978149\") " pod="openshift-must-gather-fqj4d/perf-node-gather-daemonset-mk5nk" Jul 08 19:45:53.372749 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:53.372705 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7dcf2c03-4ca0-449a-9237-8241c6978149-sys\") pod \"perf-node-gather-daemonset-mk5nk\" (UID: \"7dcf2c03-4ca0-449a-9237-8241c6978149\") " pod="openshift-must-gather-fqj4d/perf-node-gather-daemonset-mk5nk" Jul 08 19:45:53.372885 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:53.372870 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7dcf2c03-4ca0-449a-9237-8241c6978149-proc\") pod \"perf-node-gather-daemonset-mk5nk\" (UID: \"7dcf2c03-4ca0-449a-9237-8241c6978149\") " pod="openshift-must-gather-fqj4d/perf-node-gather-daemonset-mk5nk" Jul 08 19:45:53.382138 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:53.382108 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7247h\" (UniqueName: \"kubernetes.io/projected/7dcf2c03-4ca0-449a-9237-8241c6978149-kube-api-access-7247h\") pod \"perf-node-gather-daemonset-mk5nk\" (UID: \"7dcf2c03-4ca0-449a-9237-8241c6978149\") " pod="openshift-must-gather-fqj4d/perf-node-gather-daemonset-mk5nk" Jul 08 19:45:53.426130 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:53.426095 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fqj4d/perf-node-gather-daemonset-mk5nk" Jul 08 19:45:53.624291 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:53.623937 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fqj4d/perf-node-gather-daemonset-mk5nk"] Jul 08 19:45:53.626688 ip-10-0-137-244 kubenswrapper[2583]: W0708 19:45:53.626654 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod7dcf2c03_4ca0_449a_9237_8241c6978149.slice/crio-b542606f8c20832a92768944c8c02efc3f3e84947815f6b98b5109414f230ceb WatchSource:0}: Error finding container b542606f8c20832a92768944c8c02efc3f3e84947815f6b98b5109414f230ceb: Status 404 returned error can't find the container with id b542606f8c20832a92768944c8c02efc3f3e84947815f6b98b5109414f230ceb Jul 08 19:45:53.699382 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:53.699344 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fqj4d/perf-node-gather-daemonset-mk5nk" event={"ID":"7dcf2c03-4ca0-449a-9237-8241c6978149","Type":"ContainerStarted","Data":"b542606f8c20832a92768944c8c02efc3f3e84947815f6b98b5109414f230ceb"} Jul 08 19:45:54.707995 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:54.707946 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fqj4d/perf-node-gather-daemonset-mk5nk" event={"ID":"7dcf2c03-4ca0-449a-9237-8241c6978149","Type":"ContainerStarted","Data":"d5dc54647e52eb48674a7e842a56da30ba93aacceecca72c6a4c605940162b61"} Jul 08 19:45:54.708511 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:54.708338 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-fqj4d/perf-node-gather-daemonset-mk5nk" Jul 08 19:45:54.726994 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:54.726932 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-fqj4d/perf-node-gather-daemonset-mk5nk" podStartSLOduration=1.726913701 podStartE2EDuration="1.726913701s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:45:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:45:54.725137403 +0000 UTC m=+2520.004225099" watchObservedRunningTime="2026-07-08 19:45:54.726913701 +0000 UTC m=+2520.006001321" Jul 08 19:45:55.401246 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:55.401214 2583 scope.go:117] "RemoveContainer" containerID="cddbec3767f1680a339e01b49fbd7424f387783ed86a973030f424e6dced08d8" Jul 08 19:45:56.163370 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:56.163344 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cz9s6_f875980c-42c6-47d0-995b-156a37813723/dns/0.log" Jul 08 19:45:56.182896 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:56.182869 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cz9s6_f875980c-42c6-47d0-995b-156a37813723/kube-rbac-proxy/0.log" Jul 08 19:45:56.334284 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:56.334255 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zjgc5_31a4f57f-1bb0-4b8f-bb58-9a78ef1565c6/dns-node-resolver/0.log" Jul 08 19:45:56.873578 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:56.873539 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-76d95b5874-7rxrt_f19196fd-6a0a-49f7-81b3-8a14637104a0/registry/0.log" Jul 08 19:45:56.937842 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:56.937801 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jw4sj_bb8c1f0b-7584-4d24-aa13-c73cad10576c/node-ca/0.log" Jul 08 19:45:57.856251 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:57.856222 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf44g28w_cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a/istio-proxy/0.log" Jul 08 19:45:58.075355 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:58.075323 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-685cdb786f-4pcjp_02ef5a7c-9326-42c8-9933-c2e53a151ff2/kube-auth-proxy/0.log" Jul 08 19:45:58.134834 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:58.134750 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-7w2g7_5a885949-6c60-41c0-a333-7e9cdb926d69/istio-proxy/0.log" Jul 08 19:45:58.173322 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:58.173287 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-7w2g7_5a885949-6c60-41c0-a333-7e9cdb926d69/istio-proxy/1.log" Jul 08 19:45:59.029781 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:59.029747 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jd5m2_a1ace7c9-8aec-4fb8-992e-e9e2a10b2d77/serve-healthcheck-canary/0.log" Jul 08 19:45:59.576983 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:59.576950 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dkzqj_03c97b5b-1c6e-4461-9fed-e14c43705ddb/kube-rbac-proxy/0.log" Jul 08 19:45:59.599071 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:59.599043 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dkzqj_03c97b5b-1c6e-4461-9fed-e14c43705ddb/exporter/0.log" Jul 08 19:45:59.619419 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:45:59.619392 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dkzqj_03c97b5b-1c6e-4461-9fed-e14c43705ddb/extractor/0.log" Jul 08 19:46:00.728828 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:46:00.728800 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-fqj4d/perf-node-gather-daemonset-mk5nk" Jul 08 19:46:01.960330 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:46:01.960284 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-xx469_7ac398a1-e561-4a15-9217-4ae93da132f9/manager/0.log" Jul 08 19:46:02.153743 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:46:02.153706 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-sjflv_5d6c1088-bfef-4914-8c2e-d444040c89aa/manager/0.log" Jul 08 19:46:02.341188 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:46:02.341154 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-t4v2b_23b0aacd-a6a4-49a0-8881-77974f6465b1/server/0.log" Jul 08 19:46:02.430572 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:46:02.430316 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-qzg9x_4e80abce-1fd7-43ca-942a-d23acb4b1f87/manager/0.log" Jul 08 19:46:02.581970 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:46:02.581916 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-994cc9f69-7vjn8_0c940ac7-574d-41be-b59b-45dfd4db0dba/manager/0.log" Jul 08 19:46:02.606852 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:46:02.606815 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-994cc9f69-7vjn8_0c940ac7-574d-41be-b59b-45dfd4db0dba/copy-manifests/0.log" Jul 08 19:46:04.130499 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:46:04.130463 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf44g28w_cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a/istio-proxy/0.log" Jul 08 19:46:04.333575 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:46:04.333530 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-685cdb786f-4pcjp_02ef5a7c-9326-42c8-9933-c2e53a151ff2/kube-auth-proxy/0.log" Jul 08 19:46:04.383217 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:46:04.383140 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-7w2g7_5a885949-6c60-41c0-a333-7e9cdb926d69/istio-proxy/0.log" Jul 08 19:46:04.426952 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:46:04.426916 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-7w2g7_5a885949-6c60-41c0-a333-7e9cdb926d69/istio-proxy/1.log" Jul 08 19:46:05.676102 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:46:05.676069 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf44g28w_cbd46bf8-62f9-4d3a-96e5-ef0544a2d91a/istio-proxy/0.log" Jul 08 19:46:05.886586 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:46:05.886556 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-685cdb786f-4pcjp_02ef5a7c-9326-42c8-9933-c2e53a151ff2/kube-auth-proxy/0.log" Jul 08 19:46:05.921310 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:46:05.921273 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-7w2g7_5a885949-6c60-41c0-a333-7e9cdb926d69/istio-proxy/0.log" Jul 08 19:46:05.962756 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:46:05.962673 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-7w2g7_5a885949-6c60-41c0-a333-7e9cdb926d69/istio-proxy/1.log" Jul 08 19:46:08.915634 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:46:08.915605 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-mvdjl_26c73a09-c9a6-4f28-b811-c3cd7f0e3fb6/migrator/0.log" Jul 08 19:46:08.935326 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:46:08.935295 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-mvdjl_26c73a09-c9a6-4f28-b811-c3cd7f0e3fb6/graceful-termination/0.log" Jul 08 19:46:10.303006 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:46:10.302971 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-6kmdh_001f0989-80a3-4b9d-8a57-9f41846bef7b/kube-multus/0.log" Jul 08 19:46:10.479237 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:46:10.479207 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mpz48_fb6af782-3262-467f-95e4-31d549573cd6/kube-multus-additional-cni-plugins/0.log" Jul 08 19:46:10.500051 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:46:10.500019 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mpz48_fb6af782-3262-467f-95e4-31d549573cd6/egress-router-binary-copy/0.log" Jul 08 19:46:10.520299 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:46:10.520272 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mpz48_fb6af782-3262-467f-95e4-31d549573cd6/cni-plugins/0.log" Jul 08 19:46:10.540669 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:46:10.540640 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mpz48_fb6af782-3262-467f-95e4-31d549573cd6/bond-cni-plugin/0.log" Jul 08 19:46:10.562462 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:46:10.562356 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mpz48_fb6af782-3262-467f-95e4-31d549573cd6/routeoverride-cni/0.log" Jul 08 19:46:10.583798 ip-10-0-137-244 kubenswrapper[2583]: I0708 19:46:10.583770 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mpz48_fb6af782-3262-467f-95e4-31d549573cd6/whereabouts-cni-bincopy/0.log"